我有一个相当大的 Marpa 语法(用于解析 XPath),并且遇到了标记化问题。我在下面创建了一个最小的破坏示例:
use strict;
use warnings;
use Marpa::R2;
my $grammar = Marpa::R2::Scanless::G->new(
{
source => \(<<'END_OF_SOURCE'),
:default ::= action => ::array
:start ::= Start
Start ::= Child DoubleColon Token
DoubleColon ~ '::'
Child ~ 'child'
Token ~
word
| word ':' word
word ~ [\w]+
END_OF_SOURCE
}
);
my $reader = Marpa::R2::Scanless::R->new(
{
grammar => $grammar,
trace_terminals => 1,
}
);
my $input = 'child::book';
$reader->read(\$input);
此脚本打印以下内容:
Registering character U+0063 as symbol 10: [[\w]]
Registering character U+0063 as symbol 3: [[c]]
Registering character U+0068 as symbol 10: [[\w]]
Registering character U+0068 as symbol 4: [[h]]
Registering character U+0069 as symbol 10: [[\w]]
Registering character U+0069 as symbol 5: [[i]]
Registering character U+006c as symbol 10: [[\w]]
Registering character U+006c as symbol 6: [[l]]
Registering character U+0064 as symbol 10: [[\w]]
Registering character U+0064 as symbol 7: [[d]]
Registering character U+003a as symbol 1: [[\:]]
Rejected lexeme @0-5: Token; value="child"
Accepted lexeme @0-5: Child; value="child"
Registering character U+0062 as symbol 10: [[\w]]
Error in SLIF G1 read: No lexeme found at position 6
* String before error: child::
* The error was at line 1, column 8, and at character 0x0062 'b', ...
* here: book
我希望将输入标记为[Child] [DoubleColon] [word]
. 如终端跟踪所示,仅读取和处理一个冒号字符。似乎它试图将字符串的开头标记为[word] [':'] [word]
并在中途失败。| word ':' word
如果删除语法 ( )的第 10 行,将不再抛出该错误。
我尝试为 DoubleColon ( ) 创建优先级:lexeme ~ <DoubleColon> priority > 1
,但没有奏效。有人能告诉我该怎么做才能使这个语法正确解析输入字符串吗?它仍然需要能够解析child::ns:book
等。