7

我正在为一个过时的文本编辑器的脚本语言实现一个解释器，我在让词法分析器正常工作时遇到了一些麻烦。

这是该语言有问题的部分的示例：

T
L /LOCATE ME/
C /LOCATE ME/CHANGED ME/ * *
C ;CHANGED ME;CHANGED ME AGAIN; 1 *

这些/字符似乎引用了字符串，并且还充当 -type 语法中C( CHANGE) 命令的分隔符sed，尽管它允许任何字符作为分隔符。

parse_tokens(line.split())到目前为止，我可能已经实现了大约一半最常见的命令。这又快又脏，但效果却出奇的好。

为了避免编写我自己的词法分析器，我尝试了shlex.

它工作得很好，除了以下CHANGE情况：

import shlex

def shlex_test(cmd_str):
    lex = shlex.shlex(cmd_str)
    lex.quotes = '/'
    return list(lex)

print(shlex_test('L /spaced string/'))
# OK! gives: ['L', '/spaced string/']

print(shlex_test('C /spaced string/another string/ * *'))
# gives   : ['C', '/spaced string/', 'another', 'string/', '*', '*']
# desired : any format that doesn't split on a space between /'s

print(shlex_test('C ;a b;b a;'))
# gives   : ['C', ';', 'b', 'a', ';', 'a', 'b', ';']
# desired : same format as CHANGE command above

任何人都知道一个简单的方法来完成这个（用shlex或其他）？

编辑：

如果有帮助，这里是CHANGE帮助文件中给出的命令语法：

'''
C [/stg1/stg2/ [n|n m]]

    The CHANGE command replaces the m-th occurrence of "stg1" with "stg2"
for the next n lines.  The default value for m and n is 1.'''

同样难以标记X和Y命令：

'''
X [/command/[command/[...]]n]
Y [/command/[command/[...]]n]

    The X and Y commands allow the execution of several commands contained
in one command.  To define an X or Y "command string", enter X (or Y)
followed by a space, then individual commands, each separated by a
delimiter (e.g. a period ".").  An unlimited number of commands may be
placed in the X or Y command string.  Once the command string has been
defined, entering X (or Y) followed optionally by a count n will execute
the defined command string n times.  If n is not specified, it will
default to 1.'''
4

1 回答 1

0

问题可能/是不代表引号，而仅代表分隔。我猜第三个字符总是用来定义分隔符。此外，您不需要输出中的/or ;，对吗？

我刚刚对 L 和 C 命令案例进行了拆分，仅执行了以下操作：

>>> def parse(cmd):
...     delim = cmd[2]
...     return cmd.split(delim)
...
>>> c_cmd = "C /LOCATE ME/CHANGED ME/ * *"
>>> parse(c_cmd)
['C ', 'LOCATE ME', 'CHANGED ME', ' * *']

>>> c_cmd2 = "C ;a b;b a;"
>>> parse(c_cmd2)
['C ', 'a b', 'b a', '']

>>> l_cmd = "L /spaced string/"
>>> parse(l_cmd)
['L ', 'spaced string', '']

对于" * *"您可以split(" ")在最后一个列表元素上使用的可选部分。

>>> parse(c_cmd)[-1].split(" ")
['', '*', '*']
于 2012-07-19T21:19:08.630 回答