javascript - Javascript 词法分析器/标记器（在 Python 中？）

Question

有谁知道 Javascript 词法分析器或标记器（最好在 Python 中？）

基本上，给定一个任意 Javascript 文件，我想获取令牌。

例如

富 = 1

变成这样：

1. 变量名：“foo”
2. 空白
3. 运算符：等于
4. 空白
5. 整数：1

Answer 1

http://code.google.com/p/pynarcissus/ has one.

Also I made one but it doesn't support automatic semicolon insertion so it is pretty useless for javascript that you have no control over (as almost all real life javascript programs lack at least one semicolon) :) Here is mine:

http://bitbucket.org/santagada/jaspyon/src/tip/jaspyon/

the grammar is in jsgrammar.txt, it is parsed by the PyPy parsing lib (which you will have to download and extract from the pypy source) and it build a parse tree which I walk on astbuilder.py

But if you don't have licensing problems I would go with pynarcissus. heres a direct link to look at the code (ported from narcissus):

http://code.google.com/p/pynarcissus/source/browse/trunk/jsparser.py