0

我正在尝试创建一个读取表达式的标记器。当它获得令牌时，我打印类型和词位，它是运算符的数字。它目前单独打印每个字符，但我需要它一起打印两位数和十进制数。这是我的代码。

    public class Tokenizer {
String type;
String lexeme;
/**
 * @param args the command line arguments
 */
void getToken(String expression, int i) {

        if (expression.charAt(i) == '+')
        {
            type = "operator";
            lexeme = "+";       
        }
        else if (expression.charAt(i) == '-')
        {
            type = "operator";
            lexeme = "-";
        }
        else if (expression.charAt(i) == '*')
        {
            type = "operator";
            lexeme = "*";
        }
        else if (expression.charAt(i) == '/')
        {
            type = "operator";
            lexeme = "/";
        }
        else if (expression.charAt(i) == ' ')
        {
            ; //empty statement. Does nothing if the character is white space
        }
        else 
        {
            type = "number";
            lexeme = Character.toString(expression.charAt(i));
        }
}
public static void main(String[] args) {
    // TODO code application logic here
    String expression = "3+66*2.5";
    Tokenizer tokenizer = new Tokenizer();
    for (int i = 0; i < expression.length(); i++)
    {
        tokenizer.getToken(expression, i);
        System.out.print("Type: " + tokenizer.type + "\t" + "Lexeme: " + tokenizer.lexeme + "\n");
    }
  }
}

样本输出 - 实际

Type: number    Lexeme: 3
Type: operator  Lexeme: +
Type: number    Lexeme: 6
Type: number    Lexeme: 6
Type: operator  Lexeme: *
Type: number    Lexeme: 2
Type: number    Lexeme: .
Type: number    Lexeme: 5

样本输出 - 预期

Type: number    Lexeme: 3
Type: operator  Lexeme: +
Type: number    Lexeme: 66
Type: operator  Lexeme: *
Type: number    Lexeme: 2.5
4

1 回答 1

3

此语句在每次执行时覆盖词位

 lexeme = Character.toString(expression.charAt(i));

它应该是

lexeme += Character.toString(expression.charAt(i));

即在识别单个数字后，将其添加到词位。

除此之外，您的getToken方法不正确，然后通过传递i给它您假设一个词位只有一个字符长，实际上它可以是 1 个或更多，例如66or 2.5。所以你需要重新设计你的Tokenizer班级。您应该使用表达式初始化 Tokenizer 对象，然后让它有一个无参数方法getNextToken()，该方法根据需要进行尽可能多的迭代来识别令牌，然后返回它。它还应该有一种方法来检查是否还有任何输入，例如hasMoreTokens().

这是您可以建立的东西

public class Tokenizer {
    int pos;
    char[] expression;

    Tokenizer(String expression) {
        this.expression = expression.toCharArray();
        this.pos = 0;
    }

    enum Type { OPERATOR, NUMBER, UNKNOWN }

    class Lexeme {
        String type, token;
        Lexeme(String type, String token) {
            this.type = type;
            this.token = token;
        }
    }

    Lexeme getNextToken() {
        StringBuilder token = new StringBuilder();
        boolean endOfToken = false;
        Type type = Type.UNKNOWN;
        while (!endOfToken && hasMoreTokens()) {
            while(expression[pos] == ' ' && hasMoreTokens())
                pos++;
            switch (expression[pos]) {
                case '+':
                case '-':
                case '*':
                case '/':
                    if(type != Type.NUMBER) {
                        type = Type.OPERATOR;
                        token.append(expression[pos]);
                        pos++;
                    }
                    endOfToken = true;
                    break;
                case ' ':
                    endOfToken = true;
                    pos++;
                    break;
                default:
                    if(Character.isDigit(expression[pos]) || expression[pos] == '.') {
                        token.append(expression[pos]);
                        type = Type.NUMBER;
                    } else {
                        System.out.println("Systax error at position: " + pos);
                    }
                    pos++;
                    break;
            }
        }
        return new Lexeme(type.name().toLowerCase(), token.toString());
    }

    boolean hasMoreTokens() {
        return pos < expression.length;
    }

    public static void main(String[] args) {
        String expression = "3+66*2.5";
        Tokenizer tokenizer = new Tokenizer(expression);
        while (tokenizer.hasMoreTokens()) {
            Lexeme nextToken = tokenizer.getNextToken();
            System.out.print("Type: " + nextToken.type + "\tLexeme: " + nextToken.token + "\n");
        }
    }
}

输出

Type: number    Lexeme: 3
Type: operator  Lexeme: +
Type: number    Lexeme: 66
Type: operator  Lexeme: *
Type: number    Lexeme: 2.5
于 2013-09-08T19:35:54.040 回答