0

I want to speed up my program written in Go and convert regular expressions to finite state machines with ragel. I cannot figure out how to match end of input correctly when converting regular expressions similar to /cat[s]?(\b|$)/ (it matches a word border or end of input), so I made this workaround:

package main

import(
  "strings"
  "fmt"
  "unicode"
)

func Match(data []byte) bool {
  data = []byte(strings.ToLower(string(data)))

  %%{
    machine test;
    write data;
  }%%

  cs, p, pe, eof := 0, 0, len(data), len(data)
  _ = eof

  var matchPos int

  %%{
    main := ('cat' 's'?) @{matchPos = p};

    write init;
    write exec;
  }%%

  return checkMatch(data, matchPos+1)
}

func checkMatch(data []byte, p int) bool {
  if p == len(data) {
    return true
  }
  tail := string(data[p:])
  c := []rune(tail)[0]
  if !unicode.IsLetter(c) && !unicode.IsDigit(c) {
    return true
  }
  return false
}

func main () {
  vs := []string{
    "cat",
    "cats",
    "cat and dog",
    "cats and dogs",
    "caterpillar",
  }
  for _, v := range vs {
    fmt.Printf("'%s': %v\n", v, Match([]byte(v)))
  }
}

Finite State Machine Graph

the output is correct:

'cat': true
'cats': true
'cat and dog': true
'cats and dogs': true
'caterpillar': false

I do think there is a better way though. What is a "proper" way to handle end of input with ragel?

4

1 回答 1

1

当然，处理输入结束的正确方法是使用 EOF 操作。并使用一般的动作，像这样（简化Match功能）：

  var matched bool

  %%{
    action setMatched {
      matched = true
    }

    main := ('cat' 's'?) %/setMatched ([ \t] >setMatched);

    write init;
    write exec;
  }%%
  // Variable generated and not used by Ragel.
  _ = _test_trans_actions

  return matched

这会产生以下输出（注意添加了一个重要的测试用例）：

'cat': true
'cats': true
'cat and dog': true
'cats and dogs': true
'catspaw': false
'caterpillar': false

像这样工作： 在此处输入图像描述

它添加的是由 EOF 在第一台机器 ( ) 的setMatched一个最终状态 ( ) 中触发的操作，或者在进入 ( ) 第二个状态时触发的操作（几乎是，但实际上可以用内部机器替换） . 它完全消除了对的需要。%/setMatchedcats?>setMatched\bspacecheckMatch

于 2020-01-10T20:22:03.630 回答