1

我正在使用BreakIteratorJava 中的实现从字符串中删除标点符号。我需要在 Scala 中重写它，所以我认为这可能是一个用更好的库替换它的好机会（我的实现非常天真，我确信它在边缘情况下会失败）。

是否存在任何可能使用的此类库？

编辑：这是我在 Scala 中的快速解决方案：

  private val getWordsFromLine = (line: String) => {
    line.split(" ")
        .map(_.toLowerCase())
        .map(word => word.filter(Character.isLetter(_)))
        .filter(_.length() > 1)
        .toList
  }

考虑到这一点List[String]（每行一个......是的......这就是圣经 - 它是很好的测试用例）：

摩西的第二本书，称为出埃及记

第 1 章 1 这些是进入埃及的以色列人的名字。每个人和他的家人都和雅各一起来。2 流便、西缅、利未和犹大， 3 以萨迦、西布伦和便雅悯， 4 但、拿弗他利、迦得和亚设。

你会得到List[String]这样的：

List(the, second, book, of, moses, called, exodus, chapter, now, these, are, the, names, of, the, children, of, israel, which, came, into, egypt, every, man, and, his, household, came, with, jacob, reuben, simeon, levi, and, judah, issachar, zebulun, and, benjamin, dan, and, naphtali, gad, and, asher)
4

2 回答 2

2

对于这种特殊情况，我会使用正则表达式。

def toWords(lines: List[String]) = lines flatMap { line =>
  "[a-zA-Z]+".r findAllIn line map (_.toLowerCase)
}
于 2012-08-08T18:35:43.087 回答
0

这是一种使用正则表达式的方法。不过，它还没有过滤单字符单词。

val s = """
THE SECOND BOOK OF MOSES, CALLED EXODUS

CHAPTER 1 1 Now these [are] the names of the children of Israel,
which came into Egypt; every man and his household came with
Jacob. 2 Reuben, Simeon, Levi, and Judah, 3 Issachar, Zebulun,
and Benjamin, 4 Dan, and Naphtali, Gad, and Asher.
"""

/* \p{L} denotes Unicode letters */
var items = """\b\p{L}+\b""".r findAllIn s

println(items.toList)
  /* List(THE, SECOND, BOOK, OF, MOSES, CALLED, EXODUS,
          CHAPTER, Now, these, are, the, names, of, the,
          children, of, Israel, which, came, into, Egypt,
          every, man, and, his, household, came, with,
          Jacob, Reuben, Simeon, Levi, and, Judah,
          Issachar, Zebulun, and, Benjamin, Dan, and,
          Naphtali, Gad, and, Asher)
  */

/* \w denotes word characters */
items = """\b\w+\b""".r findAllIn s
println(items.toList)
  /* List(THE, SECOND, BOOK, OF, MOSES, CALLED, EXODUS,
          CHAPTER, 1, 1, Now, these, are, the, names, of,
          the, children, of, Israel, which, came, into,
          Egypt, every, man, and, his, household, came,
          with, Jacob, 2, Reuben, Simeon, Levi, and, Judah,
          3, Issachar, Zebulun, and, Benjamin, 4, Dan, and,
          Naphtali, Gad, and, Asher)
  */

这里\b描述了单词边界，正则表达式的 Javadoc 在这里。

于 2012-08-08T13:10:47.377 回答