python - 出现错误“未找到stdbuf；与perl 的通信可能因stdio 缓冲而挂起。” 当使用 MosesTokenizer 和 Python 时

Question

在 Windows 平台上使用 mosestokenizer （https://github.com/luismsgomes/mosestokenizer，通过 pypi https://pypi.org/project/mosestokenizer/安装）时出现问题（尝试了多台 Windows 机器）。

尽管上个月没有新版本发布，但之前它运行良好。我相信这与python和perl之间的通信有关。

>> MosesSentenceSplitter('en')
stdbuf was not found; communication with perl may hang due to stdio buffering.
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "...\.venv\lib\site-packages\mosestokenizer\sentsplitter.py",
    line 82, in __init__
        super().__init__(argv)
    File "...\.venv\lib\site-packages\toolwrapper.py",
    line 64, in __init__
        self.start()
    File "...\.venv\lib\site-packages\toolwrapper.py",
    line 102, in start
    self.proc = subprocess.Popen(
    File "C:\Users\MyUser\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py",
    line 854, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
    File "C:\Users\MyUser\AppData\Local\Programs\Python\Python38-32\lib\subprocess.py",
    line 1307, in _execute_child
        hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

为了重现它，只需要创建一些 moses* 类对象：

from mosestokenizer import MosesSentenceSplitter
splitsents = MosesSentenceSplitter('en')  # and here error appear.

Answer 1

安装 perl 修复了错误。

另外，正如@ikegami 提到的，这应该有$| = 1，否则你会收到stdbuf 警告。