python - UnicodeDecodeError：“utf-8”编解码器无法解码位置 434852 中的字节 0xe2：无效的继续字节

Question

我正在使用 hfcca 计算 C++ 代码的圈复杂度。hfcca 是一个简单的 Python 脚本（https://code.google.com/p/headerfile-free-cyclomatic-complexity-analyzer/）。当我尝试运行脚本以生成 xml 文件形式的输出时，出现以下错误：

Traceback (most recent call last):
    "./hfcca.py", line 802, in <module>
    main(sys.argv[1:])
    File "./hfcca.py", line 798, in main
    print(xml_output([f for f in r], options))
    File "./hfcca.py", line 798, in <listcomp>
    print(xml_output([f for f in r], options))
    File "/x/home06/smanchukonda/PREFIX/lib/python3.3/multiprocessing/pool.py", line 652, in next
    raise value
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 434852: invalid continuation byte

请在这件事上给予我帮助..

Answer 1

The problem looks like the file has characters represented with latin1 that aren't characters in utf8. The file utility can be useful for figuring out what encoding a file should be treated as, e.g:

monk@monk-VirtualBox:~$ file foo.txt 
foo.txt: UTF-8 Unicode text

Here's what the bytes mean in latin1:

>>> b'\xe2'.decode('latin1')
'â'

Probably easiest is to convert the files to utf8.

Answer 2

我在渲染 Markup("""yyyyyy""") 时也遇到了同样的问题，但我使用在线工具解决了这个问题，删除了“坏”字符。https://pteo.paranoiaworks.mobi/diacriticsremover/

这是一个不错的工具，甚至可以离线使用。