我正在工作,我需要用 Beautiful Soup 解析一个网站。该站点是http://www.manta.com但是当我尝试在 HTML 代码的元数据中查看站点的编码时,什么也没有出现。我尝试在本地解析 HTML,下载网页,但遇到一些解码错误:
# manta web page downloaded before
html = open('1.html', 'r')
soup = BeautifulSoup(html, 'lxml')
这会产生以下堆栈跟踪:
Traceback (most recent call last):
File "E:/Projects/Python/webkit/sample.py", line 10, in <module>
soup = BeautifulSoup(html, 'lxml')
File "C:\Python27\lib\site-packages\bs4\__init__.py", line 172, in __init__
self._feed()
File "C:\Python27\lib\site-packages\bs4\__init__.py", line 185, in _feed
self.builder.feed(self.markup)
File "C:\Python27\lib\site-packages\bs4\builder\_lxml.py", line 195, in feed
self.parser.close()
File "parser.pxi", line 1209, in
lxml.etree._FeedParser.close(src\lxm\lxml.etree.c:90717)
File "parsertarget.pxi", line 142, in
lxml.etree._TargetParserContext._handleParseResult (src\lxml\lxml.etree.c:100104)
File "parsertarget.pxi", line 130, in
lxml.etree._TargetParserContext._handleParseResult (src\lxml\lxml.etree.c:99927)
File "lxml.etree.pyx", line 294, in lxml.etree._ExceptionContext._raise_if_stored
(src\lxml\lxml.etree.c:9387)
File "saxparser.pxi", line 259, in lxml.etree._handleSaxData (src\lxml
\lxml.etree.c:96065)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 105-106: invalid data
我尝试在 Beautiful Soup 的构造函数中引入编码:
soup = BeautifulSoup(html, 'lxml', from_encoding= "some encoding")
我继续得到同样的错误。
有趣的是,如果我在浏览器中加载页面,然后我将编码更改为 utf-8,例如在 Firefox 中并保存它。这项工作很好。非常感谢任何帮助。谢谢你。