1

我有这个脚本，它从网页中读取文本：

page = urllib2.urlopen(url).read()
soup = BeautifulSoup(page);
paragraphs = soup.findAll('p');

for p in paragraphs:
    content = content+p.text+" ";

在网页中我有这个字符串：

Möddinghofe

我的脚本将其读取为：

Möddinghofe

我怎样才能按原样阅读它？

4

2 回答 2

1

希望这会对你有所帮助

from BeautifulSoup import BeautifulStoneSoup
import cgi

def HTMLEntitiesToUnicode(text):
    """Converts HTML entities to unicode.  For example '&' becomes '&'."""
    text = unicode(BeautifulStoneSoup(text, convertEntities=BeautifulStoneSoup.ALL_ENTITIES))
    return text

def unicodeToHTMLEntities(text):
    """Converts unicode to HTML entities.  For example '&' becomes '&'."""
    text = cgi.escape(text).encode('ascii', 'xmlcharrefreplace')
    return text

text = "&, ®, <, >, ¢, £, ¥, €, §, ©"

uni = HTMLEntitiesToUnicode(text)
htmlent = unicodeToHTMLEntities(uni)

print uni
print htmlent
# &, ®, <, >, ¢, £, ¥, €, §, ©
# &amp;, &#174;, &lt;, &gt;, &#162;, &#163;, &#165;, &#8364;, &#167;, &#169;

参考：将 HTML 实体转换为 Unicode，反之亦然

于 2012-05-14T18:16:22.417 回答
0

我建议你看一下 BeautifulSoup 文档的编码部分。

于 2012-05-14T18:25:47.247 回答