3

早期的代码给了我这个网址：http ://en.wikipedia.org/wiki/M%C3%BCnster 。现在，我想请求它，但想不出办法：

>>> requests.get('http://en.wikipedia.org/wiki/M%C3%BCnster')
<Response [400]>
>>> requests.get(urlparse.unquote('http://en.wikipedia.org/wiki/M%C3%BCnster'))
<Response [400]>
>>> requests.get(urlparse.unquote('http://en.wikipedia.org/wiki/M%C3%BCnster').decode('utf-8'))
<Response [400]>

问题是 requests 试图对引用过于聪明，实际上要求：

Request URI: /wiki/M%25C3%25BCnster
Request URI: /wiki/M%25C3%25BCnster
Request URI: /wiki/M%25C3%25BCnster

有任何想法吗？

4

3 回答 3

2

带有自定义 User-Agent 标头的简单 urlparse.unquote 似乎可以完成这项工作。

>>> s = 'http://en.wikipedia.org/wiki/M%C3%BCnster'
>>> import urllib2, urlparse
>>> headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.2; rv:9.0.1) Gecko/20100101 Firefox/9.0.1'}
>>> url = urlparse.unquote(s)
>>> req = urllib2.Request(url, None, headers)
>>> resp = urllib2.urlopen(req)
>>> print resp.code
200
>>> data = resp.read()
>>> print 'The last outstanding palace of the German baroque period is created according to plans by Johann Conrad Schlaun.' in data
True

不要将字节字符串解码为 un​​icode 对象，它会导致UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 11: ordinal not in range(128)urlopen。

于 2012-02-14T01:01:14.400 回答
2

这是请求中的错误。它已经固定在develop分支中。请参阅：https ://github.com/kennethreitz/requests/pull/387 。

于 2012-02-14T10:31:44.263 回答
1

尝试添加.decode('utf-8')：

requests.get(urlparse.unquote('http://en.wikipedia.org/wiki/M%C3%BCnster').decode('utf-8'))
于 2012-02-13T21:12:03.987 回答