0

I am copying strings containing the word cafe (but with an accented e) from a javascript source file into a python script where I need to do some processing over the data and then output some JSON. I am having some trouble getting my head around the encoding/decoding details though. This is perhaps best illustrated with an example:

$ python
>>> import urllib2, json
>>> the_name = "Tasty Caf%C3%E9"
>>> the_name
'Tasty Caf%C3%E9'
>>> the_name_unquoted = urllib2.unquote(the_name)
>>> the_name_unquoted
'Tasty Caf\xc3\xe9'
>>> json.dumps({'bla': the_name_unquoted})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc3 in position 9: invalid continuation byte

I've spent some time trying to understand how encodings work, though clearly I'm not getting it. Exactly what encoding/format (any other appropriate terminology here?) is the_name_unquoted in above and what is it about it that utf8 cannot decode correctly?

4

1 回答 1

1

那是因为 unicode 编码支持该字符。您可以通过将字符串转换为 unicode 来解决此问题。

the_name = u'Tasty Caf%C3%E9'

或者，如果字符串已经存在，您可以对其进行转换。

the_name = 'Tasty Caf%C3%E9'
the_name = unicode(the_name) 
# or..
the_name = the_name.decode('utf8', the_name)
于 2012-12-20T09:39:04.943 回答