0

我在理解 python 2.7.2 中的 unicode 时遇到问题,所以我在空闲时尝试了一些测试。有两件事被标记为“不确定”。请告诉我为什么他们失败了。至于其他项目,请告诉我我的评论是否准确。

>>> s
'Don\x92t '  # s is a string
>>> u
u'Don\u2019t '  # u is a unicode object
>>> type(u)     # confirm u is unicode
<type 'unicode'>
>>> type(s)     # confirm s is string
<type 'str'>
>>> type(s) == 'str' # wrong way to test
False
>>> isinstance(s, str)  # right way to test
True
>>> print s
Don’t       # works because idle can handle strings
>>> print u
Don’t       # works because idle can handle unicode
>>> open('9', 'w').write(s.encode('utf8')) #encode takes unicode, but s is a string,
                                            # so this fails
Traceback (most recent call last):
  File "<pyshell#28>", line 1, in <module>
    open('9', 'w').write(s.encode('utf8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 3: ordinal not in range(128)
>>> open('9', 'w').write(s) # write can write strings
>>> open('9', 'w').write(u) # write can't write unicode

Traceback (most recent call last):
  File "<pyshell#30>", line 1, in <module>
    open('9', 'w').write(u)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 3: ordinal not in range(128)
>>> open('9', 'w').write(u.encode('utf8'))  # encode turns unicode to string, which write can handle
>>> open('9', 'w').write(s.decode('utf8'))  # decode turns string to unicode, which write can't handle

Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
    open('9', 'w').write(s.decode('utf8'))
  File "C:\program files\Python27\lib\encodings\utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 3: invalid start byte
>>> e = '{}, {}'.format(s, u) # fails becase ''.format is string, while u is unicode

Traceback (most recent call last):
  File "<pyshell#33>", line 1, in <module>
    e = '{}, {}'.format(s, u)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 3: ordinal not in range(128)
>>> e = '{}, {}'.format(s, u.encode('utf8')) # works because u.encode is a string
>>> e = u'{}, {}'.format(s, u) # not sure

Traceback (most recent call last):
  File "<pyshell#36>", line 1, in <module>
    e = u'{}, {}'.format(s, u)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 3: ordinal not in range(128)
>>> e = u'{}, {}'.format(s.decode('utf8'), u) # not sure

Traceback (most recent call last):
  File "<pyshell#55>", line 1, in <module>
    e = u'{}, {}'.format(s.decode('utf8'), u)
  File "C:\program files\Python27\lib\encodings\utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 3: invalid start byte

>>> e = '\n'.join([s, u]) # wants strings, but u is unicode

Traceback (most recent call last):
  File "<pyshell#37>", line 1, in <module>
    e = '\n'.join([s, u])
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 3: ordinal not in range(128)
>>> e = '\n'.join([s, u.encode('utf8')]) # u.encode is now a string
4

2 回答 2

2

首先s不是utf-8编码字符串,它可能是cp1250编码字符串。所以使用解码utf-8总是失败。

>>> e = u'{}, {}'.format(s, u) # not sure

第一个“不确定”是因为u'{}, {}'isunicode并试图将format函数的每个参数编码为unicode字符串。但是因为它不知道s编码的内容,所以它假设s编码为ascii,因此它尝试将其解码为ascii(基本上是在做s.decode('ascii'))并且失败,因为s它是一个cp1250编码的字符串。

>>> e = u'{}, {}'.format(s.decode('utf8'), u) # not sure

第二个失败,因为您尝试将其解码为utf-8但实际上,如前所述,它采用与utf-8.

于 2013-08-14T14:17:17.197 回答
1
于 2013-08-14T14:43:56.033 回答