2

我正在尝试在 python 字典中使用 cStringIO 读取一个非常大的字符串流：

def stream_read(self, path):
    try:
        # create a string stream from the contents at 'path'
        # note: the string at self._my_dict[path] is 7MB in size
        stream = StringIO.StringIO(self._my_dict[path])
        while True:
            # buffer size is 128kB, or 128 * 1024
            buf = stream.read(self.buffer_size)
            if buf != '':
                yield buf
            else:
                raise StopIteration
    except KeyError:
        raise IOError("Could not get content")

在我的测试套件中，我通过首先测试stream_write来测试这个函数，断言数据存在于该路径中，然后调用stream_read：

def test_stream(self):
    filename = self.gen_random_string()
    # test 7MB
    content = self.gen_random_string(7 * 1024 * 1024)
    # test stream write
    io = StringIO.StringIO(content)
    self._storage.stream_write(filename, io)
    io.close()
    self.assertTrue(self._storage.exists(filename))
    # test read / write
    data = ''
    for buf in self._storage.stream_read(filename):
        data += buf
    self.assertEqual(content, data)

然而在我的测试套件中，我遇到了一个 AssertionError：

======================================================================
FAIL: test_stream (test_swift_storage.TestSwiftStorage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bacongobbler/.../test/test_local_storage.py", line 44, in test_stream
    self.assertEqual(content, data)
AssertionError: '[squelched]' != '<cStringIO.StringI object at 0x3148e70>'
----------------------------------------------------------------------
Ran 28 tests in 20.495s

FAILED (failures=1)

它看起来与我上周发布的问题有关，但我仍然不太确定我明白为什么stream在这种情况下将生成器设置为字符串。

如果有人想仔细查看源代码，请查看https://github.com/bacongobbler/docker-registry/blob/106-swift-storage/test/utils/mock_swift_storage.py

4

1 回答 1

2

调用时只存储对象：StringIOself._storage.stream_write(filename, io)

def put_content(self, path, content, chunk=None):
    path = self._init_path(path)
    try:
        self._swift_container[path] = content
    except Exception:
        raise IOError("Could not put content")

你传入content的对象在哪里。io

稍后，您再次将该文件对象传递给StringIO：

stream = StringIO.StringIO(self.get_content(path))

这调用str()，self.get_content(path)存储cStringIO.StringI()实例的字符串表示：

>>> from cStringIO import StringIO
>>> str(StringIO('test data'))
'<cStringIO.StringI object at 0x1074ea470>'

您的阅读代码工作正常，是您的编写模拟需要实际从对象中取出数据。StringIO

一个.read()电话会在这里做：

def put_content(self, path, content, chunk=None):
    path = self._init_path(path)
    try:
        self._swift_container[path] = content.read()
    except Exception:
        raise IOError("Could not put content")
于 2013-12-17T23:38:23.503 回答