2

我正在尝试测试一些使用 urllib2 和 lxml 的 python 代码。

我看过几篇博客文章和堆栈溢出文章，人们希望使用 urllib2 测试抛出的异常。我还没有看到测试成功调用的示例。

我走的是正确的道路吗？

有没有人建议让它工作？

这是我到目前为止所拥有的：

import mox
import urllib
import urllib2
import socket
from lxml import etree

# set up the test
m = mox.Mox()
response = m.CreateMock(urllib.addinfourl)
response.fp = m.CreateMock(socket._fileobject)
response.name = None # Needed because the file name is checked.
response.fp.read().AndReturn("""<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>""")
response.geturl().AndReturn("http://rss.slashdot.org/Slashdot/slashdot")
response.read = response.fp.read # Needed since __init__ is not called on addinfourl.
m.StubOutWithMock(urllib2, 'urlopen')
urllib2.urlopen(mox.IgnoreArg(), timeout=10).AndReturn(response)
m.ReplayAll()

# code under test
response2 = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10)
# Note: response2.fp.read() and response2.read() do not behave the same, as defined above.
# In [21]: response2.fp.read()
# Out[21]: '<?xml version="1.0" encoding="utf-8"?>\n<foo>bar</foo>'
# In [22]: response2.read()
# Out[22]: <mox.MockMethod object at 0x97f326c>
xcontent = etree.parse(response2)

# verify test
m.VerifyAll()

它失败了：

Traceback (most recent call last):
  File "/home/jon/mox_question.py", line 22, in <module>
    xcontent = etree.parse(response2)
  File "lxml.etree.pyx", line 2583, in lxml.etree.parse (src/lxml/lxml.etree.c:25057)
  File "parser.pxi", line 1487, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:63708)
  File "parser.pxi", line 1517, in lxml.etree._parseFilelikeDocument (src/lxml/lxml.etree.c:63999)
  File "parser.pxi", line 1400, in lxml.etree._parseDocFromFilelike (src/lxml/lxml.etree.c:62985)
  File "parser.pxi", line 990, in lxml.etree._BaseParser._parseDocFromFilelike (src/lxml/lxml.etree.c:60508)
  File "parser.pxi", line 542, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:56659)
  File "parser.pxi", line 624, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:57472)
  File "lxml.etree.pyx", line 235, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:6222)
  File "parser.pxi", line 371, in lxml.etree.copyToBuffer (src/lxml/lxml.etree.c:55252)
TypeError: reading from file-like objects must return byte strings or unicode strings

这是因为 response.read() 没有返回我期望它返回的内容。

4

3 回答 3

4

我根本不会深入研究 urllib2 的内部结构。这超出了我认为的你关心的范围。这是使用 StringIO 的一种简单方法。这里的关键是，您打算解析为 XML 的内容只需要在鸭子类型方面类似于文件，它不需要是实际的 addinfourl 实例。

import StringIO
import mox
import urllib2
from lxml import etree

# set up the test
m = mox.Mox()
response = StringIO.StringIO("""<?xml version="1.0" encoding="utf-8"?>
<foo>bar</foo>""")
m.StubOutWithMock(urllib2, 'urlopen')
urllib2.urlopen(mox.IgnoreArg(), timeout=10).AndReturn(response)
m.ReplayAll()

# code under test
response2 = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10)
xcontent = etree.parse(response2)

# verify test
m.VerifyAll()
于 2010-06-30T03:28:23.797 回答
2

与 Peter 所说的相呼应，我只想补充一点，您可能不需要关心 lxml 的内部结构，而不是 urllib2 的内部结构。通过模拟 lxml.etree，您可以完全隔离您真正需要测试的代码，您自己的。这是一个执行此操作的示例，还展示了如何使用模拟对象来测试 response.getcode() 调用。

import mox
from lxml import etree
import urllib2

class TestRssDownload(mox.MoxTestBase):

    def test_rss_download(self):
        expected_response = self.mox.CreateMockAnything()
        self.mox.StubOutWithMock(urllib2, 'urlopen')
        self.mox.StubOutWithMock(etree, 'parse')
        self.mox.StubOutWithMock(etree, 'iterwalk')
        title_elem = self.mox.CreateMock(etree._Element)
        title_elem.text = 'some title'

        # Set expectations 
        urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10).AndReturn(expected_response)
        expected_response.getcode().AndReturn(200)
        etree.parse(expected_response).AndReturn('some parsed content')
        etree.iterwalk('some parsed content', tag='{http://purl.org/rss/1.0/}title').AndReturn([('end', title_elem),])

        # Code under test
        self.mox.ReplayAll()
        self.production_code()

    def production_code(self):
        response = urllib2.urlopen("http://rss.slashdot.org/Slashdot/slashdot", timeout=10)
        response_code = response.getcode()
        if 200 != response_code:
            raise Exception('Houston, we have a problem ({0})'.format(response_code))
        tree = etree.parse(response)
        for ev, elem in etree.iterwalk(tree, tag='{http://purl.org/rss/1.0/}title'):
            # Do something with elem.text
            print('{0}: {1}'.format(ev, elem.text))
于 2011-12-18T01:14:46.927 回答
0

看起来您的失败根本与 mox 无关 - 导致错误的行是从 response2 读取的，这是对 slashdot 的直接调用。也许检查那个对象，看看它的内容是什么？

编辑：我没有看到m.StubOutWithMock(urllib2, 'urlopen')上面的行，所以我以为你在比较两个电话；一个嘲笑（响应），一个不嘲笑（响应2）。更新的答案如下。

于 2010-06-29T23:55:11.533 回答