8

Update to original post: A colleague pointed out what I was doing wrong. I'll give the explanation at the bottom of the post, as it might be helpful for others.

I am trying to get a basic understanding of the limits on network performance of python programs and have run into an anomaly. The code fragment

while 1:
    sock.sendto("a",target)

sends UDP packets to a target machine, as fast as the host will send. I measure a sending rate of just over 4000 packets per second, or 250 us per packet. This seems slow, even for an interpreted language like python (the program is running on a 2 GHz AMD opteron, Linux, python version 2.6.6). I've seen much better performance in python for TCP, so I find this a bit weird.

If I run this in the background and run top, I find that python is using just 25% of the cpu, suggesting that python may be artificially delaying the transmission of UDP packets.

Has anyone else experienced anything similar? Does anyone know if python does limit the rate of packet transmission, and if there is a way to turn this off?

BTW, a similar C++ program can send well over 200,000 packets per second, so it's not an intrinsic limit of the platform or OS.

So, it turns out I made a silly newbie mistake. I neglected to call gethostbyname explicitly. Consequently, the target address in the sendto command contained a symbolic name. This was triggering a name resolution every time a packet was sent. After fixing this, I measure a maximum sending rate of about 120,000 p/s. Much better.

4

2 回答 2

1

您可能希望发布更完整的代码示例，以便其他人可以重复您的基准测试。每次循环迭代 250μs太慢了。根据优化 Python 的日常经验，我预计 Python 的解释器开销在现代机器上远低于 1μs。换句话说，如果 C++ 程序每秒发送 200k 个数据包，我希望 Python 的速度也能达到相同的数量级。

（鉴于上述情况，通常的优化建议，例如将 sock.sendto 的属性查找移出循环，在这里不适用，因为速度慢来自另一个来源。）

一个很好的第一步是strace用来检查 Python 实际在做什么。是单线程程序还是多线程应用程序可能会浪费时间等待 GIL？是sock普通的 Python 套接字还是更复杂的 API 的一部分？当您直接调用os.write套接字时也会发生同样的情况fileno吗？

于 2012-09-03T18:29:36.417 回答
1

您是否尝试过先做connect()，然后使用send()代替sendto()？（UDPconnect()只是建立目标地址，它实际上并没有建立“连接”。）我对此很生疏，但我相信 Python 对地址参数的解释比 C 套接字更多，这可能会增加开销。

于 2012-09-03T19:19:48.113 回答