python - C 或 Python 在创建套接字连接时的性能和可扩展性？

Question

我确实看到了这篇文章，但它没有回答我的问题：C/Python 套接字性能？

我的任务是创建一个可以基于套接字创建数千个连接的应用程序。我可以在 Python 中做到这一点，但我希望有提高性能的空间。由于我过去的项目，我知道在 Python 中这是可能的，但我很好奇如果我用 C（而不是 C++）来做这个项目，性能会有多大提升？

Answer 1

It really depends on what you're doing with the sockets.

The best generic answer is: Usually Python is good enough that it doesn't matter, but sometimes it's not.

The overhead in the time taken to create and connect sockets is minimal, and reading and writing isn't much worse. But that doesn't matter, that's pretty much never any significant time spent doing that anyway.

There are reactors and proactors for Python every bit as good as the general-purpose ones available for C (and half of the C libraries have Python bindings). If you're not doing much significant work beyond the sockets, this is often your main bottleneck. If you've got a very specific use pattern and/or very tightly specified hardware, you might be able to write a custom reactor or proactor that beats out anything general-purpose. In that case, you pretty much have to go with C, not Python.

But usually, you've got significant work to do beyond just manipulating sockets.

If that work is mostly independent and highly parallelizable, C obviously beats Python (because of the GIL), unless the jobs are heavy enough that you can multi-process them (and keep in mind that "heavy enough" can be pretty heavy on Windows platforms). Except, of course, that it's incredibly easy to screw up performance (not to mention stability) writing multi-threaded C code; really, something like Erlang or Haskell is probably a better bet here than either C or Python. (If you're about to say, "But we've got people who are experienced at C but they can't learn Haskell", then those people are probably not good enough programmers to write multi-threaded code.)

If that work is mostly memory copying between socket buffers, and you can deal with a tightly-specified system, you may be able to write C code that optimizes zero-copies, and there's no way to do that in Python.

But if it's mostly typical things like waiting on disk or serialized computation, then it scarcely matters how you write the socket-stuff, because it's going to end up waiting on the real code anyway.

So, without any more information, I'd go with Python, because the time you save getting things up and running and debugged vs. C can be spent optimizing or otherwise improving whatever turns out to matter.

Answer 2

如果您使用的是 Windows 平台，请了解 IOCP 的每个核心一个线程的概念，并远离使用每个套接字使用或多或少需要一个线程的线程池。