One solution to get a little more performance is to use Eventlet. Eventlet is a library which uses what is called "Green Threads" and hacks on top of the networking libraries to give a mutli-threaded like feel to an application. As part of this blogging series for HP's Advanced Technology Group I'll write about some of the things I found out the hard way about Eventlet so hopefully you don't hit them.
What are Green Threads?
Green Threads are basically a way of doing multi-tasking on a single real thread. They use what is called "Cooperative Yielding" to allow each other to run rather than being explicitly scheduled. This has the advantage of removing the need for locks in many cases and making asynchronous IO easier. But they come with caveats which can hurt if you don't know about them.
Threading library patched
One of the first things you typically do with eventlet is "Monkey Patch" standard Python library functions so that the are compatible with cooperative yielding. For example you want the sleep() function to yield rather than hanging all the green threads up until finished.
The threading library is one of the libraries that is monkey patched and the behaviour suddenly becomes slightly different. When you try to spawn a thread control will not return back to the main thread until the child thread has finished execution. So your loop that tries to spawn X threads will suddenly only spawn 1 thread and not spawn the next until that thread has finished. It is recommended you use Eventlet's green thread calls instead (which will actually work as expected).
Application hangs
Cooperative yielding relies on the library functions being able to yield. Which means that if you use functions that do not understand this the yielding will not happen and all your green threads (including the main thread) will hang waiting. Any unpatched system call (such as executing some C/C++ functions) falls into this category.
A common place you can see this is with the MySQLdb library which is a wrapper for the MySQL C connector (libmysqlclient). If you execute some complex query that will take some time, all green threads will wait. If your MySQL connection hangs for any reason... well, you are stuck. I recommend using one of the native Python MySQL connectors instead.
Another place I have seen this is with any library that relies on epoll. Python-gearman is an example of this. It seems that Eventlet only patches the select() calls, so anything that uses epoll.poll() is actually blocking with Eventlet.
In summary there are cases where Eventlet can be useful. But be careful where you are using it or things can grind to a halt really quickly.
