Keith Devens .com

Threads in Python

In yet another link gleaned from LtU, IBM developerWorks: Charming Python: Implementing "weightless threads" with Python generators.

In an earlier "Charming Python" installment, David introduced a way of simulating full-fledged coroutines with generators and a simple scheduler. It is possible to extend this scheduler in straightforward ways to allow extremely lightweight threading of multiple processes. Much as with Stackless Python microthreads, pseudo-coroutine "weightless threads" require almost none of the context switch and memory overhead of OS -- or even userland -- threads. Here David introduces weightless threads as an elegant solution for problems whose natural solutions involve large numbers of cooperating processes.

Just what is a microthread? Basically, a microthread is a process that can run with minuscule inherent resource requirements -- and that runs within a single instance of the Python interpreter (in a common memory space, and so on). With microthreads, it is possible to run tens of thousands of parallel processes on a moderately capable modern PC, and to switch between contexts hundreds of thousands of times every second. Calls to fork() or standard OS threading calls do not even come close to this! Even so-called "lightweight" threading libraries have threads that are orders of magnitude "heavier" than those presented here.

Permalink | Comments? | id 2179

11:17 (utc-4)