Is Stackless Python for You?

Wednesday Dec 6th 2000 by Cameron Laird

Is Stackless Python for you? If you program leading-edge games, high-performance network servers, or sophisticated text processors, it might be.

Python, of course, is the "interpreted, interactive, object-oriented programming language ... often compared to Tcl, Perl, Scheme or Java." That's what the official Python home page says. It's even more than that, in my view: I often recommend Python as the one essential language for most developers. It combines ease of learning, broad applicability, and expressive power better than any other single language, including C or BASIC.

It's been a bit hard to keep up with Python this year, though. The core team of Python developers has moved--changed employers--twice this year. This handful of top programmers around Python's inventor, Guido van Rossum, began the year at the Corporation for National Research Initiatives (CNRI) in Reston, Va., worked for several months at, and now seems to have found a more permanent home at Digital Creations 2, Inc. in Fredericksburg, Va. Along the way, they've already managed to release Python versions 1.6 and 2.0, which differed more in licensing legalities than technical features.

The administrative dance has impaired the engineering progress little. Release 2.0 includes welcome enhancements in Unicode and XML support, list comprehensions and other syntactic improvements, proper recognition of HTTP 1.1, and much more.

What's Stackless Python?

All of this is only background, though, to Stackless Python. Stackless Python is a collection of surprisingly successful experiments at re-implementing specific parts of the standard Python language processor to give it new capabilities. From the outside, Stackless Python looks like standard Python, and does all the things standard Python does. It's a bit like upgrading the carburetor on the family sedan: you can still drive it to soccer practice and the grocery store as you always did--but now it's also a lot more comfortable to cruise on the interstate.

Stackless Python is Python, plus a bit more. The bit more comes in two parts: First, Stackless Python "runs smoother" because it makes more efficient use of (stack) memory and, in some cases, processing power. This means that it also has a deeper recursion limit, and is easier to fit in such memory-constrained models as the PalmOS. Second, Stackless Python makes new control structures available. Traditional Python has

, and
, along with a threading module. Different flavors of Stackless Python augment these with various combinations of co-expressions, co-routines, continuations, generators, iterators, microthreads, and other concurrency or co-processing operators.

So Who Needs Stackless Python?

If you're already a Python programmer, and you want to run under PalmOS, or you're bumping recursion limits, then of course you'll want to look into Stackless Python.

It's more challenging, but also more rewarding, to understand the value of Stackless Python's new control structures. Suppose, for example, you're one of the growing number of game programmers who embeds Python as an extension language for your application. Your game has a lot of action, and you'd like to expose the trajectories of hundreds of spaceships, ward bosses, centurions, and worker ants... simultaneously, as manageable objects. In base Python, it would be natural to implement that base class as a separate thread.

If you're working with Stackless Python, though, you can write your class with microthreads. Microthreads are lighter-weight than threads, and therefore more scalable. Working programmers are finding that modest applications written with even a couple dozen Python or Java threads congest themselves and become unresponsive. The same hardware and operating systems that choke on a small number of threads, however, can support hundreds of microthreads. Moreover, Stackless Python makes control state more amenable to persistence than base Python. The virtue of persistence has so far received relatively little attention from the current crowd of Stackless Python fans. Someday, though, this kind of persistence will make it a practical one-liner to save a game in the middle of operations, and later restore it. Moreover, a game might migrate through time from one processor to another, or even to a distributed assemblage of co-operating processors.

Programmers code multitasking and distributed (including "peer-to-peer", in the fashionable phrase) applications even with such cumbersome languages as C and Fortran. In that sense, Stackless Python brings nothing absolutely new. However, just as Python is a higher-level language than assembler or C, microthreads supply a valuable abstraction that's well suited to many problems in gaming, simulation, and high-performance networking.

Powerful Parsing

The same underlying mechanisms are helpful in a much different, and more widely needed, functional domain: text processing. Many programmers find the "regular expressions" available in Perl and other languages, including Python, an epiphany of productivity, in comparison to the byte-oriented procedural pattern-matching they first learned. However, there are language constructs as much more powerful than regular expressions as regular expressions exceed character comparison. Icon, for example, has "generators," which are, roughly speaking, expressions that return multiple values. Generators have resonances with the "functional" features--lambda, map, list comprehensions--that have been added to recent Python releases. A generator efficiently expresses such operations as, "find instances of 'entitlement' in the vicinity of a numeric year after 1994 in the top half of any formatted page." In fact, generators usually have "lazy" semantics so that a monitoring application might begin to display the first instances before the entire document has been scanned.

With the generators that Stackless Python provides, savvy programmers can build remarkably efficient and concise parsers for a variety of computing and human language problems.

How Do I Get Stackless Python?

Stackless Python is not part of the standard Python release for either version 1.6 or 2.0. You can, however, download a copy from the Stackless Python home page at Stackless Python is sufficiently well-verified that several developers, like independent consultant Gordon McMillan, already rely on it for production deliveries. It's widely expected that stackless elements, probably including microthreads first, will appear in future Python releases, perhaps as early as version 2.1. All the details of syntax and semantics remain provisional, though, as of mid-November 2000.

For deeper explanations of Stackless Python construction and capabilities, see

About the author:
Cameron Laird is vice president of Phaseit Inc., in Friendswood, Texas. He writes frequently on Python programming topics and welcomes e-mail comments.

Mobile Site | Full Site
Copyright 2017 © QuinStreet Inc. All Rights Reserved