Comments on Valued Lessons: Why are my monads so slow?

When I was optimizing pyparsing, I found the @prof...

2008-03-29T15:21:00.000-07:00

When I was optimizing pyparsing, I found the @profile decorator described on the Python wiki to be very helpful.

The most primitive monads (such as pyparsing's Literal and Word) are those that benefit most from code optimization.

Function calls are peformance death in python, but some operations are more expensive then function calls. When optimizing Literal, I found that replacing

if instring[curLoc:curLoc+literalLength]==literalstring:

with

if literalstring.startswith(instring, curloc):

was quite a healthy win. I also apparently invented a cacheing system for exceptions that is unseen elsewhere in Python. Exceptions are used everywhere to signal when one of several parsing options doesn't match - this means that thousands of Exception instances are created and almost immediately discarded. Each monad ends up raising the same exception (with some minor, er, exceptions), so I create the Exception at monad creation time, and then just keep raising the cached Exception instead of creating a new one.

AFAIK, decorators only work at function creation t...

2008-03-25T13:23:00.000-07:00

AFAIK, decorators only work at function creation time and do not penalize actual calls, except for an extra function call.
Function calls are relatively expensive in Python, though.

Python doesn't do any optimization for them at all...

2008-03-20T06:55:00.000-07:00

Python doesn't do any optimization for them at all. Monads are incredibly flexible, but as any decent C.S. major knows, everything comes at a price. Monads give a high level of abstraction and can model a huge variety of computations. There's no reason to expect them to be on par with simpler, to-the-point styles of programming.

Monads have their place in programming, but that place is Haskell and other pure languages.

Very interesting. D language version 2.x has true ...

2008-03-20T05:09:00.000-07:00

Very interesting. D language version 2.x has true closures, I'd like to see how your code works there.

One thing worth keeping in mind about Python is th...

2008-03-20T04:51:00.000-07:00

One thing worth keeping in mind about Python is that function calls are expensive in time, more so than most other languages. So to get the fastest Python code, write in an imperative style and rely heavily on builtins.

Not good news for someone who wants to write fast functional code in Python. :D

I'd be curious to know if the JIT optimization per...

2008-03-19T17:31:00.000-07:00

I'd be curious to know if the JIT optimization performed by Psyco reduces the relative penalty for working with immutable objects.

Seeing what PyPy does to this code would be even more interesting, but that project is so bleeding edge, I have no idea what other issues you would stumble on.