I've learned. I'll share.

October 14, 2008

The manycore era is already upon us (and python isn't keeping up)

For many months, perhaps years, we as programmers have realized our trouble ahead coming with the "manycore era". The attitude seems to be "someday we're really going to have to figure this concurrency thing out". But this month I have been hit with a terrible realization: the manycore era is already upon us. Technically, it's the "multicore", not "manycore", but I think that's a small distinction.

Why the sudden epiphany? A few weeks ago, I built my self a new computer and put in an Intel quad-core CPU. My computer is much faster now, and I'm very happy with it, but after a few weeks I have realized something: I never use more than 1/4 of my CPU. I have a little graph on my screen at all times showing me the CPU usage. It goes up to 25% all of the time, but I have never seen it go higher. Ever.

On one hand this is a good thing. The new computer is so fast that everything I do is instant, and only uses a tiny bit of power. If it isn't instant, it's usually disk or network bound, not CPU bound.

On the other hand, even my own software isn't using more than 25%, and it is CPU bound at times. I'd like to fix it, but it's written in python, and the short version of a long, boring story is that python has a thing called The GIL which makes it so python as currently implemented cannot use more than 25% of the CPU except under very rare circumstances.

It seems that programming languages takes a long time to be adopted, but I think concurrency is a big enough rule changer to shake up which programming languages are dominant. In my specific case, if python doesn't fix it's concurrency problems soon, I'm going to have to stop considering it because I'll never be able to get it to use more than 1 little piece of the CPU. Right now, that's 25%, but in a few years, it will be only 3%. Either python is going to have to change, or I'm going to have to change programming languages (or use something like Jython or IronPython, I suppose).

A few weeks ago, sitting behind my single core computer, I was in the "someday, we'll have to tackle concurrency" camp. Now, sitting in front of my quad core machine, never using more than 25% of its power, everything has changed. Concurrency is no longer a question of if or when, it's here right now. If you don't believe me, get yourself a quad-core machine and watch the CPU usage graph. I think you'll be surprised.


  1. It's worth taking a look at the "processing" module, which was renamed the "multiprocessing" module and put into Python 2.6. It provides a concurrency model that avoids the GIL by forking and using pipes and shared memory. Sure, that's a little ugly under the hood, but (multi)processing wraps it up in a nice looking API:

    multiprocessing documentation

  2. Mark Shuttleworth brought this up in his Keynote at Pycon UK and the reaction seemed to fall into 3 camps.

    1. Who cares?
    2. Just spawn out processes and use processing.
    3. I want threads, stop telling me how to design my app, aka Processes are too heavyweight on Windows.

    I guess there are people out there in camp 3 who need threads, not processes, and just can't live with the GIL. As Stan mentioned those in camp 2 have a solution, but it seems more like a workaround than a solution :(

    I have a recollection of a patch being posted that seemed to work on multi-core machines, but ran slower on single core machines and so was scrapped. When single-core becomes the exception, it may be that it is revisited, but who knows?

  3. I may have to try using multiple processes. I had seen that module before but didn't realize it was so complete. Thanks for the tip.

    But can we all agree that's really a kludge? It sounds a lot like when Java people defend not having first-class functions by saying "just use an interface". Sure, it's possible. But it's ugly and no fun.

  4. I agree, the whole multicore thing has been out for a while now and I'm yet to see anyone address it with much thought.

  5. I just ran into this myself, I was working on a project that is very cpu bound, but also very parallel. I could run 1000 threads if my system would not grind to a halt doing so.

    But, no matter what I set the thread count to, python only uses 25%.

    I will have to continue looking into options, but I may just have to run 4 copies of the program to allow me to access the full power of my system for now.

  6. Not to say this approach is the be-all, end-all for multi-core work, but good ol' process spawning gets a lot of real work done, fairly easily, without changing the way I have to work.

    Too bad Blogger is brain dead about pre tags, but for example here's 75% (approx) of each core being used; I think I needed to put more load on this web app to jam it up to 100% total:

    CPU %user %nice %sys
    all 60.44 0.00 13.24
    0 62.35 0.00 13.53
    1 55.88 0.00 13.53
    2 62.35 0.00 11.76
    3 60.82 0.00 14.62

    What I like about this is I can get incredible - by the standards of just a few years ago - amount of processing done, with commodity hardware, without radically changing how I have to develop.

    Again, I'm not suggesting there isn't room for improvement... and some folks will be forced to by their own needs and circumstances. But it isn't all bad that not *everyone* has to.

  7. Shuttleworth is half-right. 2 and 3 aren't mutually exclusive.

    Threads are important when scaling vertically and specially in the cloud environment where you pay per cpu/mem it's invaluable.

    Process are better when scaling horizontally, but that's a better solution for really big applications.

    I think both are necessary and as long as there are tools that allow me to abstract the scaling strategy I'm fine.


Blog Archive

Google Analytics