I've learned. I'll share.

April 28, 2008

Events in Python

I'm a huge fan of the Actor Model . I think that for most applications, it's the best way to do concurrency. As CPUs get more cores, concurrency becomes more important for programmers, and I think the Actor Model will become more important, too.

But, sometimes you need something more light-weight for handling "events". For example, imagine you have some code listening for file changes in a particular directory. What you'd like to do is make an "event" that is "fired" whenever a file change is detected. When fired, there may be a "handler" or "listener" which is notified of the event. That "handler" is ultimately just some code which is executed when the event occurs.

A while ago, I wanted an event system like this for Python. I didn't see anything builtin or any library available, so I decided to write my own. I'd like to share what I created with you.

But first, I want to follow an example through other programming languages to give you an idea of what I was trying to accomplish and how it compares with what's out there. After that, I'll give you my implemenation in Python. Our example will be listening for changes on the file system. We want to keep the "watcher" code decoupled from the rest of the code, so we use events.

The best implementation of events that I've used is in C#, so we'll start there. In C# 1.0, our file watcher would looks something like this:

public delegate void FileChangeHandler(string source_path);

class FileWatcher
{
    public event FileChangeHandler FileChanged;

    public void WatchForChanges()
    {
       ...
       if(FileChanged != null)
       {
          FileChanged(source_path);
       }
       ...
    }
}

class FileChangeLogger
{
    public void OnFileChanged(string source_path)
    {
        Console.WriteLine(String.Format("{0} changed.", source_path));
    }
}

watcher = FileWatcher();
logger  = FileChangeLogger();
watcher.FileChanged += new FileChangeHandler(logger.OnFileChanged);
watcher.WatchForChanges();


It's pretty nice. The best part is at the end, where you can write "watcher.FileChange += ...". But, you need to type the completely useless "new FileChangeHandler", and you also need to wrap it all in a separate FileChangeLogger class. Luckily, in C# 2.0, they added Anonymous Delegates, which makes this much nicer:

watcher.FileChanged += delegate(string source_path)
{
    Console.WriteLine(String.Format("{0} changed.", source_path));
}
And in C# 3.0, they've made it even nicer!
watcher.FileChanged += (source_path => Console.WriteLine(String.Format("{0} changed.", source_path)));
C# 3.0 has an event system that's downright slick, with type-inferencing and everything. I don't think it gets much better than that.

Actually, there is one thing. If there's no handler registered with the event, calling "FileChanged()" will blow up because it sees FileChanged == null until a handler is registered. This means that you have to write "if(Event != null){Event(...):}" every single time you fire the event. Every single time. If you forget, your code will seem to work fine until there's no handler, in which case it will blow up and you'll smack your forhead because you forgot that you have to repeat that line of code every single time you fire an event. I really mean every single time. It's by far the worst wart on an otherwise elgant system. I have no idea why the designers of C# thought this was a good idea. When would you ever want to fire and event and have it blow up in your face?

Anyway, let's try a different programming language, perhaps Java. It has the worst implementation of events I've ever seen. Here's our FileWatcher example:

...

...

Ok, nevermind, I don't have the heart. I can imagine the code full of IListeners, and implementations, and keeping an array of them, and iterating over them, etc, and I just can't do it. I got seriously upset at one useless line in C#. In Java, it's at least 10 times worse. If you really have the stomach for it, go look at http://java.sun.com/docs/books/tutorial/uiswing/events/index.html. To me, it appears that in Java, events are one giant hack around the lack of first-class functions. If Java had first-class functions, none of that nonsense would be necessary.

Now that we've seen a good implementation of events in C# and avoided a bad one in Java, let's make one for Python. I'd rather it be like the C# event system, so let's see what our example would look like:

from Event import Event

class FileWatcher:
    def __init__(self):
        self.fileChanged = Event()

    def watchFiles(self):
        ...
        self.fileChanged(source_path)
        ...

def log_file_change(source_path):
    print "%r changed." % (source_path,)

watcher              = FileWatcher()
watcher.fileChanged += log_file_change

I think that looks pretty good. So what does the implementation of Event look like?


class Event(IEvent):
    def __init__(self):
        self.handlers = set()

    def handle(self, handler):
        self.handler.add(handler)
        return self

    def unhandle(self, handler):
        try:
            self.handlers.remove(handler)
        except:
            raise ValueError("Handler is not handling this event, so cannot unhandle it.")
        return self

    def fire(self, *args, **kargs):
        for handler in self.handlers:
            handler(*args, **kargs)

    def getHandlerCount(self):
        return len(self.handlers)

    __iadd__ = handle
    __isub__ = unhandle
    __call__ = fire
    __len__  = getHandlerCount
Wow. That was pretty short. Actually, this is one of the reasons I love Python. If the language doesn't have a feature, we can probably add it. We just added one of C#'s best features to Python in 26 lines of code. More importantly, we now have a nice, light-weight, easy-to-use event system for Python.

For all of you how like full examples that you can cut and paste, here is one that you can run. Enjoy!


class Event:
    def __init__(self):
        self.handlers = set()

    def handle(self, handler):
        self.handlers.add(handler)
        return self

    def unhandle(self, handler):
        try:
            self.handlers.remove(handler)
        except:
            raise ValueError("Handler is not handling this event, so cannot unhandle it.")
        return self

    def fire(self, *args, **kargs):
        for handler in self.handlers:
            handler(*args, **kargs)

    def getHandlerCount(self):
        return len(self.handlers)

    __iadd__ = handle
    __isub__ = unhandle
    __call__ = fire
    __len__  = getHandlerCount

class MockFileWatcher:
    def __init__(self):
        self.fileChanged = Event()

    def watchFiles(self):
        source_path = "foo"
        self.fileChanged(source_path)

def log_file_change(source_path):
    print "%r changed." % (source_path,)

def log_file_change2(source_path):
    print "%r changed!" % (source_path,)

watcher              = MockFileWatcher()
watcher.fileChanged += log_file_change2
watcher.fileChanged += log_file_change
watcher.fileChanged -= log_file_change2
watcher.watchFiles()

17 comments:

  1. Have you ever seen
    http://pydispatcher.sourceforge.net/ ?
    not so neat interface, though, but has other values.

    ReplyDelete
  2. pydispatcher looks very interesting, but it's not lightweight. The purpose of the event class I wrote is to be very lightweight.

    To do anything more complex usually means you've got concurrency involved, for which this event system isn't good enough. At that point, I think message passing (the Actor Model) is the right technique.

    The application I am working on right now is VERY concurrent, and so there is lots of message passing. But sometimes, I need something more simple. Thus, I use both my Event class and my Actor class, and they've worked quite well together. I'll share the Actor class in another post.

    ReplyDelete
  3. Yes, of course, it's clear that your goals are different from pydispatcher creators ones.
    I think, one of the differences, that affected pydispatcher interface, is that pydispatcher supports many-to-many connections between listening points and listeners, not only one-to-many.
    If you have large app (i.e. with large DOM tree), and you have N delegates and M listeners waiting for any change, you need to have N*M connections between them.
    Of course, you can add one more delegate, but then your code will have to maintain 2 delegates instead of one each time... with pydispatcher, you need just use "Any" option.
    I found this out while thinking if adding neat interface and delegates could add value to django.

    ReplyDelete
  4. I thought this was very helpful. Thanks a lot.

    ReplyDelete
  5. Hi! I really liked your code and I'll use it in my programs. I wanted to include a comment that the code comes from your blog, but I have to ask you if you consider the code useable under GPL/LGPL?

    Thanks,
    Alex

    ReplyDelete
  6. Hey!

    I loved your code and I've used it in a project of mine - thanks!

    However, you have not answered me under what license have you provided it. As I am at the point of releasing my project under GPLv3 I will have to replace your implementation with something else, so please please please tell me the terms under which I can use it.

    Great work - thank you!

    ReplyDelete
  7. Hi,

    i had to add a "return self" to the __iadd__ __isub__ mapped functions to get this to work.

    But i have just started with py, so maybe i overlooked smth.

    ReplyDelete
  8. Finally, I had to drop your code. Anyway, thanks for the great example.

    ReplyDelete
  9. Cool, nicely done. The first time I saw "Java" I had an immediate flashback to all that boilerplate code when I used to write Swing...

    ReplyDelete
  10. This code is clever :) . Thanks for the example

    ReplyDelete
  11. Axel is a library inspired by this example: http://pypi.python.org/pypi/axel

    ReplyDelete
  12. Dude really cool example here .. Thanx alot.

    ReplyDelete
  13. That's a really good code. I've modified it to work in multiprocessing.

    ReplyDelete
  14. See multiprocessing example following your great code here:
    http://rnovitsky.blogspot.co.il/2012/10/python-event-listener-multiprocessing.html

    ReplyDelete
  15. nice example (coming from a c# and java developer). I especially like the java example ;-)

    ReplyDelete

Blog Archive

Google Analytics