I've learned. I'll share.

December 31, 2007

Immutable Data in Python (Record or Named Tuple)

A valued lesson that I have learned the hard way is that mutable data can get nasty, and that it's really nice to have immutable data structures. I've been writing a lot of python recently, and after a while I realized that I was writing a lof of this:

class SomeDataStructure:
    def __init__(self, arg1, arg2):
        self.prop1 = arg1
        self.prop2 = arg2

Not only was this repetitive, but I found that little mutability bugs were cropping up on me. It's just too easy to trip over them when the default is mutability, as it is in python. In fact, immutable data structures aren't even a built-in option in python.

Finally the pain of using mutable data structures grew too large. I looked for a way and couldn't find anything, so I created my own. It supports getters, setters, inheritance, default values, altering several values at once, and I think the syntax is nice. I've been using it for everything the last few months and it's been great. It's helped with code repetition, concurrency, and serialization.

I hope you can learn from my valued lesson and not repeat my mistakes. Here's how you use it.

class Person(Record("name", "age")):
    pass

class OldPerson(Person):
    @classmethod
    def prepare(cls, name, age = None):
        return (name, age)

peter   = Person("Peter", 26)
wes     = Person("Wes", 28)
grandpa = OldPerson("Bubba")
wes2    = wes.setAge(29)
wes3    = wes.alter(name = "Grandpa", age = 57)
print peter, grandpa, wes.name, wes2.age

Here is the code. This is actually a simplified version of the original that I rewrote on my own time. I use a more comlete/complex version in the code I'm writing for my employer.

def Record(*props):
    class cls(RecordBase):
        pass

    cls.setProps(props)

    return cls

class RecordBase(tuple):
    PROPS = ()

    def __new__(cls, *values):
        if cls.prepare != RecordBase.prepare:
            values = cls.prepare(*values)
        return cls.fromValues(values)

    @classmethod
    def fromValues(cls, values):
        return tuple.__new__(cls, values)

    def __repr__(self):
        return self.__class__.__name__ + tuple.__repr__(self)

    ## overridable
    @classmethod
    def prepare(cls, *args):
        return args

    ## setting up getters and setters
    @classmethod
    def setProps(cls, props):
        for index, prop in enumerate(props):
            cls.setProp(index, prop)
        cls.PROPS = props

    @classmethod
    def setProp(cls, index, prop):
        getter_name = prop
        setter_name = "set" + prop[0].upper() + prop[1:]

        setattr(cls, getter_name, cls.makeGetter(index, prop))
        setattr(cls, setter_name, cls.makeSetter(index, prop))

    @classmethod
    def makeGetter(cls, index, prop):
        return property(fget = lambda self : self[index])

    @classmethod
    def makeSetter(cls, index, prop):
        def setter(self, value):
            values = (value if current_index == index
                            else current_value
                      for current_index, current_value
                      in enumerate(self))
            return self.fromValues(values)
        return setter

For comparison, I've seen similar ideas at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/500261 and http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303439 but I don't like their implementation or design as much, especially since they lack proper setters.

I've also read that future versions of Python will have NamedTuples, which is something I wish it had already.

6 comments:

  1. Isn't a bit of a shame to have to declare a new class for each optional parameters ?

    I would love to have something like the Erlang record.

    -record(person, {name, age, title='Sir'}).

    Then I can create, update or read records using pattern matching.

    Can I get something close in Python ?

    Thanks for your help.

    ReplyDelete
  2. You don't have to declare a new record for optional values. I could have put the prepare method on the original Person record. I just wanted to show optional values and inheritance at the same time. They can be done separately.

    Tuples is the closest thing that you can get to pattern matching in python, and since these records are tuples, you at least get this:


    def reversed_name((name, age)):
    return reversed(name)

    assert reversed_name(Person("peter", 26)) == "retep"

    I'm sorry about the formatting. Does anyone know how to put code in comments in a decent way?

    ReplyDelete
  3. Why not subclass tuple directly? You could make it only accept a fixed number of values and add get/set methods for the named fields corresponding to the indices in the underlying tuple.

    pierre r: You could create a method-less class in Python to get the same sort of behavior. That would basically give you a C-style struct.

    ReplyDelete
  4. Nice piece of code! In your examples there is a reference to a missing alter method, here is a simple implementation following your code's logic:

    def alter(self, **values):
    new_values = \
    (values[key] if values.has_key(key) else self[index]
    for index,key in enumerate(self.PROPS))
    return self.fromValues(new_values)

    ReplyDelete
  5. omg, for which reason you do this?

    why not just
    class Person(object):
    __immutable = ('age', )
    age = None
    name = None

    def __setattr__(self, name, value):
    if name in self.__immutable:
    raise Exception('%s is immutable of %r' % (name, self))
    super(Person, self).__setattr__(name, value)

    or something like that. Much less overheat in setters/getters, coz you create separate function for EACH property.

    ReplyDelete
  6. I prefer just using a simple struct.

    class Struct:
    def __init__(self, **entries): self.__dict__.update(entries)

    >>> me = Struct(name='Homer Simpson', age=40)
    >>> me.age
    40
    >>> me.name = 'Max Power'
    >>> vars(me)
    {'age': 40, 'name': 'Max Power'}

    Though I am biased against getter/setter methods and am convinced they are evil and definitely un-pythonic. But that's another story...

    ReplyDelete

Blog Archive

Google Analytics