I've learned. I'll share.

December 31, 2007

Immutable Data in Python (Record or Named Tuple)

A valued lesson that I have learned the hard way is that mutable data can get nasty, and that it's really nice to have immutable data structures. I've been writing a lot of python recently, and after a while I realized that I was writing a lof of this:

class SomeDataStructure:
    def __init__(self, arg1, arg2):
        self.prop1 = arg1
        self.prop2 = arg2

Not only was this repetitive, but I found that little mutability bugs were cropping up on me. It's just too easy to trip over them when the default is mutability, as it is in python. In fact, immutable data structures aren't even a built-in option in python.

Finally the pain of using mutable data structures grew too large. I looked for a way and couldn't find anything, so I created my own. It supports getters, setters, inheritance, default values, altering several values at once, and I think the syntax is nice. I've been using it for everything the last few months and it's been great. It's helped with code repetition, concurrency, and serialization.

I hope you can learn from my valued lesson and not repeat my mistakes. Here's how you use it.

class Person(Record("name", "age")):
    pass

class OldPerson(Person):
    @classmethod
    def prepare(cls, name, age = None):
        return (name, age)

peter   = Person("Peter", 26)
wes     = Person("Wes", 28)
grandpa = OldPerson("Bubba")
wes2    = wes.setAge(29)
wes3    = wes.alter(name = "Grandpa", age = 57)
print peter, grandpa, wes.name, wes2.age

Here is the code. This is actually a simplified version of the original that I rewrote on my own time. I use a more comlete/complex version in the code I'm writing for my employer.

def Record(*props):
    class cls(RecordBase):
        pass

    cls.setProps(props)

    return cls

class RecordBase(tuple):
    PROPS = ()

    def __new__(cls, *values):
        if cls.prepare != RecordBase.prepare:
            values = cls.prepare(*values)
        return cls.fromValues(values)

    @classmethod
    def fromValues(cls, values):
        return tuple.__new__(cls, values)

    def __repr__(self):
        return self.__class__.__name__ + tuple.__repr__(self)

    ## overridable
    @classmethod
    def prepare(cls, *args):
        return args

    ## setting up getters and setters
    @classmethod
    def setProps(cls, props):
        for index, prop in enumerate(props):
            cls.setProp(index, prop)
        cls.PROPS = props

    @classmethod
    def setProp(cls, index, prop):
        getter_name = prop
        setter_name = "set" + prop[0].upper() + prop[1:]

        setattr(cls, getter_name, cls.makeGetter(index, prop))
        setattr(cls, setter_name, cls.makeSetter(index, prop))

    @classmethod
    def makeGetter(cls, index, prop):
        return property(fget = lambda self : self[index])

    @classmethod
    def makeSetter(cls, index, prop):
        def setter(self, value):
            values = (value if current_index == index
                            else current_value
                      for current_index, current_value
                      in enumerate(self))
            return self.fromValues(values)
        return setter

For comparison, I've seen similar ideas at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/500261 and http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/303439 but I don't like their implementation or design as much, especially since they lack proper setters.

I've also read that future versions of Python will have NamedTuples, which is something I wish it had already.

39 comments:

  1. Isn't a bit of a shame to have to declare a new class for each optional parameters ?

    I would love to have something like the Erlang record.

    -record(person, {name, age, title='Sir'}).

    Then I can create, update or read records using pattern matching.

    Can I get something close in Python ?

    Thanks for your help.

    ReplyDelete
  2. You don't have to declare a new record for optional values. I could have put the prepare method on the original Person record. I just wanted to show optional values and inheritance at the same time. They can be done separately.

    Tuples is the closest thing that you can get to pattern matching in python, and since these records are tuples, you at least get this:


    def reversed_name((name, age)):
    return reversed(name)

    assert reversed_name(Person("peter", 26)) == "retep"

    I'm sorry about the formatting. Does anyone know how to put code in comments in a decent way?

    ReplyDelete
  3. Why not subclass tuple directly? You could make it only accept a fixed number of values and add get/set methods for the named fields corresponding to the indices in the underlying tuple.

    pierre r: You could create a method-less class in Python to get the same sort of behavior. That would basically give you a C-style struct.

    ReplyDelete
  4. Nice piece of code! In your examples there is a reference to a missing alter method, here is a simple implementation following your code's logic:

    def alter(self, **values):
    new_values = \
    (values[key] if values.has_key(key) else self[index]
    for index,key in enumerate(self.PROPS))
    return self.fromValues(new_values)

    ReplyDelete
  5. omg, for which reason you do this?

    why not just
    class Person(object):
    __immutable = ('age', )
    age = None
    name = None

    def __setattr__(self, name, value):
    if name in self.__immutable:
    raise Exception('%s is immutable of %r' % (name, self))
    super(Person, self).__setattr__(name, value)

    or something like that. Much less overheat in setters/getters, coz you create separate function for EACH property.

    ReplyDelete
  6. I prefer just using a simple struct.

    class Struct:
    def __init__(self, **entries): self.__dict__.update(entries)

    >>> me = Struct(name='Homer Simpson', age=40)
    >>> me.age
    40
    >>> me.name = 'Max Power'
    >>> vars(me)
    {'age': 40, 'name': 'Max Power'}

    Though I am biased against getter/setter methods and am convinced they are evil and definitely un-pythonic. But that's another story...

    ReplyDelete
  7. We will discuss here about valued lessons and spend our lives in a good manner. If we adopt good strategy, we will enjoy our life. So, learn a good lesson about life and enjoy a successful life. Coursework writing services.

    ReplyDelete
  8. Anyone can attempt writing on your behalf, however, the question is would you pay someone to do it for you without checking their credibility? Our firm has established itself as the most trustworthy assignment help firm in Australia and globally. Join these thousands of students and achieve high distinction in each and every one of your college tasks. This is the main reason why most students search for online assignment help over the internet and choose only the most proficient and trusted academic writing experts. So, if you are searching for quality assignment help usa you can find it right here from the local experts.

    ReplyDelete
  9. Learning python language is not a piece of cake but your selected lesson can make this intimidating task a lit bit easy. Your suggested lesson is effective to create immutable data structures. Thanks for this valuable information. Assignment Writing Service

    ReplyDelete
  10. Excellent Blog! I would like to thank for the efforts you have made in writing this post. Assignment Paraphrasing Services Uk

    ReplyDelete
  11. I always like your article because you have provide every time informative post. Thanks! G.I Joe Cobra Commander Costume

    ReplyDelete
  12. Our advanced and patent shopify app developer software system is excellent for Microsoft Dynamics

    ReplyDelete
  13. We offer Next Gen SIEM and bespoke ids/ips Services that comes with built-in support for most national and international compliance standards

    ReplyDelete
  14. freelance time tracking software to help you to save your precious time

    ReplyDelete
  15. I was very happy to find this site. I really enjoyed reading this article today and think it might be one of the best articles I have read so far. I wanted to thank you for this excellent reading !! I really enjoy every part and have bookmarked you to see the new things you post. Well done for this excellent article. Please keep this work of the same quality.
    Data Science Course in Bangalore

    ReplyDelete
  16. Wow, happy to see this awesome post. I hope this think help any newbie for their awesome work and by the way thanks for share this awesomeness, i thought this was a pretty interesting read when it comes to this topic. Thank you..
    Artificial Intelligence Course

    ReplyDelete
  17. Excellent Blog! I would like to thank you for the efforts you have made in writing this post. Gained lots of knowledge.
    Data Analytics Course

    ReplyDelete
  18. What an incredible message this is. Truly one of the best posts I have ever seen in my life. Wow, keep it up.
    AI Courses in Bangalore

    ReplyDelete
  19. Awesome article. I enjoyed reading your articles. this can be really a good scan for me. wanting forward to reading new articles. maintain the nice work!
    Data Science Courses in Bangalore

    ReplyDelete
  20. I am sure it will help many people. Keep up the good work. It's very compelling and I enjoyed browsing the entire blog.
    Business Analytics Course in Bangalore

    ReplyDelete
  21. Hello there! Nice article!!! But anyways here’s one of the trusted online baccarat site we can offer you so many promo and event everyday!! 한국야동닷컴

    ReplyDelete
  22. This is very interesting, You're a very skilled blogger.
    I've joined your feed and look forward to seeking more of your great post. Also, I have shared your web site in my social networks!
    중국야동넷

    ReplyDelete
  23. Great web site you have got here.. It’s hard to find quality writing like yours these days. I really appreciate individuals like you! Take care!! 파워볼 하는법

    ReplyDelete
  24. Good write-up, I am normal visitor of one抯 web site, maintain up the nice operate, and It is going to be a regular visitor for a lengthy time. 온라인경마

    ReplyDelete
  25. Great job for publishing such a beneficial web site
    바카라사이트

    ReplyDelete
  26. I do agree with all of the ideas you’ve presented in your post.
    카지노사이트

    ReplyDelete
  27. Hard to ignore such an amazing article like this. You really amazed me with your writing talent. Thank you for sharing again.
    토토

    ReplyDelete
  28. This is really helpful post and very informative there is no doubt about it.
    바카라사이트

    ReplyDelete
  29. I need to thank you for this very good read and i have bookmarked to check out new things from your post. Thank you very much for sharing such a useful article and will definitely saved and revisit your site.
    Data Science Course

    ReplyDelete
  30. Your site is truly cool and this is an extraordinary moving article and If it's not too much trouble share more like that. Thank You..
    Digital Marketing Course in Hyderabad

    ReplyDelete
  31. Thank a lot. You have done excellent job. I enjoyed your blog . Nice efforts
    Data Science Certification in Hyderabad

    ReplyDelete
  32. These type of articles keeps the users interest in the website. I found this one pretty fascinating and it should go into my collection. I am Impressed. Please keep going to write more content… 토토

    ReplyDelete
  33. Before Commenting, I wanna say thank you for providing this great information. I feel too good to read it this awesome blog. Keep sharing this type of content. 경마사이트

    ReplyDelete
  34. We have read your blog. It was really informative and we have collected much knowledge that has assisted me a lot. People can also visit our website… 온라인카지노

    ReplyDelete
  35. I was very interested in the article. it’s quite inspiring I should admit. I like visiting you site since I always come across interesting articles like this one. 토토

    ReplyDelete
  36. Here we provide technical HELP for 123.com.setup, driver download, wireless setup, USB setup, and solutions for your HP printer related issues.

    ReplyDelete
  37. Nice blog! Really thank you for your knowledge of the phyton language and other symbols. If to be honest with me I did not strong at converting symbols and writing on phyton language and how to convert a picture in such a way that it would be transformed into a scheme or for example diagram or graph and it is quite complicated for me to write on such language, but I should do it because have such academic course at the university, so use freshessay writing agency as landing help.

    ReplyDelete
  38. Thanks for sharing this informative content.,
    Leanpitch provides online training in Scrum Master Certification during this lockdown period everyone can use it wisely.
    Join Leanpitch 2 Days CSM Certification Workshop in different cities.

    Scrum master certification online

    CSM certification online

    CSM online

    CSM online certification

    CSM online training

    CSM training online

    Scrum master training online

    ReplyDelete

Blog Archive

Google Analytics