As I wrote previously, I'm a fan of message-passing concurrency. But to use it in python, I had to write my own infrastructure. It's far from perfect, but it's working for me. I've simplified it a bit and prepared some code to share with you.
The basic idea of my framework is to be and simple and easy to use as possible. There's no Erlang-style pattern matching or process linking. There isn't even any built-in error handling. But, there's a lot of support for the typical "react loop" style of message handling.
The architecture is very simple. There's one thread per actor. An actor is just an object you can send a message to. The actor receives the messages in the order they were sent. You always know the sender of the message; it's built right in. There's also "Bridge" that allows you to interact with actors from non-actor code.
The only tricky thing you'll see is a "handles" decorator. This is used to help you identify how you want an actor to handle a given message type. It helps you avoid re-creating a message loop and dispatcher yourself for every actor.
I've included a little example of how to use the framework. It obviously quite simple, but keep in mind that I used little more for the foundation of a rather complex file synchronization application. A little bit goes a long way.
So, here it is. Cut, paste, and run.
from threading import Thread, Event from Queue import Queue, Empty class IActor: pass # send(message, sender) class HandlerRegistry(dict): # should be used as a decorator def __call__(self, typ): def register(func): self[typ] = func return func return register OtherMessage = "Other Message" class Stop(Exception): def __repr__(self): return "Stop()" class Stopped: def __repr__(self): return "Stopped()" class ActorNotStartedError(Exception): def __init__(self): Exception.__init__(self, "actor not started") class Actor(IActor): @classmethod def spawn(cls, *args, **kargs): self = cls(*args, **kargs) self.mailbox = Mailbox() start_thread(target = self.act, as_daemon = True) return self def send(self, message, sender): if self.mailbox is None: raise ActorNotStartedError() else: self.mailbox.send(message, sender) def receive(self): if self.mailbox is None: raise ActorNotStartedError() else: return self.mailbox.receive() # override if necessary def act(self): self.handleMessages() handles = HandlerRegistry() @classmethod def makeHandles(*classes): return HandlerRegistry((typ, handler) for cls in classes for (typ, handler) in cls.handles.iteritems()) def handleMessages(self): try: while True: message, sender = self.receive() self.handleMessageWithRegistry(message, sender) except Stop: pass def handleMessageWithRegistry(self, message, sender): registry = self.__class__.handles handler = registry.get(message.__class__) or registry.get(OtherMessage) if handler is not None: handler(self, message, sender) @handles(OtherMessage) def onOther(self, message, sender): pass @handles(Stop) def onStop(self, message, sender): sender.send(Stopped(), self) raise message def start_thread(target, as_daemon, name = None): thread = Thread(target = target) if name: thread.setName(name) thread.setDaemon(as_daemon) thread.start() return thread class Mailbox: def __init__(self): self.mailbox = Queue() def send(self, message, sender): self.mailbox.put((message, sender), block = False) def receive(self, timeout = None): return self.mailbox.get(block = True, timeout = timeout) class Bridge(IActor): def __init__(self): self.mailbox = Mailbox() def send(self, message, sender): self.mailbox.send(message, sender) def call(self, target, request, timeout, default = None): self.sendRequest(target, request) return self.receiveResponse(timeout, default) # targeted_requests can be an iterator def multiCall(self, targeted_requests, timeout, default = None): count = 0 for target, request in targeted_requests: self.sendRequest(target, request) count += 1 for _ in xrange(count): yield self.receiveResponse(timeout, default) def stop(self, actors, timeout): stop = Stop() return list(self.multiCall(((actor, stop) for actor in actors), timeout, default = None)) def sendRequest(self, target, request): target.send(request, self) def receiveResponse(self, timeout, default): try: message, sender = self.mailbox.receive(timeout = timeout) return message except Empty: return default if __name__ == "__main__": import time class GetInventory: pass class Task: def __init__(self, input, destination): self.input = input self.destination = destination class Worker(Actor): handles = Actor.makeHandles() def __init__(self, skill): self.skill = skill @handles(Task) def onTask(self, task, sender): output = self.skill(task.input) task.destination.send(output, self) class Warehouse(Actor): handles = Actor.makeHandles() def __init__(self): self.inventory = [] @handles(GetInventory) def onGetInventory(self, message, sender): # copy the inventory to avoid anyone mutating it sender.send(list(self.inventory), self) @handles(OtherMessage) def onTaskResult(self, result, sender): self.inventory.append(result) worker = Worker.spawn(lambda x : x * 2) positives = Warehouse.spawn() negatives = Warehouse.spawn() bridge = Bridge() for val in [1, 2, 3, -2, -4, -6]: warehouse = positives if val >= 0 else negatives worker.send(Task(val, warehouse), sender = None) print bridge.call(positives, GetInventory(), 1.0) #should be [ 2, 4, 6] print bridge.call(negatives, GetInventory(), 1.0) #should be [-4, -8, -12] print bridge.stop([worker, positives, negatives], 1.0) #should be [Stopped(), Stopped(), Stopped()] class Start: def __init__(self, target): self.target = target class Ping: def __repr__(self): return "Ping()" class Pong: def __repr__(self): return "Pong()" class Pinger(Actor): handles = Actor.makeHandles() @handles(Start) def onStart(self, start, sender): start.target.send(Ping(), self) @handles(Pong) def onPong(self, pong, sender): print "-", sender.send(Ping(), self) class Ponger(Actor): handles = Actor.makeHandles() @handles(Ping) def onPing(self, ping, sender): print "+", sender.send(Pong(), self) # should print lots of +-+-+- pinger = Pinger.spawn() ponger = Ponger.spawn() pinger.send(Start(ponger), sender = None) time.sleep(0.1) bridge.stop([pinger, ponger], 1.0)
did you took at look at
ReplyDeletehttp://twistedmatrix.com/
?
Very nice and small and clean.
ReplyDeleteA couple of thoughts:
In many ways, Erlang's selective receive is one of the most critical parts of Erlang. Without it, application code gets pretty complex, trying to manage when messages can be handled and when they need to be deferred. It's a key part of blocking RPC-like functions, which are, in turn, a key part of Erlang's gen_servers.
One other comment: Python threads are pretty expensive. Having one per actor means the number of actors you can have is pretty limited (which might be fine for you need; and this true even if they get garbage-collected unless you pool them.) In actually, actors don't need a thread per actor, just a thread per message concurrently delivered, which can be a lot less. Of course, it's a lot messier too, though.
I did look at Twisted. It promises all of the things I need for distributed programming, but it didn't deliver (for me). Anytime you pull a 3rd party dependency into your code, you have to weight the cost against the benefit. With Twisted, the cost is pretty big because it's a big, complex thing and you can't really work with part of it without using all of it. The benefit wasn't there for me either, since many of their design choices didn't match my needs.
ReplyDeleteSelective receive is very nice, especially when mixed with power pattern matching.
ReplyDeleteI started to add it, but it got complex pretty quickly, and I thought "am I ever going to really need this?". So I left it out. There have been a few times when I wish I had it, but on the whole, I haven't needed it.
The big difficulty is that to really get the power of selective receive you need to have pattern matching, but I haven't yet come up with an elegant way to do pattern matching in python.
This is one of the ways that Erlang is way beyond other programming languages.
Yes, threads are expensive. One thread per actor doesn't scale very much. So, I've also created and experimental threadless actor library. I'll post about it soon.
ReplyDeleteI'd like to try and combine threadless and threaded actors so that you have exactly as many threads as you need. The Scala guys wrote a good paper about it a few years ago, but no one seems to have done much with it yet.
And what about stackless ? http://www.stackless.com/
ReplyDeleteI copied the selective semantics that a couple of other Ruby Actor packages have: match the message against an array of objects: if the object is not a class, do an == match. If the object is a class, do an instanceof match. The message itself can have more elements than the pattern, which are ignored.
ReplyDeleteNot the same as Erlang, which can, for example, allow you to require two elements of a message be the same, but so far seems to be working for folks. I figure at some point, we'll need a catch-all case a bit more like Erlang guards for situations where that kind of matching doesn't work.
I've been looking at thread allocations for actors. Right now, I have two conflicting requirements that I'm trying to balance. On the one hand, one reason I want actors to execute in separate threads is that I want them to be able to block on I/O without blocking the rest of the world. Given that I'm not willing to restrict what an actor method can do, this pretty much requires giving them their own thread, at least for the duration of that call.
On the flip side, not restricting the number of threads allows things to go bad quickly in some cases. Doug Lea did a paper on needing to restrict threads in Java for embarrassingly parallel things like naive fib. So far I haven't seen that problem in real code, so it's not on the top of my list (though it will probably fall out of some other work to make dramatis work with GUI libraries which require single-threaded access.)
I'll be interested to see how your threadless stuff works and combines with threaded.
The Scala guys have some really good papers about combining threadless and threaded actors: http://lamp.epfl.ch/~phaller/doc/haller07coord.pdf and http://lampwww.epfl.ch/~odersky/papers/jmlc06.pdf. There approach is the one I'm going to take.
ReplyDeleteBasically, you have threadless actors that "schedule" their actions rather than executing them directly. Then, you have a "controller" (threaded) actor that hands those actions out to worker (threaded) actors. If any worker actor blocks for more than a specific amount of time, you spawn another worker actor. This way, you only spawn as many threaded actors as necessary, but many blocking IO calls can continue to run in parallel.
Your pattern matching technique sounds a lot like candygram: http://candygram.sourceforge.net/. Have you seen it? I think it's pretty good and probably as good as you can get in Python.
ReplyDeleteYes, I have seen stackless. If they are successful, it would make Python MUCH better for doing things like continuations, monads, message-passing, etc. I'm very much hoping for their success. But it's still too immature to rely on for
ReplyDeletethe application I'm writing.
The blog has a to an extraordinary degree restrain focus that opens up, shockingly into broad assortment of concerns.
ReplyDelete123movieshub.it
Thank you for the update, very nice site.. celebrities verified
ReplyDeleteLove to read it,Waiting For More new Update and I Already Read your Recent Post its Great Thanks. celebrities verified
ReplyDeleteGreat Article
ReplyDeleteFinal Year Projects for CSE in Python
FInal Year Project Centers in Chennai
Python Training in Chennai
Python Training in Chennai
Mua vé máy bay tại đại lý Aivivu, tham khảo
ReplyDeletevé máy bay đi Mỹ Vietnam Airline
vé máy bay từ los angeles về việt nam
khi nào có chuyến bay từ đức về việt nam
chuyến bay từ nga về việt nam hôm nay
ReplyDeleteThanks for sharing this informative content.,
Leanpitch provides online training in Scrum Master Certification during this lockdown period everyone can use it wisely.
Join Leanpitch 2 Days CSM Certification Workshop in different cities.
CSM online training
CSM training online
Now it very well may be used for intermittent family unit use to standard every day printer in the office.if you need all data
ReplyDeleteThis content helped me alot you can check my article here. Flutter App development company
ReplyDeleteIn my opinion you shoud check the
ReplyDeleteclass Stopped:
def __repr__(self):
return "Stopped()"
class ActorNotStartedError(Exception):
def __init__(self):
Exception.__init__(self, "actor not started")
there is something missing in the operators
Additional thick for warmth, yet breathable for use while working out, our men's coats and covers are made with the best cotton wool material.
ReplyDeleteSuch a great blog .I am very happy to read this blog .This blog is really informative and knowledgeable for your readers .Thanks for sharing this blog and good information.
ReplyDeleteNow it's time to avail town car service Houston for more information.
تعتبر اسعار ماكينات تعبئة وتغليف البهارات ليست بالثمينه مقارنةً بباقي اسعار ماكينات التعبئة والتغليف، حيث يمكنك اقامة مشروع تعبئة وغتغليف متكامل بسبب هذه الماكينة فتقوم بتعبئة مختلف انواع البودر مثل البهارات والتوابل والدقيق والملح والخ الخ، كما تقدم ماسترتك ضمان صيانة دورية للماكينة لتحافظ الماكينة علي ادائها و جودتها العالية.
ReplyDeleteA study conducted by the University of Florida in 2005 found that more than 50% of college students had experienced some form of stress-related injury, which included ankle sprains. According to the American Academy of Orthopedic Surgeons, between 20 and 30 percent of people who have ankle sprains suffer from chronic ankle instability (CAI), which is defined as recurring ankle sprains and pain.
ReplyDeleteThe procedure will go smoothly from beginning to end becauseCEMENT TREATED BASE services in USA of our skilled team of contractors' attention to detail. Because we are aware of how crucial projects with a short turnaround time
ReplyDeleteThis competition aims to promote brands404 esports in Dubai while fast raising brand recognition.
ReplyDeleteDue to TACB's considerable experience with a range Offshore credit limits in Dubaiof financial instruments, the sale will be completed without a hitch.
ReplyDelete, and chemical injection solutions tochemical injection pumps in Dubai the international oil and gas industry
ReplyDelete. Through our business focused approach, we PPC marketing dubaithrive to deliver state of the art
ReplyDeleteWe're confident that you'll find the ideal piece of chocolatebest chocolate in saudi to suit every mood with such a huge variety to choose from.
ReplyDeleteIf you're looking for a reliable and trustworthy service to "take my online exam", MyAssignmentHelp.sg is the perfect choice. The tutors are committed to delivering top-quality work and ensuring that you receive the best possible grades.
ReplyDeleteThis is such a great article. I am very glad to read this article, because it contains many useful information which helps us a lot. The article is very interesting for the readers. Keep sharing with us the more article. Now its time to avail Limo service West Palm Beach for more information.
ReplyDeletePython is such a highly programming used in future. Such an interesting and useful article is. It contains many important points. Thanks for sharing with us. Now its time to avail Limo service West Palm Beach for more information.
ReplyDeleteI am lookig for that type of python script thanks for sharing. I truly value this post because it's such a well-done piece. You offer information that is helpful. It really gave me some new information, and I do agree that reading blogs can help us learn more about a variety of subjects by allowing us to obtain solid information. I really appreciate this wonderful post that you have provided for us. I assure you this would be beneficial for most people. Now its time to avail party bus dc for more information.
ReplyDeleteYou are doing really good work. Python is very difficult language but you explained very well. I am impressed from your this work. Now its time to avail Sign and Blind Services in Eastbourne for more information.
ReplyDeleteYour blogs are really good and interesting. It is very great and informative. You always know the sender of the message; it's built right in. There's also "Bridge" that allows you to interact with actors from non-actor code bankruptcy lawyers virginia beach. I got a lots of useful information in your blog. Keeps sharing more useful blogs..
ReplyDeletePython programming is one of the latest programming. This is an admirable Python model that you mention in this blog post which is very informative. Now it's time to avail Jupiter Florida Airport for more information.
ReplyDeleteLeyes Violencia Doméstica Nueva JerseyImpressive insights! Looking forward to more engaging content like this.
ReplyDeleteNew York Divorce Waiting Period
ReplyDeleteThe game's gameplay is exhilarating, offering an immersive experience for both newcomers and seasoned gamers. Its intuitive controls make it easy to engage, and its depth of strategy provides endless creative problem-solving opportunities. The pacing is well-balanced, keeping excitement high without overwhelming players. The game's dedication to delivering a top-notch gaming experience makes it a must-play for fans of the genre.
The author regrets not having specific information or reviews for "valuedlessons" or any related entity up to September 2021. They suggest using search engines, social media platforms, or online communities and forums to find up-to-date information and user experiences about this relatively new or niche entity. They suggest using these resources to access relevant discussions and reviews related to "valuedlessons."abogado flsa en virginia
ReplyDeleteThe speaker acknowledges that they lack access to real-time information about "valuedlessons reviews" and recent reviews, as their knowledge only extends to September 2021. They encourage asking specific questions or requesting general answers based on their current knowledge, but acknowledge that they cannot access the internet to access recent reviews or information.Abogado FLSA
ReplyDeleteThe article "Message Passing Concurrency (Actor Model) in Python" offers a comprehensive introduction to the Actor Model, a crucial concept in concurrent programming. It breaks down key components and demonstrates their implementation in Python, providing practical examples and case studies. The article also offers suggestions for further reading and resources, making it an educational and well-written resource for those interested in concurrent programming in Python.
ReplyDeleteLeyes Matrimoniales de Nueva York Divorcio
This comment has been removed by the author.
ReplyDeleteThe author regrets not providing real-time information due to limited knowledge and access to current data. They suggest checking a reliable financial news website, using a stock market tracking app, or consulting a financial advisor for the latest market updates and recommendations on top stocks, percent gainers, and percent losers for the NYSE, Nasdaq, and AMEX. Hudson County driving without a license lawyer
ReplyDeleteExploring the intricacies of Message Passing Concurrency (Actor Model) in Python. 🐍💬 For a smoother ride in the programming world, just like your car after a visit to Car Wash Prices Ottawa – where cleanliness meets code! #PythonProgramming #Concurrency #OttawaCarWash #CodeCleanliness"
ReplyDeleteMessage Passing Concurrency, a Python method utilizing the Actor Model, enhances scalability and code organization by encapsulating state and behavior within actors. Python libraries like Pykka and Thespian make this model more accessible. Although it requires developers to consider program structure and message handling, when used correctly, it can improve application performance and responsiveness. Abogado de Conducción Imprudente Condado Hudson
ReplyDeleteThe article "Message Passing Concurrency (Actor Model) in Python" offers a comprehensive guide to implementing the Actor Model in Python for concurrency. It provides a clear explanation of the concept and practical applications, with clear code examples that can be helpful for developers. The author's strong grasp of the topic makes it a valuable resource for Python developers.Affordable Divorce Lawyers in Queens New York
ReplyDeleteExploring the realms of Message Passing Concurrency (Actor Model) in Python opens doors to innovative solutions. Just as we strive for efficiency in coding, ensure your vehicle gets a top-notch treatment withCar Wash Ottawa in services. Keep your code and your car in peak performance!
ReplyDeleteThe Actor Model in Python offers a scalable and intuitive parallel programming approach, promoting clean code and modular design. Its asyncio library facilitates efficient communication and enhances code reliability. This powerful paradigm balances simplicity and performance in Python concurrent programming. Motivos de Divorcio Nueva York
ReplyDeleteExploring Message Passing Concurrency, especially in Python, is a fascinating journey into parallel programmingCatering services Monohans Texas, where technology and innovation thrive, consider catering services to keep your programming teams nourished and productive. Just as effective message passing enhances concurrency, quality catering ensures your team's energy and focus during challenging coding sessions.
ReplyDelete
ReplyDeleteYour message-passing concurrency framework in Python is a practical solution, and the simplicity makes it accessible for various applications. Thanks for sharing this valuable tool – looking forward to exploring its potential in different projects!
División de el Activos Divorcio Nueva York
Exploring Message Passing Concurrency through the Actor Model in Python is a fascinating journey into parallelism and efficient communication. Dive into the realm of concurrent programming! Meanwhile, illuminate your living or working spaces with Omnium Electric DubaiDiscover the perfect synergy between cutting-edge technology and lighting solutions for a bright and innovative environment.
ReplyDeleteI gathered more useful information. Keep sharing more related blogs.abogado dui carroll va
ReplyDeleteMessage Passing Concurrency is a robust approach to concurrent programming that offers clear communication channels between processes, facilitating efficient collaboration among tasks and enhancing system performance. Its simplicity and reliability make it an effective choice for concurrent applications, managing shared resources and avoiding common concurrency issues. It is a valuable tool for developers building scalable and responsive systems.
ReplyDeletePrevention of Domestic Violence Act New Jersey
It helps manage shared resources, prevent race conditions, and enables efficient collaboration among different components. It's a valuable paradigm for building scalable and responsive concurrent systems. Motorcycle accidents
ReplyDeleteIt oversees shared assets, forestall race conditions, and empowers proficient cooperation among various parts. It's a significant worldview for building versatile and responsive simultaneous frameworks. bankruptcy chapter 7 attorneys near me || how to remove reckless driving from record in virginia || is new jersey a no fault state for divorce.
ReplyDeleteThe Actor Model in Python is a powerful paradigm in concurrent programming, providing clear explanations and practical examples for readers of all levels. It demonstrates how to implement the Actor Model in Python, equipping developers with tools for building scalable and resilient systems. This guide is essential for both aspiring and seasoned Python developers, highlighting the benefits of message passing concurrency.
ReplyDeletedomestic violence unfit parent in virginia
best digital marketing agency in chennai, Thank you to the blog creator for the amazing content! Your posts are a source of inspiration and knowledge. I appreciate the effort you put into sharing valuable insights with readers. Your dedication to creating informative and engaging content is truly commendable.
ReplyDeleteThank you to the creator of the blog for sharing valuable insights, sparking inspiration, and enriching the digital landscape with your unique perspective.pole fitness accessories
ReplyDeleteThank you to the brilliant mind behind the blog for sharing valuable insights, sparking inspiration, and enriching our lives with your knowledge and creativity. Gratitude abounds!chennai multi speciality hospital
ReplyDeleteThis is hands down one of the clearest explanations I’ve seen on this topic. The writer has a remarkable way of turning something that could be overwhelming into something anyone can understand. I didn’t expect to feel this confident about the subject after reading, but here I am! Major thanks to the writer for making it so easy to follow. Visit our link for ISO Certification in Saudi Arabia
ReplyDeleteThis post on message-passing concurrency is insightful! It offers a great perspective on the Actor model and its applications in programming. Visit our link for ISO Certification In Philippines
ReplyDeleteVery nice article and straight to the point. Keep it up, Thank you, so awesome!
ReplyDeleteI have read several excellent stuff here. Keep on writing great, thanks
ReplyDeleteI truly loved browsing this blog posts. Wonderful site you have here! Thanks
ReplyDeleteThis is the right blog. Great stuff you have here, just great! Thankyou!
ReplyDelete