SlideShare a Scribd company logo
1 of 21
Inter-Process/Task Communication With Message Queues William McVey < [email_address] > PyOhio July 26, 2009
Intro ,[object Object],[object Object],[object Object]
Description of the Problem ,[object Object],[object Object],[object Object],[object Object]
Architecture
Key Difficulties ,[object Object],[object Object],[object Object],[object Object]
Lesson learned Do  not  screw with Apache's process model.
Rediscovering Queues ,[object Object],[object Object],[object Object],[object Object],[object Object]
Generic message broker ,[object Object],[object Object],[object Object],[object Object],[object Object]
beanstalkd/beanstalkc ,[object Object],[object Object],[object Object]
The need for something more ,[object Object],[object Object],[object Object],[object Object]
memcacheq ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
AMQP ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
AMQP
AMQP Message Routing Image from:  Messaging Tutorial - AMQP Programming Tutorial for C++, Java, Python, and C# Copyright © 2008 Red Hat, Inc. Under the Open Publication License
ØMQ -  http://zeromq.org/ ,[object Object],[object Object],[object Object],[object Object],[object Object]
RabbitMQ ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
amqplib / carrot py-amqplib  is a client library around the AMQP protocol.Fairly low level for my needs though, so a little digging found  carrot
carrot sample >>>   from   carrot.messaging   import  Publisher, Consumer >>>   class   PostOfficePublisher (Publisher): ...   exchange  =   &quot;sorting_room&quot; ...   routing_key  =   &quot;jason&quot; >>>   class   PostOfficeConsumer (Consumer): ...   queue  =   &quot;po_box&quot; ...   exchange  =   &quot;sorting_room&quot; ...   routing_key  =   &quot;jason&quot; ... ...   def   receive ( self , message_data, message): ...   &quot;&quot;&quot;Called when we receive a message.&quot;&quot;&quot; ...   print ( &quot;Received:  %s &quot;   %  message_data)
carrot sample >>>   from   ConfigParser   import  ConfigParser >>>  config  =  ConfigParser() >>>  config . read( &quot;application.ini&quot; ) >>>   from   carrot.connection   import  AMQPConnection >>>  amqpconn  =  AMQPConnection( ...   hostname  =  config . get( &quot;broker&quot; ,  &quot;host&quot; ), ...   port  =  config . get( &quot;broker&quot; ,  &quot;port&quot; ), ...   userid  =  config . get( &quot;broker&quot; ,  &quot;userid&quot; ), ...   password  =  config . get( &quot;broker&quot; ,  &quot;password&quot; ), ...   vhost  =  config . get( &quot;broker&quot; ,  &quot;vhost&quot; )) >>>  PostOfficePublisher(connection = amqpconn) . send( ...   { &quot;My message&quot; : [ &quot;foo&quot; ,  &quot;bar&quot; ,  &quot;baz&quot; ]}) >>>  PostOfficeConsumer(connection = amqpconn) . next() Received: { &quot;My message&quot; : [ &quot;foo&quot; ,  &quot;bar&quot; ,  &quot;baz&quot; ]}
multiprocessing ,[object Object],[object Object],[object Object],[object Object],[object Object]
In Summary ,[object Object],[object Object],[object Object],[object Object],[object Object]

More Related Content

What's hot

Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring frameworkAssurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring frameworkGosuke Miyashita
 
XMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol StudyXMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol StudyValérian Saliou
 
NetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology OverviewNetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology Overviewkvamsi
 
How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...Gosuke Miyashita
 
Communication in Python and the C10k problem
Communication in Python and the C10k problemCommunication in Python and the C10k problem
Communication in Python and the C10k problemJose Galarza
 
Expand Your Testing with Virtual Services
Expand Your Testing with Virtual ServicesExpand Your Testing with Virtual Services
Expand Your Testing with Virtual ServicesAmber Race
 
LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Godreamwidth
 
Real-Time Web applications with WebSockets
Real-Time Web applications with WebSocketsReal-Time Web applications with WebSockets
Real-Time Web applications with WebSocketsStanislav Zozulia
 
What To Learn First in PHP
What To Learn First in PHPWhat To Learn First in PHP
What To Learn First in PHPTessa Mero
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comIlya Grigorik
 
Debugging over tcp and http
Debugging over tcp and httpDebugging over tcp and http
Debugging over tcp and httpKaniska Mandal
 
parenscript-tutorial
parenscript-tutorialparenscript-tutorial
parenscript-tutorialtutorialsruby
 
Augmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPPAugmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPPSam Keen
 
Locker: distributed consistent locking
Locker: distributed consistent lockingLocker: distributed consistent locking
Locker: distributed consistent lockingKnut Nesheim
 
Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.Evgeniy Kuzmin
 

What's hot (20)

Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring frameworkAssurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
 
XMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol StudyXMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol Study
 
Perl comet
Perl cometPerl comet
Perl comet
 
NetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology OverviewNetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology Overview
 
How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...
 
Communication in Python and the C10k problem
Communication in Python and the C10k problemCommunication in Python and the C10k problem
Communication in Python and the C10k problem
 
Expand Your Testing with Virtual Services
Expand Your Testing with Virtual ServicesExpand Your Testing with Virtual Services
Expand Your Testing with Virtual Services
 
Cramp websockets
Cramp websocketsCramp websockets
Cramp websockets
 
LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Go
 
Real-Time Web applications with WebSockets
Real-Time Web applications with WebSocketsReal-Time Web applications with WebSockets
Real-Time Web applications with WebSockets
 
What To Learn First in PHP
What To Learn First in PHPWhat To Learn First in PHP
What To Learn First in PHP
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
 
Debugging over tcp and http
Debugging over tcp and httpDebugging over tcp and http
Debugging over tcp and http
 
parenscript-tutorial
parenscript-tutorialparenscript-tutorial
parenscript-tutorial
 
Php Debugger
Php DebuggerPhp Debugger
Php Debugger
 
Augmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPPAugmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPP
 
Gearman and Perl
Gearman and PerlGearman and Perl
Gearman and Perl
 
A few words about WAMP
A few words about WAMPA few words about WAMP
A few words about WAMP
 
Locker: distributed consistent locking
Locker: distributed consistent lockingLocker: distributed consistent locking
Locker: distributed consistent locking
 
Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.
 

Viewers also liked

DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)Gerardo Pardo-Castellote
 
IPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared MemoryIPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared MemoryHEM DUTT
 
OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)Shteryana Shopova
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsPushkar Chivate
 
Distributed messaging with AMQP
Distributed messaging with AMQPDistributed messaging with AMQP
Distributed messaging with AMQPWee Keat Chin
 
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFOWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFPaul Mooney
 
Docker - container and lightweight virtualization
Docker - container and lightweight virtualization Docker - container and lightweight virtualization
Docker - container and lightweight virtualization Sim Janghoon
 
Crypto With OpenSSL
Crypto With OpenSSLCrypto With OpenSSL
Crypto With OpenSSLZhi Guan
 
Interprocess Communication
Interprocess CommunicationInterprocess Communication
Interprocess CommunicationDeepak H L
 
The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP Eberhard Wolff
 
Inter process communication
Inter process communicationInter process communication
Inter process communicationRJ Mehul Gadhiya
 
Interprocess communication
Interprocess communicationInterprocess communication
Interprocess communicationSushil Singh
 
Microservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitectureMicroservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitecturePaul Mooney
 
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffArchitecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffJAX London
 
Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014P. Taylor Goetz
 
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computationStorm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computationnathanmarz
 
Realtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and HadoopRealtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and HadoopDataWorks Summit
 

Viewers also liked (20)

DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)
 
IPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared MemoryIPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared Memory
 
Inter Process Communication
Inter Process CommunicationInter Process Communication
Inter Process Communication
 
OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure tools
 
Distributed messaging with AMQP
Distributed messaging with AMQPDistributed messaging with AMQP
Distributed messaging with AMQP
 
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFOWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
 
Docker - container and lightweight virtualization
Docker - container and lightweight virtualization Docker - container and lightweight virtualization
Docker - container and lightweight virtualization
 
Crypto With OpenSSL
Crypto With OpenSSLCrypto With OpenSSL
Crypto With OpenSSL
 
Interprocess Communication
Interprocess CommunicationInterprocess Communication
Interprocess Communication
 
The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP
 
Inter process communication
Inter process communicationInter process communication
Inter process communication
 
Interprocess communication
Interprocess communicationInterprocess communication
Interprocess communication
 
Microservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitectureMicroservice vs. Monolithic Architecture
Microservice vs. Monolithic Architecture
 
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffArchitecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
 
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache StormResource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
 
Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014
 
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computationStorm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computation
 
Realtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and HadoopRealtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and Hadoop
 
Yahoo compares Storm and Spark
Yahoo compares Storm and SparkYahoo compares Storm and Spark
Yahoo compares Storm and Spark
 

Similar to Inter-Process/Task Communication With Message Queues

Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet
 
Enterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQEnterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQelliando dias
 
Service Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMixService Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMixBruce Snyder
 
quickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmqquickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmqjorgesimao71
 
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...zznate
 
The Art of Message Queues - TEKX
The Art of Message Queues - TEKXThe Art of Message Queues - TEKX
The Art of Message Queues - TEKXMike Willbanks
 
Hands on with CoAP and Californium
Hands on with CoAP and CaliforniumHands on with CoAP and Californium
Hands on with CoAP and CaliforniumJulien Vermillard
 
JMS Introduction
JMS IntroductionJMS Introduction
JMS IntroductionAlex Su
 
AMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARISAMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARISKel
 
What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31Ronald Hsu
 
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1Axiros
 
Introduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ ArtemisIntroduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ ArtemisYoshimasa Tanabe
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Miguel Zuniga
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Puppet
 
Fast SOA with Apache Synapse
Fast SOA with Apache SynapseFast SOA with Apache Synapse
Fast SOA with Apache SynapsePaul Fremantle
 
Implementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMixImplementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMixAdrian Trenaman
 
AWS Lambda from the trenches
AWS Lambda from the trenchesAWS Lambda from the trenches
AWS Lambda from the trenchesYan Cui
 

Similar to Inter-Process/Task Communication With Message Queues (20)

Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
 
Enterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQEnterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQ
 
Service Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMixService Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMix
 
quickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmqquickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmq
 
Tbp
TbpTbp
Tbp
 
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
 
The Art of Message Queues - TEKX
The Art of Message Queues - TEKXThe Art of Message Queues - TEKX
The Art of Message Queues - TEKX
 
Hands on with CoAP and Californium
Hands on with CoAP and CaliforniumHands on with CoAP and Californium
Hands on with CoAP and Californium
 
JMS Introduction
JMS IntroductionJMS Introduction
JMS Introduction
 
AMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARISAMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
 
What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31
 
Tomcat 6: Evolving our server
Tomcat 6: Evolving our serverTomcat 6: Evolving our server
Tomcat 6: Evolving our server
 
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
 
2 Asp Dot Net Ajax Extensions
2 Asp Dot Net Ajax Extensions2 Asp Dot Net Ajax Extensions
2 Asp Dot Net Ajax Extensions
 
Introduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ ArtemisIntroduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ Artemis
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014
 
Fast SOA with Apache Synapse
Fast SOA with Apache SynapseFast SOA with Apache Synapse
Fast SOA with Apache Synapse
 
Implementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMixImplementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMix
 
AWS Lambda from the trenches
AWS Lambda from the trenchesAWS Lambda from the trenches
AWS Lambda from the trenches
 

Recently uploaded

My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 

Recently uploaded (20)

My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 

Inter-Process/Task Communication With Message Queues

  • 1. Inter-Process/Task Communication With Message Queues William McVey < [email_address] > PyOhio July 26, 2009
  • 2.
  • 3.
  • 5.
  • 6. Lesson learned Do not screw with Apache's process model.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13. AMQP
  • 14. AMQP Message Routing Image from: Messaging Tutorial - AMQP Programming Tutorial for C++, Java, Python, and C# Copyright © 2008 Red Hat, Inc. Under the Open Publication License
  • 15.
  • 16.
  • 17. amqplib / carrot py-amqplib is a client library around the AMQP protocol.Fairly low level for my needs though, so a little digging found carrot
  • 18. carrot sample >>> from carrot.messaging import Publisher, Consumer >>> class PostOfficePublisher (Publisher): ... exchange = &quot;sorting_room&quot; ... routing_key = &quot;jason&quot; >>> class PostOfficeConsumer (Consumer): ... queue = &quot;po_box&quot; ... exchange = &quot;sorting_room&quot; ... routing_key = &quot;jason&quot; ... ... def receive ( self , message_data, message): ... &quot;&quot;&quot;Called when we receive a message.&quot;&quot;&quot; ... print ( &quot;Received: %s &quot; % message_data)
  • 19. carrot sample >>> from ConfigParser import ConfigParser >>> config = ConfigParser() >>> config . read( &quot;application.ini&quot; ) >>> from carrot.connection import AMQPConnection >>> amqpconn = AMQPConnection( ... hostname = config . get( &quot;broker&quot; , &quot;host&quot; ), ... port = config . get( &quot;broker&quot; , &quot;port&quot; ), ... userid = config . get( &quot;broker&quot; , &quot;userid&quot; ), ... password = config . get( &quot;broker&quot; , &quot;password&quot; ), ... vhost = config . get( &quot;broker&quot; , &quot;vhost&quot; )) >>> PostOfficePublisher(connection = amqpconn) . send( ... { &quot;My message&quot; : [ &quot;foo&quot; , &quot;bar&quot; , &quot;baz&quot; ]}) >>> PostOfficeConsumer(connection = amqpconn) . next() Received: { &quot;My message&quot; : [ &quot;foo&quot; , &quot;bar&quot; , &quot;baz&quot; ]}
  • 20.
  • 21.