Microservice communication with RabbitMQ - Tetiana Chupryna

•

0 recomendaciones•286 vistas

Ruby Meditation

Video: https://youtu.be/QogP24IONFc Ruby Meditation #19 November 25, 2017 Kyiv

Tecnología

Ruby Meditation #19
Microservice
communication with
RabbitMQ
Tetiana Chupryna

Татьяна Чуприна
Person of a million names

Agenda
❖ Inter-service communication problematics
❖ JSON over HTTP option
❖ Message queues option
❖ RabbitMQ
❖ Integration with Ruby

–Steven McConnell, Code complete
“Important developments often arise out of
analogies. By comparing a topic you understand
poorly to something similar you understand better,
you can come up with insights that result in a
better understanding of the less-familiar topic.”
The importance of Metaphors

Luggage for dummies
Drop-off
Sorting
Sorting

Luggage for dummies
Drop-off
Sorting
c a r o u s e l
Sorting

Microservice communication
??
?
?
? ?
?
?
?
?
?
??
?
? ?

Inter-service communication
❖ New problem which doesn’t exist in monolith
❖ So many ways to success
❖ So many pitfalls

JSON over HTTP
Re use API
at least Basic Auth
100% control

Sidekiq to the resque
❖ Handle “other side” failures gracefully
❖ Retry if needed

Procs
❖ Easy
❖ Quick
❖ Dirty
❖ Great choice for small apps

Cons
❖ Bad in scaling
❖ Pain for maintenance
❖ Net::HTTP

Break it. Make it.
❖ Decouple service interaction
❖ Keep messages in temporary storage (mailbox?)
❖ Make all asynchronous

Message queues
❖ Message-oriented middleware
❖ Applicable not only for web-applications
❖ Async communication protocol

Kafka
JBossMessaging
Oracle OpenMQ
mail.ru Tarantool

Reasons to use
❖ Written in Erlang
❖ Fast&Reliable
❖ Easy to setup
❖ Client libraries on all major languages (Ruby yippee!)
❖ Advanced Message Queuing Protocol

Queue properties
❖ Name
❖ Durable
❖ Exclusive
❖ Auto-delete
❖ Arguments

Bindings
❖ Direct: by routing_key
❖ Multiple bindings
❖ Mask routing

Security
❖ TLS (transport layer security)
❖ SSL
❖ authentication
❖ authorization

Setup Rabbit
❖ Install and run
❖ No painful dependancies

Bunny
❖ Ruby client for RabbitMQ
❖ Lightweight
❖ Up&go

Sneakers
❖ Background processing framework for Ruby and
RabbitMQ
❖ Can be used for high workload
❖ Up&go (as always)

Procs
❖ Scalable
❖ Maintainable
❖ Lightweight

Cons
❖ Need additional infrastructure eq(Money)
❖ Overkill for small projects
❖ Need additional knowledge

Afterwords
❖ Decoupling message ﬂow from app changes app design
and in general makes all easier.
❖ Decoupling message ﬂow makes programs more safe
and scalable.
❖ Message queues are a save choice for inter-service
communication where RabbitMQ is a solid option.
❖ Ça dépend

Share your stories of inter-service
communication

Thanks Thanks
ThanksThanks
Thanks Thanks
Thanks
Thanks
Thanks
Thanks
ThanksThanks
Thanks
Thanks
ThanksThanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
ThanksThanks
Thanks Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks
Thanks

Más contenido relacionado

Similar a Microservice communication with RabbitMQ - Tetiana Chupryna

I was presenting the new trends and possibilities for a decade in open source monitoring 3 years ago at OSMC. It's time now to see if we can make some monitoring without Nagios. We'll then explore: - Collectd, Diamond, Packetbeat, StatsD and Logstash for collecting metrics and events - Graphite, InfluxDB for storing time series data - Elasticsearch for storing events - Kibana and Grafana for dispaying and searching metrics and events. - Seyren and Cabot for notifications What is possible with such a solution? How does it work compared to Nagios…? Is it iso-functionnal, maybe better than a Nagios based solution? Is there any migration path from Nagios? We'll try to answer all theses questions and maybe more!

OSMC 2014: Time to say goodbye to your Nagios setup | Oliver Jan

NETWAYS

"Enterprise Messaging with RabbitMQ" by Luis Majano is a comprehensive presentation that explores the concept of enterprise messaging and demonstrates how RabbitMQ, a widely used message broker, can facilitate effective communication within complex software systems. This presentation aims to provide a deep understanding of messaging patterns, architectures, and the benefits of adopting RabbitMQ in enterprise-level applications. The presentation begins by introducing the fundamental principles of enterprise messaging and its significance in modern software development. It emphasizes the need for scalable, reliable, and decoupled communication between components and systems to ensure seamless operation and flexibility. Luis Majano, an expert in enterprise software development, then delves into the key features and capabilities of RabbitMQ. He explains how RabbitMQ enables the implementation of various messaging patterns such as publish/subscribe, request/reply, and message routing. The presentation highlights the versatility of RabbitMQ in handling different messaging scenarios and discusses its integration possibilities with different programming languages and frameworks. Moreover, the presentation addresses crucial concepts related to RabbitMQ, including exchanges, queues, bindings, and message acknowledgment. It explains how these components work together to enable message routing, filtering, and delivery guarantees in complex distributed systems. Luis Majano also demonstrates the practical implementation of RabbitMQ through code examples and architectural diagrams. He illustrates how to set up RabbitMQ clusters for high availability and scalability, and how to handle common messaging challenges such as message persistence, fault tolerance, and load balancing. Throughout the presentation, Luis Majano shares best practices, tips, and strategies for designing and implementing robust messaging solutions using RabbitMQ. He discusses message serialization, error handling, monitoring, and performance optimization techniques to ensure optimal message throughput and system responsiveness.

Enterprise Messaging with RabbitMQ.pdf

Ortus Solutions, Corp

Realtime web2012

Timothy Fitz

Rabbit MQ

Tran Thanh Thi

Rigadevdays - Communication in a microservice architecture

Irina Scurtu

P2P for mobile devices

immanuelnoel

Evaluating persistent, replicated message queues

Adam Warski

The Java microservice lib. QBit is a reactive programming lib for building microservices - JSON, HTTP, WebSocket, and REST. QBit uses reactive programming to build elastic REST, and WebSockets based cloud friendly, web services. SOA evolved for mobile and cloud. QBit is a Java first programming model. It uses common Java idioms to do reactive programming. It focuses on Java 8. It is one of the few of a crowded field of reactive programming libs/frameworks that focuses on Java 8. It is not a lib written in XYZ that has a few Java examples to mark a check off list. It is written in Java and focuses on Java reactive programming using active objects architecture which is a focus on OOP reactive programming with lambdas and is not a pure functional play. It is a Java 8 play on reactive programming. Services can be stateful, which fits the micro service architecture well. Services will typically own or lease the data instead of using a cache. CPU Sharded services, each service does a portion of the workload in its own thread to maximize core utilization. The idea here is you have a large mass of data that you need to do calculations on. You can keep the data in memory (fault it in or just keep in the largest part of the histogram in memory not the long tail). You shard on an argument to the service methods. (This was how I wrote some personalization engine in the recent past). Worker Pool service, these are for IO where you have to talk to an IO service that is not async (database usually or legacy integration) or even if you just have to do a lot of IO. These services are semi-stateless. They may manage conversational state of many requests but it is transient. ServiceQueue wraps a Java object and forces methods calls, responses and events to go through high-speed, batching queues. ServiceBundle uses a collection of ServiceQueues. ServiceServer uses a ServiceBundle and exposes it to REST/JSON and WebSocket/JSON. Events are integrated into the system. You can register for an event using an annotation @EventChannel, or you can implement the event channel interface. Event Bus can be replicated. Event busses can be clustered (optional library). There is not one event bus. You can create as many as you like. Currently the event bus works over WebSocket/JSON. You could receive events from non-Java applications. Find out more at: https://github.com/advantageous/qbit

The Java Microservice Library

Rick Hightower

API Design and WebSocket

Frank Greco

Microservices: Benefits, drawbacks and are they for me?

Marian Marinov

NoSQL afternoon in Japan Kumofs & MessagePack

Sadayuki Furuhashi

NoSQL afternoon in Japan kumofs & MessagePack

Sadayuki Furuhashi

Ride The Bus!

Mogens Heller Grabe

Bereits vor drei Jahren habe ich auf der OSMC über die neuen Trends und Möglichkeiten im Bereich Monitoring für dieses Jahrzehnt gesprochen. Jetzt ist es an der Zeit, die besten Alternativen und Ergänzungen eines auf Nagios basierten Setups im Detail kennenzulernen. Graphite, InfluxDB für zeitserien Daten Grafana als Interface für Graphite ELK für System- und Applikationsmeldungen und Logverarbeitung Skyline und Oculus zur Anomalieerkennung Seren und Cabot für Benachrichrtungen Was ist aktuell mit solch einer Lösung möglich und wie gestaltet sich der Vergleich zu Nagios?

OSMC 2014 | Time to say goodbye to your Nagios based setup? by Oliver Jan

NETWAYS

Django in enterprise world

Simone Federici

Similar a Microservice communication with RabbitMQ - Tetiana Chupryna (15)

OSMC 2014: Time to say goodbye to your Nagios setup | Oliver Jan

Enterprise Messaging with RabbitMQ.pdf

Realtime web2012

Rabbit MQ

Rigadevdays - Communication in a microservice architecture

P2P for mobile devices

Evaluating persistent, replicated message queues

The Java Microservice Library

API Design and WebSocket

Microservices: Benefits, drawbacks and are they for me?

NoSQL afternoon in Japan Kumofs & MessagePack

NoSQL afternoon in Japan kumofs & MessagePack

Ride The Bus!

OSMC 2014 | Time to say goodbye to your Nagios based setup? by Oliver Jan

Django in enterprise world

Más de Ruby Meditation

Talk of Sergey Sergyenko, CEO at Cybergizer, at Ruby Meditation #30 Online 24.10.2020 Next conference - http://www.rubymeditation.com/ Description: I hope that the code I write continues to be used after I die. Which is why I only ever write legacy code" - this is what Patterson mentioned on his Twitter last September. In this talk, I will define metrics and try to distinguish code from Legacy to the one that Survived. Also, I will introduce the so-called "Code Survival" refactoring approach. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30

Ruby Meditation

Talk of Roman Dubrovsky, Backend developer at Datarockets, at Ruby Meditation #29 Kyiv 14.12.2019 Next conference - http://www.rubymeditation.com/ Description: More than two years we’re developing and supporting GraphQL API, tried to make it public and integrated our app with other GraphQL APIs. In this talk, I'll try to summarize our experience, say about good practices, issues we found and can resolve. And which we can’t. I will show you how to set up and how to use them. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...

Ruby Meditation

Talk of Viacheslav Miroshnychenko, Software Engineer at Human Agency, at Ruby Meditation #29 Kyiv 14.12.2019 Next conference - http://www.rubymeditation.com/ Description: Why do we need to think about licenses and check them in our projects? Let's try to figure out it. During this talk, you will learn tools that help you to automate the process of checking licenses for all libraries in your project. I will show you how to set up and how to use them. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29

Ruby Meditation

Talk of Aleksandra Stolyar, Ruby developer at DataArt, at Ruby Meditation #29 Kyiv 14.12.2019 Next conference - http://www.rubymeditation.com/ Description: I will talk about dry-rb ecosystem and it’s major components - dry-validation and dry-schema which are very helpful for validations. This year dry-rb introduced a major update to its’ gems and I faced some problems when decided to marry these updates with a project. This talk will cover some of differences and specifics of dry-validation and dry-schema updates. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...

Ruby Meditation

Talk of Bohdan Parshentsev, senior backend engineer at P2H, at Ruby Meditation #28 Kyiv 26.10.2019 Slideshare - ** Next conference - http://www.rubymeditation.com/ This presentation is connected with presentation How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28 Why we need a Rabbit? Let's try to cook it tasty. And what else? - Live demo of several useful dishes. Be ready to use Telegram to have fun with us. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation * The channel of the organizers of the meetup https://t.me/incredevly

How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28

Ruby Meditation

Talk of Serhiy Nazarov, founder QPARD, system architect, at Ruby Meditation #28 Kyiv 26.10.2019 Next conference - http://www.rubymeditation.com/ This presentation is connected with presentation How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 Why we need a Rabbit? Let's try to cook it tasty. And what else? - Live demo of several useful dishes. Be ready to use Telegram to have fun with us. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28

Ruby Meditation

Talk of Julik Tarkhanov, senior backend engineer, WeTransfer, Amsterdam, at Ruby Meditation #28 Kyiv 26.10.2019 Next conference - http://www.rubymeditation.com/ It is often a choice, sometimes a whim, and sometimes an act of desperation. We idolise reuse while sometimes the road not taken is just as exciting. Let's chat about where it is appropriate to "do the thing again", take the scenic route and enjoy the view. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation * The channel of the organizers of the meetup https://t.me/incredevly

Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...

Ruby Meditation

Speech of Nihad Abbasov, Senior Software Engineer at Digital Classifieds, at Ruby Meditation 27, Dnipro, 19.05.2019 Slideshare - Next conference - http://www.rubymeditation.com/ How fast is your code? Performance is crucial as your startup grows, and optimizing your application can make a huge impact on user experience. During this talk, you will learn hints, techniques and best practices for improving the overall speed of your Ruby application. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...

Ruby Meditation

Speech of Ruslan Tolstov, Ruby Developer at MLSDev, at Ruby Meditation 27, Dnipro, 19.05.2019 Slideshare - Next conference - http://www.rubymeditation.com/ During this talk, we’ll discuss serverless technologies. In particular, we will talk about popular serverless platforms like AWS Lambda, Google CF, and Microsoft AF and also dwell upon serverless frameworks for managing and deploying your functions. As well, we will touch upon the potential use cases of this new approach in your existing projects. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...

Ruby Meditation

Speech of Orban Botond, Ruby Developer at Toptal, at Ruby Meditation 27, Dnipro, 19.05.2019 Slideshare - Next conference - http://www.rubymeditation.com/ Software development is a domain where everybody can make a beautiful sculpture or can quickly build an unsolvable maze. According to my observations the later happens more often unfortunately. In the title for my presentation both the If Jungle and the Civilised Railway Station are methaphors representing the opposite ends of the scale of quality code. In my presentation I am going to present my personal experiences on how to get out from the trap of the if jungle by making the code to adhere to the SRP and DRY principles. I am also going to show the advantages of the Railway Oriented Programing using the 3 different libraries. The examples are going to be stereotypical errors, fun and easy to follow. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...

Ruby Meditation

Speech of Valentyn Ostakh, Ruby Developer at Ruby Garage, at Ruby Meditation 27, Dnipro, 19.05.2019 Slideshare - Next conference - http://www.rubymeditation.com/ This talk explores basic concepts of GraphQL. The main goal is to show how GraphQL works and of what parts it consists of. From the Ruby side we will look at how to create a GraphQL schema. In addition, we will consider what pitfalls can be encountered at the start of work with GraphQL. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27

Ruby Meditation

Speech of Nihad Abbasov, Backend Engineer at Digital Classifieds, at Ruby Meditation #26 Kyiv 16.02.2019 Next conference - http://www.rubymeditation.com/ What to expect from Rails 6? Review of new features. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26

Ruby Meditation

Speech of Tetiana Chupryna, Backend developer at GitLab, at Ruby Meditation #26 Kyiv 16.02.2019 Next conference - http://www.rubymeditation.com/ We’ll talk about different types of vulnerabilities, scanning tools and the whole process per se. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26

Ruby Meditation

Speech of Dmytro Shapovalov, Infrastructure Engineer at Cossack Labs, at Ruby Meditation #26 Kyiv 16.02.2019 Next conference - http://www.rubymeditation.com/ Most modern applications live in a close cooperation with each other. We will talk about the ways to effectively use the modern techniques for monitoring the health of applications and look on tasks and typical implementation mistakes through the eyes of an infrastructure engineer. And we will also consider the Ruby libraries that help to implement all of this. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...

Ruby Meditation

Speech of Ike Kurghinyan, Senior Software Engineer at VChain LLC, at Ruby Meditation #26 Kyiv 16.02.2019 Next conference - http://www.rubymeditation.com/ «Is there one grand unifying rule to software engineering? Can so-called best practices be somehow derived from that unifying rule? This speech is a humble attempt to explore the subject and provide you with tools for your own explorations.» Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation * The channel of the organizers of the meetup https://t.me/incredevly

Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26

Ruby Meditation

Speech of Alexey Vasiliev, Software Engineer at Railsware, at Ruby Meditation #25 Kyiv 08.12.2018 Next conference - http://www.rubymeditation.com/ In this talk, Alexey will tell about the project in which was necessary to implement A/B testing and what came out of it in result Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25

Ruby Meditation

Speech of Dmytro Herasymuk, WEB developer at Softermii, at Ruby Meditation #25 Kyiv 08.12.2018 Next conference - http://www.rubymeditation.com/ The modern world demands to be faster and faster. Engineers create more powerful CPUs every day, but our app depends on third party apps too often in the WEB world. So, even the fastest CPU can't cope with I/O delays. On the one hand multithreading could help in such cases, but on the other hand, it seems to be a hard decision of using concurrency in production. Dmytro would like to share his experience in this field Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...

Ruby Meditation

Speech of Dmytro Shapovalov, Infrastructure Engineer at Cossack Labs, at Ruby Meditation #25 Kyiv 08.12.2018 Next conference - http://www.rubymeditation.com/ Making secure applications is not easy, especially when encryption tools are difficult and incomprehensible. We will talk about typical data security problems in web apps and how to implement encryption properly. We will review cryptographic approaches and exact tools that ensure that no sensitive data leaks from the application or database. Announcements and conference materials https://www.fb.me/RubyMeditation News https://twitter.com/RubyMeditation Photos https://www.instagram.com/RubyMeditation The stream of Ruby conferences (not just ours) https://t.me/RubyMeditation

Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...

Ruby Meditation

Rails App performance at the limit - Bogdan Gusiev

Ruby Meditation

GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23

Ruby Meditation

Más de Ruby Meditation (20)