As your application grows, you soon realise you need to break up your application into smaller chunks that talk to each other. You could just use web services to interact, or you could take a more robust approach and use the message broker RabbitMQ. In this talk, we will take a look at the techniques you can use to vastly enhance inter-application communication, learn about the core concepts of RabbitMQ, cover how you can scale different parts of your application separately, and modernise your development using a message-oriented architecture.

1. Adding 1.21 Gigawatts to
Applications with RabbitMQ
James Titcumb
Dutch PHP Conference 2015

2. James Titcumb
www.jamestitcumb.com
www.roave.com
www.phphants.co.uk
www.phpsouthcoast.co.uk
@asgrim
Who is this guy?

3. What is a message?

4. What is message
queueing?

5. Separation of Concerns

6. Scaling with Rabbit
RabbitMQApplication
Background processing

7. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing

8. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing

9. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing

10. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing
Background processing

11. Real world uses?

12. SOA

13. Why RabbitMQ?

14. https://github.com/asgrim/rmq-slides
Follow along here...

15. (on trusty64 Vagrant, other OSs may vary)
Installing RabbitMQ

16. ● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
● apt-get update
● apt-get install rabbitmq-server
● rabbitmq-plugins enable rabbitmq_management
● sudo service rabbitmq-server restart
Using Apt

17. http://localhost:15672/

18. http://localhost:15672/

19. http://localhost:15672/

20. http://localhost:15672/

21. http://localhost:15672/

22. Basic Message Queue

23. Objective: Basic Queuing
Producer Consumer
test_queue
1 2 3 4 5

24. composer.json
composer require videlalvaro/php-amqplib
Approved

25. Please wait, connecting...
1 use PhpAmqpLibConnectionAMQPConnection;
2
3 $connection = new AMQPConnection(
4 'localhost', 5672, 'guest', 'guest', '/');
5
6 $channel = $connection->channel();

26. basic/producer.php
1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->queue_declare(
4 'test_queue', false, true, false, false);
5
6 $message = new AMQPMessage('my test message');
7 $channel->basic_publish($message, '', 'test_queue');

27. basic/consumer.php
1 $channel->basic_consume(
2 'test_queue', '', false, true, false, false,
3 function(AMQPMessage $message) {
4 echo $message->body . "n";
5 });
6
7 while (count($channel->callbacks))
8 $channel->wait();

28. What to expect...

29. Exchanges: Fanout

30. Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5

31. 1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->exchange_declare(
4 'test_exchange', 'fanout', false, false, false);
5
6 $message = new AMQPMessage('test msg #' . $id);
7 $channel->basic_publish($message, 'test_exchange');
fanout/producer.php

32. fanout/consumer.php
1 $q = $channel->queue_declare(
2 '', false, false, false, true);
3 $queue_name = $q[0];
4
5 $channel->exchange_declare(
6 'test_exchange', 'fanout', false, false, false);
7 $channel->queue_bind($queue_name, 'test_exchange');

33. What to expect...

34. A word on Temporary Queues
test_exchangeProducer
Messages
go nowhere

35. Exchanges: Direct

36. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
BK = orange, banana,
apple
Consumer

37. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= ORANGE
BK = orange, banana,
apple
Consumer

38. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= BANANA
BK = orange, banana,
apple
Consumer

39. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= APPLE
BK = orange, banana,
apple
Consumer

40. direct/producer.php
1 $channel->exchange_declare(
2 'test_direct', 'fanout', false, false, false);
3
4 $messageContent = 'test msg, key=' . $routingKey;
5 $message = new AMQPMessage($messageContent);
6 $channel->basic_publish(
7 $message, 'test_direct', $routingKey);

41. direct/consumer.php
1 $q = $channel->queue_declare('', false, false, false, true);
2 $queue_name = $q[0];
3 $channel->exchange_declare(
4 'test_direct', 'direct', false, false, false);
5
6 // Bind for each routing key we want (BINDING KEY)
7 $channel->queue_bind($queue_name, 'test_direct', 'apple');
8 $channel->queue_bind($queue_name, 'test_direct', 'orange');
9 $channel->queue_bind($queue_name, 'test_direct', 'banana');

42. What to expect...

43. Exchanges: Topic

44. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

45. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

46. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

47. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

48. Real World Example

50. Fetch message
Logging Sequence
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

51. Flexibility!

52. Parallel Processing

53. test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Producer
1
1
2
2
3
3
4
4
5
5
Consumer
Consumer
Consumer
Consumer
Consumer
Consumer
Consumer
Parallel Processing

54. Message
Acknowledgement

55. 1 $channel->basic_consume(
2 'test_queue', '', false, false, false, false,
3 function(AMQPMessage $message) {
4 echo $message->body . "n";
5 throw new Exception('oh noes!!!');
6
7 $channel = $message->delivery_info['channel'];
8 $tag = $message->delivery_info['delivery_tag'];
9 $channel->basic_ack($tag);
10 });
Message Acknowledgement

56. RPC

57. RPC example
Producer Consumer
test_queue
1
reply_to: amq.gen-Xa2
1’

58. TTL

59. TTL - per queue message
Producer Consumer
test_queue
1 2 3 4 5
}
10s

60. TTL - per message
Producer Consumer
test_queue
1 2 3 4 5
5s 3s 7s 1s 9s

61. TTL - queue
Producer
test_queue
}
5s (if no consumers)

62. DLX

63. DLX
test_exchange
amq.KfgPZ3PE
Producer 1
dlx_exchange
dlx_queue
1

64. Scheduling / Delayed messages

65. Priority

66. http://tryrabbitmq.com/

67. Infrastructure

68. Problem: SPOF

69. Solution 1: Clustering

70. Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 5
RabbitMQ
Node 6
Load Balance / Floating IP / Low TTL DNS etc.

71. Everything Replicates
(except queues…)

72. Disk / RAM

73. Configuration...

74. /etc/rabbitmq/rabbitmq.config
[
{rabbit, [
{loopback_users, []},
{vm_memory_high_watermark, 0.8}
]}
].

75. /etc/rabbitmq/rabbitmq.config
[{{{[{{[{{}}{][[[{[{{}[[}{[[{}[][}{}}}{}}{{,},]{
[[{rabbit, [{{}[[}{,,{}[][}{[][][{}{{{{}}}}[[}{{
{{}}{loopback_users, []},[][][]{}{}{}<}{[}[][][}
[{{[{vm_memory_high_watermark, 0.8}]]{}{[[[]]{}]
{{]}[{[{{}[[}{]]{}[][,{}[][}{[][][{}.[]}{]][][]}
]...{}[][,]{.}[][}{}[[[{}{][]}{}{}[}{}{}{]{}{}}[

76. node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
Creating a cluster

77. node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.
Creating a cluster

78. Starting/Stopping Nodes

79. node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.
Removing Nodes

80. Solution 2: HA

81. HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.

82. https://github.com/asgrim/rmq-slides
That link again...

83. Any questions? :)
https://joind.in/14253
James Titcumb @asgrim