2. 

CTO
2011 7 16 2

3.  Palmscape
 1997 Xiino
 Palm OS ( )
 IBM WorkPad, Sony CLIE
 NTT DoCoMo
 M.I.T. TR100/2002

 Web
 IPA 2004 –
2011 7 16 3

4. (2)
 :
 2005 8 2010 11
 Japanize
 Web UI


 Q4M
 (MySQL )
2011 7 16 4

5. (3)
 :
 2010 11
 IT
 2011 5 CTO
2011 7 16 5

6.  DeNA
 OSS
 DeNA OSS

2011 7 16 6

7. 2011 7 16 7

8. Q4M
2011 7 16 8

9. What is Q4M?
 A message queue
 runs as a storage plugin of MySQL 5.1
 Why is it a MySQL plugin?
 accessible by using existing MySQL clients
 no need for a new client library
 administrable by using SQL
 friendly to DB admins
 First release: Jan 2008
2011 7 16 9

10. Design Goals of Q4M
 Robust
 Does not lose data on OS crash or power failure
 necessary for Tokyo wo. nuclear power plants… orz
 Fast
 Transfer thousands of messages per second
 Easy to Use
 Use SQL for access / maintenance
 Integration into MySQL
 no more separate daemons to take care of
2011 7 16 10

11. Users of Q4M
 Many leading web services in Japan
 DeNA Co., Ltd.
 livedoor Co., Ltd.
 mixi, Inc.
 Zynga Japan (formerly Unoh, Inc.)
2011 7 16 11

12. What is a Message Queue?
2011 7 16 12

13. What is a Message Queue?
 Middleware for persistent asynchronous
communication
 communicate between fixed pairs (parties)
 a.k.a. Message Oriented Middleware
 MQ is intermediate storage
 RDBMS is persistent storage
 Senders / receivers may go down
2011 7 16 13

14. Minimal Configuration of a Message Queue
 Senders and receivers access a single
queue
Sender Receiver
Queue
2011 7 16 14

15. Using MQ as a Relay
 Separate queue for sender and receiver
 Messages relayed between queues
Relay
Sender Receiver
Queue Queue
2011 7 16 15

16. Merits of using Message Relays
 Destination can be changed easily
 Relays may transfer messages to different
locations depending on their headers
 Robustness against network failure
 no loss or duplicates when the relay fails
 Logging and Multicasting, etc.
2011 7 16 16

17. Message Brokers
 Publish / subscribe model
 Separation between components and their
integration
 Components read / write to predefined queues
 Integration is definition of routing rules between
the message queues
 Messages are often transformed (filtered) within
the relay agent
2011 7 16 17

18. What about Q4M?
 Q4M itself is a message queue
 Can connect Q4M instances to create a
message relay
 Provides API for creating message relays
and brokers
2011 7 16 18

19. Performance of Q4M
 over 7,000 mess/sec.
 message size: avg. 512 bytes
 syncing to disk
 Outperforming most needs
 if you need more, just scale out
 Can coexist with other storage engines without
sacrificing their performance
see http://labs.cybozu.co.jp/blog/kazuhoatwork/2008/06/q4m_06_release_and_benchmarks.php
2011 7 16 19

20. Q4M in DeNA
 uses Q4M for sending notifications
asynchronously
http://engineer.dena.jp/2010/03/dena-technical-
seminar-1-2.html
2011 7 16 20

21. Q4M in DeNA
 Q4M
 2,000 /
 DeNA


2011 7 16 21

22. HandlerSocket Plugin
2011 7 16 22

23. Memcached

 RDBMS /


 RDBMS


 Memcached RDBMS

 primary storage
2011 7 16 23

24. Memcached
 RDBMS


 +
 :

 SQL
2011 7 16 24

25. SQL
( )
2011 7 16 25

26. Mycached
 2009 8
 mysqld memcached
 memcached get
 get
 SQL
2011 7 16 26

27. Mycached –
mycached
http://dev.mysql.com/doc/refman/5.1/en/pluggable-storage-overview.html
2011 7 16 27

28. Mycached –
 1,200
 JSON, msgpack,
 Picoev (I/O )
 MySQL
( )

 get
2011 7 16 28

29. Mycached –
 SQL 40,000qps
2011 7 16 29

30. Mycached


2011 7 16 30

31. HandlerSocket Plugin
 DeNA
 2010 8

 MySQL Contributor Award 2011
 MySQL SQL
 qps
 mycached
2011 7 16 31

32. …
2011 7 16 32

33. …
2011 7 16 33

34. …
…
2011 7 16 34

35. …
…
…
2011 7 16 35

36. 2011 7 16 36

37. 





2011 7 16 37

38. HandlerSocket Plugin
 mycached


 FastEther

2011 7 16 38

39. 2011 7 16 39

40. 2011 7 16 40

41. –




?
2011 7 16 41

42. –

 Server::Starter –
 Parallel::Prefork –
 Starlet – PSGI
2011 7 16 42

43. –

 fork
 : mod_bumpy_life (Apache)
 by hirose31++
 MaxRequestsPerChild

 Parallel::Prefork
2011 7 16 43

44. –

MaxRequestsPerChild

fork …
2011 7 16 44

45. – : Slow Restart

 fork
listen
accept
fork & exec
SIGUSR1
accept
logic
app.
spawn
app. servers
fork & exec
app. logic
SIGHUP
2011 7 16 45

46. – OSS
 Perl github
CPAN

 …
2011 7 16 46

47.  Q. ?
$SIG{HUP} = sub { $got_sig = 1 };
…
while (! $got_sig) {
sleep(1000); //
}
2011 7 16 47

48.  Q. ?
$SIG{HUP} = sub { $got_sig = 1 };
…
while (! $got_sig) {
// 1000
sleep(1000); //
}
2011 7 16 48

49.  Parallel::Prefork


…
 : pselect
 FAQ
 : Perl pselect
 XS (Perl C )
 Perl XS
2011 7 16 49

50. OSS DeNA
2011 7 16 50

51.  Perl …
 2009 Plack
 Perl WSGI or Rack
 Furl ( HTTP )
 DeNA id:xaicron

 MySQL
2011 7 16 51

52. 
 MySQL
 YAPC
 MySQL Conference


 : MySQL Casual


2011 7 16 52

53. 2011 7 16 53

54.  OSS


OSS


2011 7 16 54