Explains the origin of Private Cloud at Rakuten, how we built it, promote it, and operate it.

1. Rakuten’s Private Cloud
Jonathan Levine
http://www.rakuten.co.jp/

2. 2
What’s Rakuten?

3. Rakuten Facts
• Founded in 1997 by Hiroshi Mikitani
• Over 9,000 staff worldwide
• Rakuten Ichiba and Rakuten Travel are largest businesses;
we have a total of more than 50 services
• Domestically, more than 130 million products offered by
more than 130,000 providers

4. Corporate History - Domestic1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013

5. Corporate History – Global1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013

6. Domestic Marketplace Sales Growth
Source: Rakuten
*: Domestic E-Commerce = Mall (Fixed-price, Group purchase), Mobile, Business, Golf, Books, Auction (Flea market), download, tickets,
Internet Supermarket, Check Out, Off-track betting, toto, ShowTime, Media rental, Dining , Salon (added from Q2/12), and Mart and
Kenko.com (added from Q3/12)

7. International Marketplace Sales Growth
Source: Rakuten
*: Marketplace = Taiwan Rakuten Ichiba + Tarad Premium Mall + Lekutian + Buy.com marketplace + PriceMinister marketplace +
Rakuten Belanja Online + Rakuten Deutschland + Play.com marketplace + Rakuten Brasil marketplace (from Q2/12).

8. 8
Why is Rakuten Ichiba Different?

9. Our Mission
Empowerment

10. 10
Unique E-Commerce Model
Rakuten’s Empowerment Model is Unique and Difficult to Copy

11. 11
Typical E-Commerce Site

12. 12
Rakuten’s E-Commerce Site

13. 13
Why Cloud?

14. My First Computer

15. Moore’s Law
Source: Wikipedia

16. My Current “Computer”
<

17. 17
Server CPU Utilization, by Service
新サービス系サーバー別 CPU稼働率(2009年3月)
0
10
20
30
40
50
60
70
80
90
100
wfbtool
wplaza
misml
wdeli
wenq
wmoney
wdigi
wincu
woubo
wiseek
ddin
wnikki
btmail
bplaza
wisdir
bdigi
wspace
ddeli
wtmail
bfbtool
wapitic
wpsp
wgame
wwoman
apdelicti
wportalgl
bnikki
wdigiadm
wmile
wdeliimg
bmedia
asearchr
wbrtic
wportaladm
mtic
wgameapp
bauto
サーバー群
CPU使用率(%）
CPU使用率最大値
CPU使用率平均値
１０％

18. Domestic Marketplace Sales Growth
Source: Rakuten

19. 19
Cloud at Rakuten

20. 20
Step 1: Virtualization
Virtual Server Pool
Server Server
Shared Storage
Hard Disk
Storage Network
Old： 1 physical == 1server New：1 physical == n servers
System
Benefits of Virtualization
•Faster Deployment Time for New Servers
•Automatically Handle Downtime
•Improve Equipment Utilization
•Easy Replacement of Hardware
OS
App
OS
App
OS
App
OS
App
OS
App
Virtual Server Pool

21. 21
“Pcloud” production Environment
Server Farm:
104 Physical
Storage Farm:
14 Physical
iSCSI

22. 22
Private Cloud Progress – Stage 1
VMs

23. Why so Slow?

24. Technology.

25. 25
“RIaaS" production Environment
Fabric Network
(10G, FCoE)
3PAR Storage
Blade Server
VMware

26. 26
Private Cloud Management Tool
・VM management
・Server Setup (with Chef)
・DNS setting
・SLB control

27. Not Just Technology.

28. Not Just Technology.
It’s an Internal Business.

29. Source: Amazon re:Invent

30. • Tools
• Training
• Support
• Roadmap
• Communication
== Product Management

31. RIaaS is crossing over the Chasm!!!
We are here!
Early Adopter Early Majority Late Majority
Proof of solution Willing to use Step by Step
Test -> Production
Case study
Sensitive
Communication Word of mouth Interactive Passive
SLA No Risk No Return Reasonable requirement Demanding
Priority Speed
Experience
Balanced Risk/Return Stability
PCloud
RIaaS

32. 32
Private Cloud Progress – Stage 2
Jul-10
Aug-10
Sep-10
Oct-10
Nov-10
Dec-10
Jan-11
Feb-11
Mar-11
Apr-11
May-11
Jun-11
Jul-11
Aug-11
Sep-11
Oct-11
Nov-11
Dec-11
Jan-12
Feb-12
Mar-12
Apr-12
May-12
Jun-12
Jul-12
Aug-12
Sep-12
Oct-12
Nov-12
Dec-12
Jan-13
Feb-13
Mar-13
VMs

33. 33
Case Study

34. 34
Exciting Event …?

35. 35
2005/1/1
2005/6/1
2005/11/1
2006/4/1
2006/9/1
2007/2/1
2007/7/1
2007/12/1
2008/5/1
2008/10/1
2009/3/1
2009/8/1
2010/1/1
2010/6/1
2010/11/1
2011/4/1
2011/9/1
2012/2/1
2012/7/1
2012/12/1
SuperSale Network Effect

36. 36
Private Cloud for SuperSale
0
1000
2000
3000
4000
5000
6000
Jul-10
Aug-10
Sep-10
Oct-10
Nov-10
Dec-10
Jan-11
Feb-11
Mar-11
Apr-11
May-11
Jun-11
Jul-11
Aug-11
Sep-11
Oct-11
Nov-11
Dec-11
Jan-12
Feb-12
Mar-12
Apr-12
May-12
Jun-12
Jul-12
Aug-12
Sep-12
Oct-12
Nov-12
Dec-12
Jan-13
Feb-13
Mar-13
VMs

37. 37
Next Steps

38. High Capacity Racks
Patch
Patch
EqualLogic
R610
R610
2
1
5
4
3
8
7
6
11
10
9
14
13
12
17 R610
16
15 R610
20
19 R610
18
27 R610
24
23 R610
25 R610
31 R610
30
28
35
34
33 R610
32
41 Patch
40
39
38
37
36
21 R610
22
26
29 R610
42 Patch
39 Nexus 2232PP
2U4N32
42 Nexus 2232PP
41 Nexus 2232PP
40 Nexus 2232PP
38
37
36
35
34
33
31
2U4N30
29
2U4N28
27
2U4N26
25
2U4N24
23
2U4N22
21
2U4N20
19
2U4N18
17
2U4N16
15
2U4N14
13
2U4N12
5
2U4N4
11
2U4N10
9
2U4N8
3
2U4N2
1 Management 48port switch
7
2U4N6
Standard Rack
(３KVA)
Hicap Rack
(15KVA)
48 Nodes
9 Cables
12 Nodes
36 Cables

39. Site Recovery Manager for Migration
Old DC New DC

40. 40
Global DC Expansion
EU DC
Japan DC
U.S. DC

41. Private Cloud Roadmap
• IaaS (Compute as a Service) improvements
– Add APIs
– Add App team access
• DBaaS (Database as a Service)
• STaaS (Storage as a Service)
• PaaS (Platform as a Service)

42. 42
Final Thoughts

43. 43
•12 Employees
•27,000,000 users
•2 years

44. 44
•31 Employees
•#16 site in US (as of May, 2012; source: Alexa)
•30 months

45. 45
But…

46. • Technology is really important…
• … but it isn’t everything

47. • Cost is really important…
• … but it isn’t everything

48. • Make sure you can have enough
scale to be competitive…
• Then compete to win!

49. Thank you!