2. http://genn.ai/
Who am I?
Takeshi NAKANO
Senior Researcher / Architect.
Co-authered Getting started with Solr in Japanese.
Co-authered Hadoop Hacks in Japanese.
2
Writing Getting started with Kafka in Japanese.
3. http://genn.ai/
3
Company Name Recruit Holdings Co., Ltd
Founded March 31, 1960 (incorporated: August 26, 1963)
※governance changed from Recruit Co., Ltd since October 1st, 2012
Headquarters Tokyo, Japan
Number of Group Employees 22,368
(as of the end of March,2012)
Number of Group Companies 102
(Consolidated Subsidiaries/as of the end of June,2012)
Capital 3,002.64 million yen
(since March 1st, 1995)
Consolidated Sales 806,661 million yen
(from April 1st 2011 to March 31st 2012)
Consolidated Ordinary
Income
117,617 million yen
( from April 1st 2011 to March 31st 2012)
4. http://genn.ai/
4
Recruit
Holdings
Headquarters
Function
Global Business
Business R&D
Recruit Career Co., Ltd
Recruit Jobs Co., Ltd
Recruit Sumai Company Ltd
Recruit Marketing Partners Co., Ltd
Recruit Lifestyle Co., Ltd
Recruit Administration Co., Ltd
Recruit Technologies Co., Ltd
Recruit Communications Co., Ltd
Operating Companies Domain
Recruit Staffing Co., Ltd
STAFF SERVICE HOLDINGS CO., LTD
5. http://genn.ai/
What is our challenge?
Clients can get the information about their customers
checking their items on our web sites.
Providing a printed report monthly.
▽
Send them more detailed report in real time.
In real time fashion, the value of the information will be
dramatically increased!
5
6. http://genn.ai/
Agenda
What is genn.ai?
Overview
How to gather clicks
0) Injecting our javascripts
6
Genn.ai core details
Structure
1) Setting up filters
2) Filtering customers
3) Visualizing them
Future
Dealing with historical streams
Genn.ai wants friends.
7. http://genn.ai/
Overview of using Genn.ai
7
Customer
A
Customer
B
Customer
C
Realtime Analysis
Platform
Genn.ai
じゃらん
Web Sites
Client
(Hotels)
Marketer
Analyst
8. http://genn.ai/
Demo videos!
•They are based on a hotel reservation site like
Expedia.com.
•An animation and a screencapture what the
marketting staff see using Genn.ai.
At this moment ..
•What prefectures are all my users searching for? (0:26)
•Comparing the number of searches in today s and
yesterday s regarding places. (2:50)
8
9. http://genn.ai/
Overview of using Genn.ai
9
Customer
A
Customer
B
Customer
C
Realtime Analysis
Platform
Genn.ai
じゃらん
Web Sites
Client
(Hotels)
Marketer
Analyst
10. http://genn.ai/
Overview of using Genn.ai
10
Customer
A
Customer
B
Customer
C
じゃらん
Web Sites
Realtime Analysis
Platform
Genn.ai
Client
(Hotels)
Marketer
Analyst
11. http://genn.ai/
How to gain access to Genn.ai
11
Customer
A
Customer
B
Customer
C
App
(ex.Tomcat)
Apache
Genn.ai will be able to be attached!
12. http://genn.ai/
How to gain access to Genn.ai
A custom apache module for embeding our
code on returning html sources from App.
12
Customer
B
Apache
Genn.ai will be able to be attached!
mod
Genn.ai
App
(ex.Tomcat)
15. http://genn.ai/
Steps for getting results
There are 3 steps! (after getting clicks)
15
1 2 3
Pouring clicks
into Kafka
Recording them
as behaviors
Visualizing them
0
gather clicks
16. Backend for Genn.ai
http://genn.ai/
Structure (Pouring into Kafka)
16
Storm
Topology
(Filter)
Kafka
kakan.puTomcat
iPad
(APP)
MongoDB
321
Genn.ai Front
(Based on Finagle)
Query
genn.ai/web)Click Streams
Frontend for Genn.ai
Visualizer for Genn.ai
Zoomdata
17. Backend for Genn.ai
http://genn.ai/
Structure (Recording as behaviors)
17
Storm
Topology
(Filter)
Kafka
kakan.puTomcat
iPad
(APP)
MongoDB
Genn.ai Front
(Based on Finagle)
Query
genn.ai/web)Click Streams
321
Visualizer for Genn.ai
Frontend for Genn.ai
Zoomdata
18. Backend for Genn.ai
Visualizer for Genn.ai
http://genn.ai/
Structure (Visualizing customers)
18
Storm
Topology
(Filter)
Kafka
kakan.puTomcat
iPad
(APP)
MongoDB
Genn.ai Front
(Based on Finagle)
Query
genn.ai/web)Click Streams
321
Frontend for Genn.ai
Zoomdata
19. http://genn.ai/
How is Genn.ai composed?
Genn.ai Frontend! : Gathering clicks.
Genn.ai Backend! : Recording behaviors.
Genn.ai Visualizer! : Visualizing customers’ attrs.
↑
Kafka : Used for message bus
MongoDB : Storing Additional tables
Storm : Processing click streams
19
29. http://genn.ai/
Time Shift function?
In our 3 min time window, 1.5 min are captured.
2 clicks have been recorded.
29
1.5 mins
Now
Time window
▲
Click
▲
Click
2 clicks are captured.
t
33. http://genn.ai/
Functions for recording behaviors
Time shift function
Time sensitive counter (old item will be expired and deleted)
Facet ( like Solr )
Categorized counter
Range Facet ( like Solr )
Dynamic categorized counter
33
321
34. http://genn.ai/
Functions for recording behaviors
Time shift function
Time sensitive counter (old item will be expired and deleted)
Facet ( like Solr )
Categorized counter
Range Facet ( like Solr )
Dynamic categorized counter
34
321
35. http://genn.ai/
Counting the frequency
This function is under the effect of the Time
shift function.
35
{
"_id": 154,
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
},
"largeAreaCntPre": {
"Tokyo": 4,
"Berlin": 3
},
"rateAvgAcl": 4200,
"rateAvgPre": 6200,
"rateCntAcl": [
4, 1, 0, 0, 0
],
"rateCntPre": [
4, 1, 2, 0, 0
],
"lastUpdate": "2013-04-16T04:04:37.432Z"
}
36. http://genn.ai/
Counting the frequency
This function is under the effect of the Time
shift function.
You have booked 1 stay in Berlin and 3 stays
in Tokyo in this 3 weeks.
36
▲
L.A.
▲
S.F.
t
Now
Time window
▲
Berlin
▲
Tokyo
1.5 week 3 weeks
▲
Tokyo
"largeAreaCntAcl": {
"Tokyo": 3,
"Berlin": 1
}
▲
Tokyo
43. http://genn.ai/
Zoomdata
Zoomdata has the functions like..
WebAPI for streaming data.
Own event processor for vizualizing.
43
321
The bar and bubble graph on default setting.
Customizable visualizing templates.
45. http://genn.ai/
Genn.ai Evolution
Adding functions on Genn.ai core.
Genn.ai should have filtering / aggrigation functions
for federating with other systems.
Historical rewinding function will be implemented with
the Hadoop cluster.
45
46. http://genn.ai/
Future plan
Genn.ai wants friends!
We are looking for co-challenging partners for tuning
and improving Genn.ai.
You can use Genn.ai on your web sites for totally free
(infrastructure etc..) if we could share the dream.
Genn.ai codes will be opensourced in the future.
Please feel free to contact us!
http://genn.ai/
46