1. RubyConf 2009 2009-11-19
Inside buzztter
設樂 洋爾 (Yoji Shidara)
yoji.shidara@enishi-tech.com

2. My name is
Yoji SHIDARA.

3. Enishi Tech Inc.
Small Company.
Only 2 staffs.

4. Small Office.
(CEO’s flat)

5. I’m from Sapporo!
@darashi (me)
Sapporo
1.4hrs 9.5hrs
Tokyo San Fransisco!

6. Sapporo is a
beautiful provincial
city of Japan.

7. Delicious foods are
waiting for you!

8. Do you know
buzztter.com?

9. RubyConf 2009 2009-11-19 LT
Outside buzztter
設樂 洋爾 (Yoji Shidara)
yoji.shidara@enishi-tech.com

10. http://buzztter.com

11. http://buzztter.com
buzz-phrases (auto update)
Full-text Search Results
(auto update)

12. Servers are in my room!
next to my bed

13. We crawls:
1,932,823 tweets/day (Japanese)
607,749 tweets/day (English)
(We’re now focusing on Japanese users
due to API limits)
We have:
1,336,444 PVs / Month

14. Server 1:
Phenom 9950
Memory 8GB
SSD 80GB + HDD 80GB
Server 2:
PhenomII x4 955
Memory 8GB
HDD 500GB

15. @buzztter

16. Bot for English was
banned because it tweeted
spam as hot
:-<

17. Buzztter:

18. buzz-phrase
detection
+
real time full text
search

19. ... like official
“trending topics”

20. buzztter released
before the official
one launched!

21. buzztter supports
English and
Japanese!

22. 1-2 minutes for
buzz analysis par a
language.

23. HOT notification
bot is available.

24. RubyConf 2009 2009-11-19 LT
Inside buzztter
設樂 洋爾 (Yoji Shidara)
yoji.shidara@enishi-tech.com

25. System Diagram:
Rough Sketch
Web Notification
Frontend module
crawler Fulltext Analyzer
module Search module
BuzzDAS: Buzz Data
Analysis System

26. System Diagram: Crawler
Typhoeus
Twitter crawler EventMachine
RabbitMQ MQ
crawl crawl scheduling
controller
libtextcat
Users DB langugage
language,
guesser
PostgreSQL post frequency

27. notifier pagecache
crawler bot varnish
Twitter4R Web Frontend
net/irc
Rails
MQ Web Service API
Sinatra
memcached
groonga
fulltext
importer search analyzer
engine
BuzzDAS: Buzz Data Analysis System

28. analyzer
tokenizer phrase extractor
detects change of phrase occurrence
Reference Recent
Index Index
keeps post in 24 hours keeps recent 1 hour

29. Try http://buzztter.com and give
me your feedback please!
If you Interested in these
keywords:
groonga, AMQP, RabbitMQ, Typhoeus,
EventMachine, PostgreSQL, libtextcat,
Sinatra, Rails, Twitter4R, net/irc,
memcached, PrefixSpan ,MeCab, TF-IDF, ...
or our BuzzDAS Engine,
please contact me!
My name is Yoji Shidara.