2. who am i
Name: Tatsuro Hisamori
twitter, PAUSE, etc..: myfinder
Came from: Tokyo / JAPAN
I’m called “maa-in-der(まいんだー)” from
engineers in japan.
3. who am i
please call me “maa-
in-der”
“(まいんだー)”
when you call my
name.
4. why I came here
- I was awarded the
best runner-up talk of
YAPC::Asia 2013
- talk page link
- today’s my talk is a
part of that talk.
5. I’m working at
- Internet Ad Tech Company at Tokyo Japan.
- that company has a RTB business brand
“doubles” in U.S. and Europe.
- https://www.dobleas.com/
6. RTB?
- Real-time Bidding
- programmatic
instantaneous auction
- similar to financial
markets "High
Frequency Trade"
7. RTB?
- We Developed that
system by Perl.
- Talk about the “make
test” of this system.
8. Today’s topics
- Test case(.t files) is increased to more than
1000
- problems with this situation.
- the method and system to be executed at a
high speed.
- how to develop that test system.
9. we have Problems
- more than 40 minutes uses to test one
branch.
- execution of the “make test” is serialized
- time-consuming
- wait and wait and wait…..
- motivation is lost ;-(
11. we have Problems
- and developers will not to run the “make test”.
- and the risk of accidents will increase.
- and Product will not be able to maintenance ;-
(
13. Improvement aka KAIZEN
- requirement
- faster more and more of course.
- that can be scaled out and easy
maintenance.
- without changing the test already exeists.
14. Strategies of Implementations
- run across multiple servers.
- distributed a ".t" file-by-file basis.
- easy to add server.
- provisioning by puppet.
- easy to maintenance.
- deploy by capistrano “cap shell”.
15. implementation image
Master Node
Cluster Node
Test List
First
count the number of test
files(.t files).
shuffle and assign test
cases to nodes.
Second
execute tests.
receive results from nodes and summarize.
Therd
notification and visualize.
16. First
- count and shuffle uses File::Find, List::
MoreUtils
- and assign .t files to cluster node.
- see gist.
26. Ukigumo::Server
- very easy to
understand the results.
- Ukigumo-chan
designed by
MIKIHOSHI (PAUSE)
- he is my Colleague
27. running environment
- we made 16 nodes
cluster by surplus PC
parts
- the servers has been
worked production env
until years ago :D
28. running environment
- If you try this, you
should to use the AWS
EC2 :-)
- use the spot instance
in particular.
- it’s cheap :-)
29. result
- reduced about
40 min -> 3 min
- becomes even faster by adding more servers.
- we got more faster “make test” cluster!!!
30. conclusion
- talked about more faster “make test”.
- First, it’s simple, made from well-known perl
modules.
- Second, it’s very scalable.
- Therd, easy to summarize and notifications
and visualize.
31. one more thing.
- a nice surprise.
Developers has came
"git push first, if them make a new branch"
therefor
developer activities became more visible.