OPEN SOURCE DEVELOPERS CONFERENCE http://osdconf.in/ ★ April 26-27th, Noida ★ Keynote Session By Piyush Kumar (Lead of Infrastructure and Website Operations at MakeMyTrip)

1. OSDConf 2014
“In love with Open Source : Past,
Present and Future…“
~/Piyush
http://piyush.me

2. In real open source,
you have the right to
control your own
destiny.
~Linus Torvalds
OSDConf 2014 : http://piyush.me 2

3. Mostly Everyone Uses/Loves Open
Source Directly or Indirectly
• Linux
• Android
• Mozilla Firefox
• LibreOffice/OpenOffice
• Apache HTTP server
• Asterisk (PBX)
• VLC media player
• Wordpress
OSDConf 2014 : http://piyush.me 3

4. 4 C’s of Open Source
• Community
• Collaboration
• Collective Ownership of intellectual property
rights
– Distributed Credit
• Continuous, public process
– Software releases part of the process
– Open peer review
– Public Bug Tracking
– Testing
OSDConf 2014 : http://piyush.me 4

5. My entry to open source…
• I started with using Linux
• then started using a classic LAMP stack as Dev
• And then in sea of Infrastructure (WebOps) it
was open source all around
• Now playing with BigData Stack again … its
OSS … making our life easier !
OSDConf 2014 : http://piyush.me 5

6. Journey….
• 2003-2004 -> Linux User back there in my
college days
• 2005 -> became first Red Hat Certified
Engineer of college
• 2006-2008 -> NexTag Journey as QE
• 2008-2013 -> MakeMyTrip, Website
Operations Team
• 2013-till now -> BI / Lead Data Platform Team
OSDConf 2014 : http://piyush.me 6

7. 2006 -> 2008
• Started as QE – Quality Engineer at
NexTag.com (27 months) part of QA-
Infrastructure team having worked on
managing QA Servers, Testing of NexTag
Application (Search Indexes, Cache System,
Monitoring Applications) and Automation
• Performance Testing of OS, Apache HTTP
server, Tomcat + Every New Release
• Managing F5 BigIP LTM in QA & Google Mini
(Search Engine)
OSDConf 2014 : http://piyush.me 7

8. Key Learning's
• COMMUNITY : IRC #centos channel freenode
– Inspired by awesome z00dax : Karanbir Singh
• HABIT : IRC #noc channel for production updates @ NexTag Infra
• Learned the art of
– ./configure && make && make install
• Love for BASH #!/bin/bash
• SOA – Service Oriented Architecture
• Performance Testing - Jmeter/Replay ( Load / Stress )
• Automated Testing – Sahi / Selenium
• Code Coverage - Cobertura
• Importance of managing Infrastructure like your web applications
– Continuous updates / improvements
– Automation
– Monitoring
OSDConf 2014 : http://piyush.me 8

9. 2008 -> till date
• I played various roles in Web Operations
(which demanded multidisciplinary skills )
– System Engineer
– Security Operations
– Lead Infrastructure
– ISM – Information Security Manager
– Lead DataPlatformTeam
OSDConf 2014 : http://piyush.me 9
hacker@makemytrip

10. DDoS Protection
Firewall - ASA
IPS – Intrusion Detection / Prevention System
WAF – Web Application Firewall
Load Balancer
Web Server
Engine / SOA / APIs
RDBMS - Transactional Database
NoSQL
Caching Technologies
Virtualization / Cloud Stack
Email / DNS / AUTH / Config Management / CI/Deployment
CDN – content delivery network
Log Management - SIEM
Typical Web Infra Stack
Obviously including Hardware, Network & Power 
OSDConf 2014 : http://piyush.me 10

11. MMT OSS powered Tech Stack
• Programming Languages
– JAVA / PHP / Python
• Operating System – CentOS 5/6
• Caching proxy
– ATS – Apache Traffic Server
• Webserver
– Apache HTTPD
– Nginx
• Application Server
– Apache Tomcat / Jetty
– JBoss
• RDBMS – MySQL
OSDConf 2014 : http://piyush.me 11

12. • NoSQL
– SOLR / ElasticSearch (Lucene) – Search
– Cassandra
– Couchbase
• Monitoring
– Graphite / Diamond
– Nagios
– NOCMATE-Rx
• Lmetric – Logging Infrastructure
– Syslog
– Logstash
– ElasticSearch
– Kibana – Web Dashboard
OSDConf 2014 : http://piyush.me 12

13. • Cheetah
– Selenium , TestNG , Pyvnc2swf
– Jenkins
• Neerikshan
– Observium (network monitoring)
• Version Control
– GIT, SVN , CVS
– GitLab – Git Management
• Continuous Integration
– Jenkins
• Performance Testing
– Jmeter / ab / soapui
• IDE
– Eclipse / vim
OSDConf 2014 : http://piyush.me 13

14. • Virtualization
– XenServer
• Configuration Management - Puppet
• Application Frameworks
– Spring (JAVA)
– Laravel (PHP)
– Django / Flask (Python)
• Knowledge Management
– Twiki / Wordpress
• CMS
– Drupal
• Security
– OSSIM / AlienVault
– OWASP ZAP
• …..and many more ….
OSDConf 2014 : http://piyush.me 14

15. Cheetah
• cheetah : Test automation and business
service monitoring tool
– uses Selenium and Jenkins(Hudson) integration for
Real Time monitoring of website with nice
Reporting and Alerting
– record sessions using Pyvnc2swf
OSDConf 2014 : http://piyush.me 15

16. Few other hacks
• BizEye SynC Tool
• MyClient
• Image Thumbnail Processing
– ImageMagick
• MyIdeas
– Drupal
OSDConf 2014 : http://piyush.me 16

17. LBManager
• Managing LTM and GTM using Django /
PyControl
• Single interface for managing the
LoadBalancers hosted in multi-colocations /
IDCs
OSDConf 2014 : http://piyush.me 17

18. Graphite/Diamond & NOCMATE-Rx
• Monitoring for Operational Excellence
– Easy interface to collect data / Metrics
– Trends, Dashboards, Alerting , Reporting
– User specific/LOB specific dashboards
• Desi principle: If it moves, we track it :)
• “You cannot manage what you do not measure and
to measure you need to monitor”
• 5 Categories of Metrics
– System | Network | Security | Business |
Application
• Poller’s – DBPoller, SNMP, Script etc.
OSDConf 2014 : http://piyush.me 18

19. Graphite : Trends / Analysis / Dashboards
OSDConf 2014 : http://piyush.me 19

20. Lmetric (Logging Metric)
• Centralized Log and event storage and search !
• Powered by :
– Logstash
– ElasticSearch
– Kibana – Web Dashboard
OSDConf 2014 : http://piyush.me 20

21. Personare : BigData Stack
• To collect valuable user interaction information for
subsequent use in personalization,
recommendations, analysis and reporting.
• Layered Architecture (Lambda):
– Batch Layer : Hadoop M/R
– Speed Layer : Storm NRT Computation
– Serving Layer
OSDConf 2014 : http://piyush.me 21

22. Personare : Architecture
OSDConf 2014 : http://piyush.me 22

23. OSS powered BigData Stack
• Log Collection: Logstash, Syslog-NG
• Distributed messaging system : Kafka , RabbitMQ
• ETL : Pentaho/ Talend
• Hadoop Ecosystem
• NRT Computation: Twitter Storm
• Document-Oriented NoSQL DB : Couchbase
• Distributed Search: ElasticSearch
• Distributed messaging system : Kafka , RabbitMQ
• NoSQL : Cassandra, Redis, Neo4J (Graph)
• API Management : WSO2 API Manager
• Programming Languages : Java , Python, R
OSDConf 2014 : http://piyush.me 23

24. My next hope ! …goal ?
• Would I be able to apply what all I have
learned in technology through OSS to solve
some issues faced by masses around the
world ..?
– Using technology to make world a better place for
everyone !
and I am sure OSS will have major role to play in
it !!
OSDConf 2014 : http://piyush.me 24

25. Thanks!
Questions (If any ) ?
Contact:
@piykumar
http://piyush.me
OSDConf 2014 : http://piyush.me 25