Architecting for Huper Growth and Great Engineering Culture
by Ifnu Bima
@SARCCOM MEETUP
http://sarccom.org
https://www.meetup.com/Software-Architect-Indonesia
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
Architecting for Huper Growth and Great Engineering Culture
1. Architecting for Hyper Growth and Great
Engineering Culture
Software Architect Indonesia Community #2 Meetup
Ifnu, 26 November 2016
2. Who am I?
• Ifnu Bima
• Ilmu Komputer IPB
• Join blibli in 2012
• Was working for Deustche Bank Singapore
• Employee no 69
• Engineers no ~13
• Development Manager
– blibli.com Web UI
– Mobile Apps : Android, iOS, Windows Phone
– Content Management System
– Search
– Product Discovery : Wishlist, Product Review
3. Architecture Goals
• Hyper Growth
– X times every year: traffic, transaction, feature,
codebase and head count
• Great Engineering Culture
– Foster innovation
– Iteration speed
4. Software Architecture
Software architecture refers to the fundamental
structures of a software system, the discipline of
creating such structures, and the documentation
of these structures
Software architecture is about making
fundamental structural choices which are costly
to change once implemented
Source : https://en.wikipedia.org/wiki/Software_architecture
5. Software Architect / Principal
• Design architecture
– Monolithycs vs Microservices
– SQL vs NoSQL
– Datacenter vs Cloud
• Pick Platform, library and provide template
– Java vs PHP vs Ruby vs Scala vs Go Lang
• Hiring standard
– Technical interview
– Hiring decision
– Training and Certification
6. Functional vs Non Functional Requirement
• Functional requirement in Bussiness
Requirement Documentation (BRD)
• Non Functional Requirement
– External Quality
• Speed, Performance
• Scalability, Elasticity, Reliability
• Security
– Internal Quality
• Unit testing coverage
• Code readability, design pattern
• Learning curve for new hire
7. Speed – Performance : Server Side
• HardDrive & Database is the slowest link
• Lots lots lots of caching :
– Redis : small but lots of item
– Guava : big and slow item, refresh cache with
background process
• SSD for IO intensive operation : Database,
static & image server
• Non blocking IO using reactive system
• Slow Query monitoring and indexing
8. Speed – Performance : Client Side
• Bandwith is the slowest link
• Small bandwith : gzip, image compression,
minimize css & js, image & html lazy load
• Render first, script later: defer 3rd party script
• Layout first, content later: layout only using
CSS, inline CSS for viewport
• Reduce page load & render from 12s to 4s
9. Scalability, Elasticity, Reliability
• Stateless application
• Externalize session to repository. Currently we use Redis
• Externalize scheduler
• Microservices
– http://martinfowler.com/articles/microservices.html
• Reactive System
– Non blocking IO
– Parallel execution
– Future / callback
• NoSQL
– Solr, elasticsearch
– Redis
– MongoDB
• Go lang ?
10. Performance Monitoring
• Newrelic.com
• AppDynamic
• Realtime and Archived performance monitoring
• Performance metrics
– Response time, both server and client side
– Drill down of time spent in application
– Slow query monitoring
– Cache monitoring. Cache hit ratio.
– Server monitoring : CPU, Memory, I/O and Network
11. Performance Testing and Profiling
• JMeter and WAPT, other alternative is Gatling
• Dedicated team to run it
• Java Profiling
– VisualVM
– XRebel
13. What we learn about customer
• Fast UI
– Speed & Performance
• Website stay up during promotion and flash sale
– Scalability & Elasticity
• Consistently good user experience
– Great UI/UX
– Timely order delivery
– Original and high quality product
• New features
– Rapid Deployment
– Continuous Delivery
14. Architecting for Great Engineering Culture
• Optimize for iteration speed
• Push relentlessly toward automation
• Build the right software abstractions
• Develop a focus on high code quality
with code reviews
• Maintain a respectful work
environment
• Build shared ownership of code
• Invest in automated testing
• Allot 20% time
• Build a culture of learning and
continuous improvement
• Hire the best
15. Optimize for iteration speed
• Microservices Architecture
• Blibli.com SOA architecture circa 2013
Store Front
Back officeMerchant
ESB
16. Monolithic vs SOA vs Microservices
Source : http://www.pwc.com/us/en/technology-forecast/2014/cloud-computing/features/microservices.html
22. Push relentlessly toward automation
• Compile, build, unit test every commit using maven / sbt run automaticly using
bamboo CI
• Functional test by QA, automated using JBehave and Appium
• API Test using JMeter
• Release versioned artifact (war / zip) to artifactory
• Deploy to UAT, Performance environment and Production using Chef and Jenkins
– Install latest java
– Setup properties
– Install latest tomcat
– Do additional steps : setup JCE, deploy static file to static server, run database change using
Flyway / MongoBee, etc
– Deploy application
Source : http://www.martinfowler.com/articles/microservices.html
23. Build the right software abstractions
• Microservices template
– Java with Spring
– PlayFramework Scala
– Java with Spring and RXJava
– Go Lang ??
• SSO using CAS
• OAuth API abstraction using CAS
• Example of software abstractions
– Google BigTable and MapReduce
– Facebook ReactJS and ReactNative
– Amazon AWS
24. Develop a focus on high code quality with code reviews
• Use GitFlow for code management
– http://nvie.com/posts/a-successful-git-branching-model/
– Every feature and bug will have their own branch
– Developer commit to their own branch and pull request to
feature / bug / develop branch
– Peer code review and successful build from bamboo is
mandatory to merge branch
• Pair Programming if possible
25. Develop a focus on high code quality with
code reviews
• TDD and Unit testing using JUnit and KarmaJS
• Static code analysis using Sonar Cube
– Calculate unit test coverage
– Calculate code complexity, less than 3
– Code smell
– Security analysis: SQL Injection vulnerability, XSS, etc
– Potential bug in runtime: NullPointerException,
Forever loop, etc
• 90% test coverage expected. Build failed if code
coverage drop compared to previous build. If
build failed, cannot merge to mainstream branch.
26. Maintain a respectful work environment
• Open culture
• Using Slack for open communication
• Share everything
• RESPECT as core value
– Risk taking
– Excellent
– Serving
– Passionate and Proud
– Employee great place to work
– Customer focus
– Teamwork
27. Build shared ownership of code
• Access to source code from day 1
• Some degree of rotation to make sure no part
of code that only known to 1 person
28. Invest in automated testing
• We do not have QA as title, it is Software
Development Engineer in Test (SDET)
• 2017 we really focus on automation
– Bamboo CI
– Chef and Jenkins
– Nightly Build
– Data Preparation
– JBehave & Appium
– JMeter, WAPT & gatling
– Monitoring and alert
– Elasticity on Datacenter
– Etc
29. Allot 20% time
• Internal coding competition
• Internal Hackaton
• Skunk Work
30. Build a culture of learning and continuous improvement
• Rotation for people to learn different things
• Survey and Management action for people who want to change their role or team
• Weekly sharing session : BrownBag
• Training
– Java Training and Certification, Sun Certified Enterprise Architect
– iOS development & testing
– SOLR training
– 7 Habit of effective people
– Presentation and public speaking
– Project management
– Leadership essential
– Management assessment
– Evident base interview
– User Experience from Somia
• Conference
– JavaOne in San Francisco
– Solr Revolution in Austin, Texas
– TechInAsia Jakarta
• Scholarship for Master Degree
31. Hire the Best
• Blibli.com future program
– https://www.blibli.com/page/future-program/
• Employee get Employee
– Best people will attract best people, law of
attraction
• Excruciating round of interview
• Very high hiring bar