Ashwini Malthankar Organization: ThoughtWorks Topic: Effective use of CI by QA Presented in vodQA - THE TESTING SPIRIT! on Oct 7, 2010 in ThoughtWorks, Pune

1. Thursday, October 7, 2010

2. Effective use of Continuous
Integration by QA
 Ashwini Malthankar
 Lead Consultant QA
 ThoughtWorks

3. Agenda
Problem Statements
About Continuous Integration
How does Continuous Integration help resolve problem
Case study on large scale distributed project
Open source / commercial tools
Q&A

4. Scenario for Thought Exercise
Given it’s a Thursday today when we have Demo
to customer then what is the state of a Tester?
We have to showcase new facebook feature.

5. Problem Statements
Identifying good build for Demo, Release or Regression testing
Testing on various environments like the Grid shown below
OS App Browsers Versions DB
Servers
Windows WebSphere Firefox ? Oracle
Linux Weblogic IE ? MySql
mac JBoss Safari ? Teradata

6. Why is it important to solve
Constant availability of a "current" build for testing, demo, or release
purpose and to avoiding last-minute chaos at release time.

7. About CI ?

8. High level Overview of Build Server
http://twistcruise:8153/cruise/tab/pipeline

9. Pipeline in the box
Pipeline
Stages
Jobs
Jobs
Jobs
Jobs

10. Project/Product Development Process
Flow
R
Code e Functional
Test Plan + Unit Coverage p Automation
Test case Testing o Test Suites
r
Product t
s System
Requirement Design Coding
Testing
Build
Scope Test
Document framework
+ Test cases BackEnd
Continuous Build
Testing
Integration
API level
Defects Cycle testing
Reports
Reports

11. Development Tools usage Flow
R
Mingle e UI Automation /
p Cucumber
Test Plan + github RCov
o
Test case
r
t
Ruby on Rails s
System
Requirement MSL
Integration
RSpec
Testing
Postgress
Usecases Stub
+ Test cases available
for Testing
Go
Defects Cycle
Reports
Reports

12. How does Continuous Integration help
to resolve current problem
• MATs /Smoke tests, SRGs, LRGs running it continuously to quickly
Identify Demo, Release candidate build on any environment.
• frequent code check-in pushes developers to create modular, less
complex code
• metrics generated from automated testing and Continuous
Integration

15. Give visual representation of CI
QA
Builds
Platform Specific
Builds
Schedules
Performance
Builds

16. Q&A

17. Case study on large scale distributed
Project
Project details
 “largest distributed agile project“
 for every check in, a build is triggered, installers deployed
continuously
 New release every 6 weeks.
 260 Build & Deployment Servers
 3 active development branches, 'branch per release pattern', SVN
model.
 Application has Service Oriented Architecture, having multiple tiers
and is Web-based
 15 Independent Installers to deploy on Dev, QA, UAT, Performance,
Staging, Production environments

18. Deployment Architecture &
Environments
Channels channels channels
Third Party
Services(15)
Apps
Database

19. Iteration -0
Build and Installer
Compile
SVN Unit test
Smoke test Publish
Integration Test
Build test

20. Iteration - 1
Build and
Installer
Compile
Unit test Last good
Last good
Build and
Installer
SVN Smoke Test Publish
Compile
Unit test
Build and
Installer Last good
Compile
Unit test
Iteration – 1
Build test

21. Build Cycle and Flow
Build&Installer
Compile
SVN CodeQaulity
Unit test
Integration
Smoke
QA/validation Showcase
Verification
UAT Performance Staging

22. Build Layers
Dev- Trunk Build
Generate Install the Publish the
feedback(28 Test the installer
installer installer installer
build in parallel)
Install the Verify the Publish the
Get the installer
application installer installer
Test Box
Deployment
Get the installer
Install the Verify the Publish the Showcase
application installer installer
env Deploy

23. Continuous Integration things to watch
• Initial setup time required
• Well-developed test-suite required to achieve automated testing
advantages
• Large-scale re-factoring can be troublesome due to continuously
changing code base
• Hardware costs for build machines can be significant since CI like
production makes more sense
• Final feedback is only with system in production

25. What more can be Done
• Every build with Performance test execution to know the
performance of the application per check-in.
• Code health check apart from Code Coverage which we can be
integrated in the build with tools like Code City. It can give the code
complexity metrics to know how complex the code is getting per
check-in.
• Continuous Integration  Continuous Deployments  Continuous
Regression  Continuous Performance  Continuous Software
Health

26. What are other commercial tools
available in the market
AnthillPro — continuous integration server by Urbancode
Apache Continuum — continuous integration server
Bamboo — commercial continuous integration server by Atlassian
Software
Go — Commercial agile build and release management software-
ThoughtWorks
Hudson — MIT-licensed, written in Java, runs in servlet container
TeamCity — commercial continuous-integration server by JetBrains
with free professional edition

27. Q&A