The document is a presentation on DevOps and the role of testing within a DevOps approach. It discusses what DevOps is, some of its key principles and benefits, and the challenges it can pose for quality assurance teams. It then describes a case study of an organization called "Code Name - Manhattan" that implemented DevOps practices to enable faster and more reliable software releases through continuous integration, automated testing, and closer collaboration between development, QA and IT operations teams.
Exploring the Future Potential of AI-Enabled Smartphone Processors
DevOps: Where in the World Is Test?
1. T12
Special Topics
5/8/2014 11:15:00 AM
DevOps: Where in the World Is
Test?
Presented by:
Erik Stensland
Pearson Learning Technology
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Erik Stensland
Pearson Learning Technology
Software quality architect for Pearson Learning Technologies, Erik Stensland is leading the
technical testing efforts with a concentration on functional, performance, and integration testing.
In the software test and quality engineering field for more than twenty years, Erik focuses on
implementing innovative ideas around process and tools to help teams become more effective
and efficient. His previous roles have included senior QA management positions and QA
consultant for Internet startups and DoD industries. A board member for the Software Quality
Association of Denver, Erik has been an international presenter at leading testing conferences
in Europe and the United States. Follow Erik at testarch.blogspot.com/ and on Twitter
@erikstensland60.
3. 4/26/2014
1
DevOps: Where in the
World is Test?
StarEast 2014 Conference
Date: May 8th, 2014
Author: Erik Stensland
A Little about me
Blog: http://testarch.blogspot.com
LinkedIn : http://www.linkedin.com/in/erikstensland/
Twitter : https://twitter.com/erikstensland60
email: estensla@yahoo.com
9. 4/26/2014
7
DevOps Principles
Benefits of
DevOps
Faster time to market (reduced cycle times
and higher deploy rates)
Increased Quality (i.e., increased availability,
increased change success rate, fewer failures)
Increased organizational effectiveness
(increased time spent on value adding activities
vs. waste, increased amount of value being
delivered to the customer).
14. 4/26/2014
12
Keys to Test/QA in
DevOps
Your purpose is to Prevent Defects.
Focus lies on Risks to the product instead
of just the functionality of the software
changes.
QA must be testing early and often using
“Walk Away Automation” to Integrate QA
into the automated deployment flow.
Team members need to have the “Right
Attitude”
The Case Study:
Code Name - Manhattan
16. 4/26/2014
14
Software Engineering
• Few Builds a week. ~1 Deployment every week or two.
• Limited Unit testing
• Manually Build and Deploy to Pre-Prod
Software Quality Engineering
• Very little architecture review for testability.
• Heavy Manual testing.
• Limited UI automated tests, Little web services automated tests.
• Automated test manually kicked off.
• Some performance testing.
IT Operations
• Manual recompiling of Code for different source code repository
• Manual Deployments and Configurations of Production servers
• Limited monitoring of systems , No monitoring of Applications
• No review of Systems before Code is delivered for Deployment
Yesterday’s Environment
The “Discussion”
17. 4/26/2014
15
Extend Eng
into
Production
• Created a Continuous Integration and Release
process using Jenkins.
• Deeply integrating QA into the release process by
creating “walk away automation”.
• Ensure production readiness of the Code as well as
the Environment by spinning up new
environments before code deployment and testing.
18. 4/26/2014
16
Production
Feedback into
Eng
• Implemented Systems and Application Monitoring
in pre-prod and prod.
• Use one incident reporting system for Dev, QA
and Ops to monitor during production releases and
incidents.
• Created blameless postmortems for all releases
and incidents.
Embed Eng
into IT Ops
• Development and QA resources added to the
rotation for production support.
Embed IT Ops
into Eng
• Created reusable stories for non-functional such as
Monitoring, Environment creations.
19. 4/26/2014
17
TODAY – Software Eng
• 100’s of Builds a day.
• Large amounts of Unit tests
• Multiple Deploys a day to pre-prod
• Push Button Deploys to all pre-prod
environments.
TODAY – Test/QA
• Same Repository for Tests as Code
• Large sets of Automated tests.
• New feature Exploratory testing.
• Low maintenance automation tests.
• Continuously running all Automated
Tests in All Pre-Prod environments.
20. 4/26/2014
18
TODAY – IT Operations
• Same Repository for Code and
Environment scripts.
• Push Button Deployments to
Production.
• Monitoring at system and application
on all new deployments.
Ask Yourself What “IF” ?
