Adsmurai training about QA in CI/CD processes.

1. LOGO
QA IN CI/CD PROCESS
KAILIN CHEN
ADSMURAI

2. ADSMURAI
Contents
What Is CI/CD (QA)?1
The QA perspective2
API Automation CI Demo3
Q&A4

3. LOGO
What Is CI/CD (QA)
ADSMURAI

4. ADSMURAI
What Is Continuous Integration?
What?

5. ADSMURAI
What Is Continuous Integration?
Put together
everyone’s changes
all the time

6. ADSMURAI
What Is Continuous Integration?
Continuous Integration (CI) is a development
practice that requires developers to integrate code
into a shared repository several times a day.
Each check-in is then verified by an automated
build, allowing teams to detect problems early.

7. ADSMURAI
What Is Continuous Integration?
Traditionally, CI adoption starts with the development
team. Writing small, easily testable portions at a time,
developers can commit their code to the main repository
often.
The CI system then builds and tests the software
continuously, often at scheduled periods or triggered with
repository updates.
After the tests have been run, the system generally
provides feedback in the form of statistics, output logs,
post-build steps and so on. Some sophisticated systems
even update issue-tracking software or send emails to
development staff.

8. ADSMURAI
Small changes?

9. ADSMURAI
What Is Continuous Integration?
Automation of the build process is at the center of
CI — automatically building upon code commits,
executing test suites for each build, automated and
consistent test environment handling, etc.
With a comprehensive CI system in place, each
developer receives immediate feedback on every
code push, exposing any errors in the process for
immediate attention.

10. ADSMURAI
Why?
Why?

11. ADSMURAI
The cost of integration
Merging the code
Duplicate changes
Testing again
Fixing bugs
Impact on stability

12. ADSMURAI
The cost of integration
cost of merging &
fixingbugs
time

13. ADSMURAI
Benefits of a CI system
Solve problems quickly
Continuous Integration brings multiple benefits to your
organization:
Say goodbye to long and tense integrations
Increase visibility enabling greater communication
Catch issues early and nip them in the bud
Spend less time debugging and more time adding features
Build a solid foundation
Stop waiting to find out if your code’s going to work
Reduce integration problems allowing you to deliver software
more rapidly

14. ADSMURAI
What Next?
More?

15. ADSMURAI
Enabling Continue Delivery
Iteration
Analysis + Design
Development
Testing + Showcase
0 1 2 3 4
Integration + QA Release and operation
Customer
Centralized QA IT Operations
"Agile" team
The "last mile"

16. ADSMURAI
Enabling Continue Delivery
Customer
Delivery team
Constant flow of new features into production
Software always production-ready
Releases tied to business needs, not operational
constraints

17. ADSMURAI
Business needs
User
Interface
Business logic
Database /
Service

18. ADSMURAI
What Is Continuous Delivery?
Continuous Delivery is the ability to get changes of all types—
including new features, configuration changes, bug fixes and
experiments—into production, or into the hands of
users, safely and quickly in a sustainable way.
Our goal is to make deployments—whether of a large-scale
distributed system, a complex production environment, an
embedded system, or an app—predictable, routine affairs that
can be performed on demand.
We achieve all this by ensuring our code is always in a
deployable state, even in the face of teams of thousands of
developers making changes on a daily basis. We thus
completely eliminate the integration, testing and hardening
phases that traditionally followed “dev complete”, as well as
code freezes.

19. ADSMURAI
Feedback From Developer?
Love?
Hate?

20. ADSMURAI
Feedback From Developer?

21. ADSMURAI
What Is Continuous Delivery?
Developers love the idea of Continuous Delivery
mainly because they get much more rapid feedback
on their code.
If feature is delivered to production quickly, they
can get feedback from the live users, and find out
what does and doesn’t work directly from the
customer base.

22. ADSMURAI
What Is Continuous Delivery?
Rapid, reliable and high quality releases mean happier
customers, which most likely equates to increased
revenue for the business.
Development and operations teams are justifiably
excited by the idea of moving faster and, with support
from the business, tend to be the driving forces
behind a transition to Continuous Delivery.
But with most of the focus on development and
operations, it’s easy for testers to feel left out in the
cold.

23. LOGO
The QA Perspective
ADSMURAI

24. ADSMURAI
How?
How?

25. ADSMURAI
The QA Perspective

26. ADSMURAI
Traditional CI Diagram
The below diagram, of traditional CI, shows the flow chart of the
development cycle. Traditionally, when developers check-in code, CI starts a
build on a build server, and verifies build quality. The assessments of build
quality, deployment to the QA environment, and to other environments are
manual steps.

27. ADSMURAI
QA CI Approach Diagram
During the traditional CI approach, defects are not uncovered until a build is deployed
to an environment. This means that defects may exist in the code for days or weeks
before they are uncovered through manual testing.
The QA CI approach shown in the diagram below extends the build process to
automatically deploy on the QA CI server and begin automated tests immediately.

28. ADSMURAI
Continuous Delivery
CD is driven by CI. In CI, the code is logged-in several times a day and then re-
compiled, generating multiple QA feedback loops. Thus, it is necessary for
development teams to collaborate and make frequent deployments, which
mandates greater levels of process automation. Also, to successfully apply CI,
automation of the build and deployment process is critical; this ensures a self
testing build and makes the entire process transparent and agile.

29. ADSMURAI
Automated testing

30. ADSMURAI
QA CI Automated tests
QA CI automated tests ensure the quality of a build, by
running configured tests on a deployed environment.
Unlike unit tests, QA CI tests use a realistic test
environment, database and other integrated components.
QA CI tests help find the defects as soon as they are
introduced.
Any bugs found during the QA CI tests can be addressed
by the development team. The code then follows the QA
CI path again.
A build log gets updated with test results, indicating
whether or not the build is ready for further exploratory
tests.

31. ADSMURAI
QA CI Automated tests
QA CI Automated test are the most important part
of any CI/CD process. Efficient and effective
automated tests which provide an excellent
coverage, without false positive results, are
the key factors in successful CI/CD process.
Automated tests are usually divided into multiple
“suites”, each with their own objective. The list
below gives a small overview.

32. ADSMURAI
QA CI Automated tests
Unit tests: This is the suite that is run first, often
by developers, before they add their changes to the
repository. Unit tests normally test individual classes
or functions. When those classes or functions need
access to external resources, those resources are
often provided as “mocks” or “stubs”.

33. ADSMURAI
QA CI Automated tests
Integration tests: This is the next level up from the
unit tests. Integration tests ensure all the modules of
an application work properly with each other. They are
normally run in a production like environment. Often
times it is a clone of the production environment using
similar database and integration endpoints.

34. ADSMURAI
QA CI Automated tests
Regression tests: These are the type of software
tests that seek to uncover new software bugs, or
regressions, in existing functional and non-
functional areas of a system after changes such as
enhancements, patches or configuration changes,
have been made to them.

35. ADSMURAI
QA CI Test Environment
QA CI environment is equally important as the
quality of the tests:
In order to avoid erroneous test results, automated
tests should run in clean environments, every time.
Ideally, tests should start from a clean slate by
resetting the test environment, restoring a virtual
machine snapshot, restoring backups of databases,
etc.
QA CI tests should run in an environment that is as
close to production as possible. Ideally, a clone of
the production environment is preferred.

36. ADSMURAI
Maintain a Single Source Repository

37. ADSMURAI
Automate the build

38. ADSMURAI
Make your build self testing

39. ADSMURAI
Everyone commits to the mainline every day

40. ADSMURAI
Every commit builds main on CI Server

41. ADSMURAI
Keep the build faster

42. ADSMURAI
Test in a clone of production

43. ADSMURAI
Make it easy to get the latest artifact

44. ADSMURAI
Make it visible

45. ADSMURAI
If it’s broken, Fix it !

46. ADSMURAI
Automate deployment

47. ADSMURAI
Team discipline

48. LOGO
CI DEMO
ADSMURAI

49. ADSMURAI
API Automation CI Demo
GITHUB
CIRCLECI
REST ASSUERED
TESTNG
REPORT
DEMO
Composition

50. ADSMURAI
GITHUB

51. ADSMURAI
REST ASSURED

52. ADSMURAI
TESTNG & MAVEN

53. ADSMURAI
CIRCLECI

54. ADSMURAI
REPORT
Extend Report
TestNG
Surefire

55. ADSMURAI
Challenges Of Working In A Continuous Delivery Environment
It’s not unusual for testers to struggle with the transition
to Continuous Delivery. In this brave new world, there’s
likely to be a strong focus on:
Automating as much of the testing as possible
Delivering features in small chunks, many of them without
a user interface to test against
Features being developed in a matter of hours or days, not
weeks or months, with little time for planning and
preparation
Time to market over perfect software.

56. ADSMURAI
Challenges Of Working In A Continuous Delivery Environment
These changes in perspective require testers not only
to optimize their existing skillset, but to learn some
new tricks too.
When delivering software continuously, testing needs
to be continuous also.
It needs to be infused into every stage of the
development process, from identifying the story as a
business requirement, to figuring out whether or not
customers like and are using it once the feature is
delivered to the production environment.

57. ADSMURAI
How do you make your testing continuous?
How?

58. Company Logo
Critical Thinking
In a Continuous Delivery world it’s all too easy for a tester to fall into
the trap of focusing exclusively on the technical considerations
(automating all the things!) and forget that what they’re actually on the
team to do is to think. So when you’re working on a Continuous
Delivery project, do just that. Think!

59. Company Logo
Critical Thinking
1
Whether the team is building the
right thing in the first place
2
Whether you can add value by
working with the team to identify and
express acceptance criteria,
scenarios and examples
3
Where the waste is and how it
can be removed

60. Company Logo
Critical Thinking
4
How to approach non-functional
testing, performance, security,
accessibility etc.
5
What devices, browsers, operating
systems and environments the
product needs to support and how
you can test those
6
What data and configuration are
required across all of the
environments in your build pipeline

61. LOGO
Q&A
ADSMURAI

62. ADSMURAI
Q&A

63. ADSMURAI
Game Time !

64. ADSMURAI
Perfect place

65. LOGO
ADSMURAI