How often have you heard "it's gonna be hard or impossible to cover this part of our backend application"? I've heard a lot. Because our modern application always involve databases, message queues, 3rd party services. What if I tell you, that I can cover even logs with tests, values of custom application metrics, apps that involve Google Cloud Storage, Google Data Storage, Kafka, Redis, Lambdas, DynamoDB, Aerospike, S3. I work in highload project without manual QA's at all. And I'm confident in my everyday releases, so should be you.
10. Patron, dog mascot of the State Emergency
Service of Ukraine
*Patron translates as “bullet”
11. Mac
OS
Windo
ws
Linux
Intel 1 2 ∞
Arm 3 4 ∞
What are the options to run 3rd parties? (MongoDB)
Pros:
- easy to run in all
envs
01 02
Mock it Run real db on your computer
Cons:
- Will not represent
production
- Chance to
implement mocks
wrongly
Pros:
- Familiar way to
start DB
- Real DB
Cons:
- Different setup for
local envs and for CI
12. Where we can run 3rd parties? (MongoDB)
04
03 Use dedicated cloud infra
for tests
Run everything in docker
ecosystem
Pros:
- As close to
production as
possible
Cons:
- Hard to isolate test
runs
- Might require VPN
access to internal
resources
- Can cost money
Pros:
- easy to run in all
envs
- Close to production
Cons:
- Initial setup might
take some time
- Debug in containers
is harder
13. Running tests in Docker infra is
hard?
01
04
02
05
03
Build image for
an app
Spin all the
services
Run tests in
container
Change the
code or the
tests
Repeat 1 and 3
and 4?
14. What If I tell you that we can solve
all our problems with one command?
docker compose run --rm nodejs-app-test
21. How our production looks like? (2)
Frontend
Backend
Queue
Node.js app
HTTP
Requests
22. What’s wrong with Kafka tests?
● Recreating topic is time
consuming
● There is no key-value access
● Failed test runs can produce
more messages then read
37. What if our production production is
even more complex? (4)
38. We tested all the gears, but will it
work a gearbox..
39. No way that you can write
that complex test…*
You need more than
1 team to develop,
maintain and test
that
*I know you can,,,
40. Where we can run 3rd parties? (Google services)
04
03
Use dedicated cloud infra
for tests
Run everything in docker
ecosystem
Pros:
- As close to
production as
possible
Cons:
- Hard to setup
isolation for
separate tests
- Might require VPN
access to internal
resources
Pros:
- easy to run in all
envs
- Close to production
Cons:
- Initial setup might
take some time
- Debug in containers
is harder
41. Continuous QA in Qa environment to the rescue!
Service that
constantly
sends data
in QA
(emulates
clients)
HTTP
Requests
Grafana
(any monitoring)
Zoo of services
and 3rd parties
43. How I know that my
product (apps and
infrastructure) works
properly in
production?
1) I don’t receive calls from
support and my boss
2) Metrics and alerts
44. Metrics and alerts on:
- Status codes
- Number of error logs
- Business metrics
- Container restarts
- Resources usage
How can I know, that
my product works in
QA?
45. Example of continuous test
Client
automation
HTTP
Requests
- 10 valid POST
/pets requests
with unique pets
- 5 invalid
requests
- 5 valid GET
/pet/:id requests
- 15 of 200 status codes
- 5 of 400 status codes
- Incoming records per second in
topic should be 10
- Increased number of items in
your database by 10
- Cloud function was executed 10
times
- Cloud function executions
finished without errors
- No logs with severity > error
- No restarts in all containers
- CPU, memory, disk metrics
should be < 80%
- …..
Sends similar
requests
Every second
46. Conclusion
- Use Docker compose infrastructure to
organize your local development
infrastructure and as infrastructure for
tests locally and on CI/CD
- Use continuous testing in your QA
environment by making automated
requests to your infra and monitoring
should catch problems if they occur
47. Thanks!
Do you have any questions?
Discord: #track-b, or Voice Chat: Andrii Shumada
eagleeyes91@gmail.com
https://eagleeye.github.io
https://gitlab.com/andrii.shumada/to-cover-uncoverable
https://github.com/eagleeye/to-cover-uncoverable
https://u24.gov.ua/
Remember to support Ukraine!