Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

[QE 2017] Dawid Pacia, Tomasz Janiszewski - SQA w erze TestOps

136 visualizaciones

Publicado el

Prezentacja z VI edycji Quality Excites. Tomasz i Dawid przedstawili ewolucję pracy testera, która w dzisiejszych czasach nierzadko wymaga umiejętności tworzenia i utrzymania odpowiednich środowisk do testów, do których stworzenia potrzebna jest wiedza na poziomie DevOpsa.
Celem wystąpienia było przestawienie przykładowego szkieletu takiego środowiska do testów oraz narzędzi, które powinny się znaleźć w przyborniku TestOpsa.

Publicado en: Software
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

[QE 2017] Dawid Pacia, Tomasz Janiszewski - SQA w erze TestOps

  1. 1. SQA in a TestOps era
  2. 2. SQA in a TestOps era Tomasz Janiszewski Dawid Pacia
  3. 3. THE TRADITIONAL MODEL OF SOFTWARE TESTING
  4. 4. THE TRADITIONAL MODEL OF SOFTWARE TESTING TESTS QUALITY ASSESSMENT RUN TESTS SYSTEM UNDER TEST ANALYSIS PASS? FAIL?
  5. 5. “To improve is to change; to be perfect is to change often.” Winston Churchill
  6. 6. ➜ Facebook users send on average over 31 million messages and view 2.77 million videos every minute ➜ Twitter users generates 347,222 Tweets each minute – or 21 million Tweets per hour ➜ Google provides search results for over 1 billion search queries each day ➜ Amazon deploys new software to production every 11.6 seconds. The times has changed...
  7. 7. Let’s start with the first set of slides 1. TRANSITION HEADLINE
  8. 8. EXPOSURE CONTROL TEST CASES AND MONITORS THE TESTOPS MODEL OF SOFTWARE TESTING EXPOSED USERS AND SYSTEMS SYSTEM UNDER TEST PRODUCTION USERS AND SYSTEMS ACTIVE MONITORS ANALYSIS QUALITY ASSESSMENT PRODUCTION USAGE TELEMETRY DATA KPIs / PATTERNS
  9. 9. CONTINUOUS BUILD CODE CHANGE / FIX SCM (Git, Svn, Bazaar ;) NEW BUILD (Ant, Maven, Gradle, etc.) CI TOOLS (Jenkins, Bamboo, Travis CI, Ansible, etc.) CONTINUOUS VALIDATION TEST ENVIRONMENTS BU ILD D EPLO YM EN T TEST RESULTS CONTINUOUS DELIVERY PRODUCTION ENVIRONMENTS CONTINUOUS MONITORING MONITORING (Kibana, Grafana, Scalyr, Cacti, Amazon CloudWatch, etc.) SYNTHETIC / SEMANTIC MONITORING
  10. 10. ➜ Synthetic monitoring (or semantic monitoring) runs a subset of an application's automated tests against the live production system on a regular basis. Continuous Monitoring
  11. 11. ➜ Integration with DevOps infrastructure ➜ What tests to run and when ➜ No need to stand up the whole system to test TestOps: What Does This Mean?
  12. 12. WHY SHOULD WE CARE SO MUCH?
  13. 13. FAKAPY ● Zgłaszanie błędu w komentarzy w Jirze - 500 klientom nie działało menu nawigacyjne po aplikacji ● Dobrze zorganizowane glow w Gicie - pull req, praca miedzy zespolami ● Szkiletet ● Jak pisac kod (python) ● Nazwy obiektow! ● Re-uzywalnosc kodu ● Korzystanie z paczek ● Pop ● Pliki readme ● Jak unikac zlych praktyk ● Konwencja komitowanie (a nie ‘fixed small issue’) ● Konsekwencja pracy przez 2-mc na wlasnym branchu ● Dlaczego nie tylko produkt powinien byc testowany, ale również same testy
  14. 14. BUT IN A WORLD OF FALSE... False Negatives False Positives The test is marked as Failed, when the functionality is working The test is marked as Passed, when the functionality is not working.
  15. 15. ...YOU CAN ENCOUNTER SOMETHING WORSE False Negatives False Positives Flaky Tests TESTS FAILING INTERMITTENTLY
  16. 16. RISKVALUE CONTINUOUS: VALUE vs RISK PLAN CODE BUILD TEST RELEASE DEPLOY OPERATE COLLABORATION AGILE DEVELOPMENT CONTINUOUS INTEGRATION CONTINUOUS DELIVERY DEVOPS
  17. 17. Dawid Pacia “More automatically, more continuously, more risky...”
  18. 18. TESTOPS TOOLBOX AND BEST PRACTICES
  19. 19. GIT PROPER MODEL 19 time / commits master development feature 1 feature 2 release The Great QA Wall
  20. 20. Pipelines pipeline { agent none stages { stage('error') { steps { parallel( "Wait": { sleep 1 }, "NodeExec": { node(label: 'master') { isUnix() } … } stage('ExternalJob') { steps { build(job: 'TestMulti1', propagate: true) } ... Example
  21. 21. But here it comes… Blue Ocean!
  22. 22. ➜ You can use compose to bring up any number of instances of the services in your stack, example: > docker-compose up --scale web=10 database=3 This is great for testing database clusters or load balancers, or resiliency with Pumba Docker - Scalability Testing
  23. 23. ➜ Docker can be useful for testing different system configurations like for example different DB backends: ○ Bring up new stack (with MySQL backend), Seed with data, Run API tests, Tear down stack. ○ Bring up new stack (with Postgres backend), Seed with data, Run API tests, Tear down stack. ○ Report on any differences in test results between MySQL and Postgres backends. ○ Integrates easily with Jenkins ! Docker - Configuration Testing
  24. 24. Selenium Grid RIP
  25. 25. Regression Tests Jenkins CI Real-time reporting of Selenium tests on Kibana UI Tests Acceptance Tests Apache Kafka Elasticsearch Logstash Jobs Logs Logs Transformed documents Kibana Queryresults
  26. 26. Final Results
  27. 27. ➜ Setup overview ○ Configure Apache JMeter (through user.properties file) to Create a CSV File: /opt/apache/jmeter/apache-jmeter-3.2/bin/jmeter -n -t Basic Elasticsearch Test.jmx -l ./results.csv ○ Configure and Run Logstash ○ Visualize the Results Real-time reporting of JMeter results on Kibana
  28. 28. Final Results
  29. 29. ➜ TestOps provides a unique aspect for automated testing ➜ It works with the Dev team to set up CI/CD/CM ➜ It creates a cloud or virtual technology to set up a testing environment on demand ➜ It plays a vital role to integrate the automated testing with Dev team for CI ➜ Explore and investigate! Takeaway
  30. 30. Thanks! Any questions? You can find us at: janiszewski.tomasz@aeon.network pacia.dawid@aeon.network web: aeon.network

×