Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

Testing the unknown: the art and science of working with hypothesis

Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Próximo SlideShare
Agile practice guide
Agile practice guide
Cargando en…3
×

Eche un vistazo a continuación

1 de 29 Anuncio

Testing the unknown: the art and science of working with hypothesis

Descargar para leer sin conexión

Testing what we know, or have a clear understanding of, is relatively straight forward, as is making decisions based on the expected result. But today’s world is presenting us with the Unknown and the Ambiguous, which can only be approached by hypothesizing and experimenting - a lot! This requires intentional thinking, and a different strategy to observe in context.

This session will uncover how testers are helping their teams and product owners, by basing their testing on the science behind creating hypotheses and running experiments. A testing mindset and probing the context around use cases are some of the most valuable competencies testers bring to the team in order to enable decisions based on data.

Testing what we know, or have a clear understanding of, is relatively straight forward, as is making decisions based on the expected result. But today’s world is presenting us with the Unknown and the Ambiguous, which can only be approached by hypothesizing and experimenting - a lot! This requires intentional thinking, and a different strategy to observe in context.

This session will uncover how testers are helping their teams and product owners, by basing their testing on the science behind creating hypotheses and running experiments. A testing mindset and probing the context around use cases are some of the most valuable competencies testers bring to the team in order to enable decisions based on data.

Anuncio
Anuncio

Más Contenido Relacionado

Presentaciones para usted (20)

Similares a Testing the unknown: the art and science of working with hypothesis (20)

Anuncio

Más reciente (20)

Anuncio

Testing the unknown: the art and science of working with hypothesis

  1. 1. Agile and Testing the Unknown: The Art and Science with guest speaker Ardita Karaj Agenda: Presentation 6:15 p.m. to 7:15 p.m. Q&A 7:15 p.m. to 7:30 p.m. Event Survey October 27, 2020
  2. 2. Welcome! All people will start the meeting in Mute mode. We ask you to remain in Mute for the duration of the meeting to avoid background noise. Questions can be put into the chat window. Someone will monitor them and consolidate similar questions into one. The meeting will be recorded.
  3. 3. Welcome from the TASSQ Board of Directors Desislava Nikolova President GuillaumeVulliez Vice President Amar Gadgil Board Member Neil Price-Jones Treasurer Venugopal Pula Board Member Vida Shafaghat Secretary RobVirdee Board Member Bob Thompson Board Member Tatiana Andronache Board Member
  4. 4. Agile Practices and Testing: The Art & Science of Testing the Unknown Ardita Karaj with TASSQ October 27, 2020
  5. 5. About me Enterprise coach, trainer and change agent, experienced in Product Management, Business improvement and Lean Startup. 
 Started the career as a software engineer, worked with teams that pushed edges of technology and won awards. Taking the mentor and leader stance, I found that success came when business and technology worked closer together. 20 years of experience working with many organization of different types (from startup to government), different markets (insurance, banking, retail, entertainment, etc) and different sizes. Worked with many leaders to find innovative and collaborative ways for their organizations to increase customer satisfaction and build successful businesses. With a vision for using technology as a competitive advantage, and understanding the need for organizations to become leaders by disrupting markets, founded Industrial Logic Canada offering professional services with focus on Continuous Delivery.
 I organize and speak in several conferences around the World every year.
  6. 6. Experimentation 6
  7. 7. Future of Testing is Postmodern https://medium.com/@jarbon/ Testing is the art and science of taking a piece of software and trying all sorts of ways to break it…. Jason Arbon …Our community has spent too much time finding which school is correct: “Analytical”, “Risk-Based”, “Quality Assurance”, “Modern”, “Context-Driven”, etc.
  8. 8. How testers evolved testing, deliberately Evolution on development came from Testing mindset • Chaos Monkey/Gorilla/Kong • Property based testing • Exploratory testing • Use of AI for dynamic features • Testers as Machine Learning engineers • …
  9. 9. • Defects • Performance • Correct data • Security • Accessibility • Ease of Use • Richness of features • No unused features • Flow of actions • User friendly • Desired by client • Refactored • Tested • Clean • Integrated • Easy to read • Easy to change Product Quality Code Quality
  10. 10. Where do we need testing to start? Expand testing mindset to Business • Shift left, early in the process • Not just test performance and matching requirement • Start by testing the need for those requirement • Start by testing the business idea behind those requirements
  11. 11. STRATEGY An approach to achieve an end goal with the means we have Strategy provides a framework on how to make decisions based on a set of values and a period of time ahead
  12. 12. Cynefin a sense-making framework https://www.cognitive-edge.com/cynefin-st-davids-day-2020-1-of-n/ Aporetic/Confused
  13. 13. How much structure do we need? Clear/Obvious – Just build it. If it’s too big, slice it to stories/scenarios, and do the most valuable ones first. Complicated/Needs some planning – Do Story Mapping, write all the stories, and prioritize the most valuable and/or most risky ones first Complex/A lot of unknowns – Don’t try to find all the stories. Identify one or two experiments that will provide some value and teach you something about the problem and its solution. Use what you learn to evolve on the rest. Chaotic – Put out the fire first; writing and splitting stories probably isn’t important right now. Confused/Disordered – Figure out in which situation you’re in before acting so you don’t take the wrong approach. Is the request: 13
  14. 14. Forming scientific hypothesis • For a hypothesis to be scientific, the scientific method requires that one can test it.
 • Hypothesis is Testable, Precise and Discrete (see David Bland https://precoil.com)
 • Hypothesis is a proposed explanation of a situation/context
 • To validate/invalidate an hypothesis, we need to understand the variables that affect the results
 • Changing more than 1 variable for each experiment will result in hard to define affects Variables: - Quality of seeds - Water - Temperature - Fertilizer
  15. 15. Forming and implementing an Hypothesis 5 steps: 1.Ask a question 2.Initial research 3.Formulate hypothesis 4.Run experiment 5.Answer question or Refine Hypothesis 1. How can I be healthier? 2. Food, Exercise, Mental health, Sleep 3. I believe that by moving to a vegetarian diet, I will have less muscle inflammatory pain. I will know I am successful when I feel 70% less muscle pain every day. 4. No meat for 2 weeks. Measure pain 5. I did have less pain but was 40%. I also noticed better sleep. • New: I believe that by moving to a plant based diet, I will have less muscle inflammatory pain. I will know I am successful when I feel 70% less muscle pain every day.
  16. 16. 17
  17. 17. Where is your Test column on the board? At the end: Testing is simply a verification, an after thought, a late insight for improvement of the solution At the front: Testing leads the development and brings better ways to architect the solution and how to offer it to clients
  18. 18. Learning Release(s) —> Earning Release https://medium.com/@Ardita_K/the-learning-release-70374d2450b3 What is expected from a Release?
  19. 19. 20 Product Teams use testing strategies to Instrument the product. Products need to be instrumented to gather info/metrics 
 
 How do we know we achieved the ROI?
  20. 20. Instrumentation Framework ● No instrumentation has ambiguity ● Bad instrumentation is costly 21
  21. 21. Example on setting up an experiment 22
  22. 22. Example on setting up an experiment 23
  23. 23. Hypothesis We believe that by doing ____ we will see ____ as result We know we are successful when we observe _____
  24. 24. Fast, Frugal Learning with a Feature Fake 25 https://www.industriallogic.com/blog/fast-frugal-learning-with-a-feature-fake/ The business goal: Increase engagement of the community Feature: Group chat
  25. 25. Fast, Frugal Learning with a Feature Fake 26 https://www.industriallogic.com/blog/fast-frugal-learning-with-a-feature-fake/ Experiment lasted 3 days Did not provide compelling data to make a real investment on GroupChat feature
  26. 26. Lost my name The business goal: Increase user experience on checkout 
 Feature: New wrapping paper 27
  27. 27. Takeaways… • Testing mindset is needed early in Agile Product teams. Do we know the problem? • Test not only the functionality, Test the Business Idea • Embed Learning in the Product (Instrumentation) • Experiment to Learn before deciding how much to earn • Build Hypothesis and run Experiments in 5 steps • Define the parameters and experiment by changing one at a time • Testers can help their agile teams early in the process; bring focus
  28. 28. Thank you! QUESTIONS AND DISCUSSIONS Hypothesis, Experiment,Testing mindset, Shift Left … Ardita Karaj, ardita@IndustrialLogic.ca

×