![Motivations
• Proving causality is important
• Traditionally requires controlled experiments
• Medicine and biology do it without controlled experiments [1]
• Add that they do it with observational studies
Methodological Issues in Observational Studies, @NyytiSaarimaki | 2
[1] J. W. Song and K. C. Chung,
“Observational studies: cohort and case-control studies,” 2010](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (19)
Similar a Observational Studies in Empirical Software Engineering
Similar a Observational Studies in Empirical Software Engineering (20)
Último
Último (20)
Observational Studies in Empirical Software Engineering
- 1. Observational Studies in Empirical Software Engineering Early Stage Nyyti Saarimäki Supervisor: Davide Taibi Tampere University, Finland IDoESE, 18 September 2019
- 2. Motivations • Proving causality is important • Traditionally requires controlled experiments • Medicine and biology do it without controlled experiments [1] • Add that they do it with observational studies Methodological Issues in Observational Studies, @NyytiSaarimaki | 2 [1] J. W. Song and K. C. Chung, “Observational studies: cohort and case-control studies,” 2010
- 3. Why Observational Studies? • Widely used in SE (MSR) without guidelines • Lots of historical data available • When properly conducted: • High level of evidence • Lower costs | 3Methodological Issues in Observational Studies, @NyytiSaarimaki
- 4. Goals • Adapt observational studies from epidemiology • Provide guidelines for conducting and reporting them Methodological Issues in Observational Studies, @NyytiSaarimaki | 4
- 5. Epidemiology • The study of disease human populations • Frequency • Distribution • Determinants • Assumption: • Diseases are not randomly distributed • Controlled experiments not always feasible • Resources (expensive) • Ethical issues | 5Methodological Issues in Observational Studies, @NyytiSaarimaki
- 6. Epidemiology • The study of disease human populations • Frequency • Distribution • Determinants • Assumption: • Diseases are not randomly distributed • Controlled experiments not always feasible • Ethical issues | 6Methodological Issues in Observational Studies, @NyytiSaarimaki Observational Studies
- 7. Level of Evidence of Methodologies (Medicine) Level of Evidence Qualifying Studies Used to Prove Causality 1 • Randomized controlled trial (High-quality) SLR x 2 • Randomized controlled trial (Lesser quality) SLR x 3 • Retrospective comparative study • Case-control study SLR x 4 • Case series 5 • Expert opinions • Case studies | 7Methodological Issues in Observational Studies, @NyytiSaarimaki Chung, K.C., Swanson, J.A., Schmitz, D., Sullivan, D. and Rohrich, R.J., 2009. Introducing evidence-based medicine to plastic and reconstructive surgery.
- 8. Methodologies and Guidelines (ESE) • Controlled experiment • Jedlitska et al. (2005) • Case Study • Runeson and Höst (2009) • Replications • Carver (2010) | 8Methodological Issues in Observational Studies, @NyytiSaarimaki • Systematic Literature Review • Kitchenham et al. (2009) •Mapping Study • Petersen et al. (2015) • Multivocal Literature Review • Garousi et al. (2019) • Observational Studies • ???
- 9. Observational Studies • Widely adopted in Epidemiology | 9Methodological Issues in Observational Studies, @NyytiSaarimaki Exposure Outcome Disease (Lung Cancer) Risk factor Treatment (Smoking) ?
- 10. Observational Studies | 10Methodological Issues in Observational Studies, @NyytiSaarimaki Exposure Outcome Disease (Lung Cancer) Risk factor Treatment (Smoking) ?
- 11. Observational Studies - Subjects • Two groups: • Cases • Controls • Similar characteristics between the groups • Inclusion criteria | 11Methodological Issues in Observational Studies, @NyytiSaarimaki
- 12. Observational Studies - Types • Cohort Studies • Case-Control Studies • Cross-Sectional Studies • Cannot prove causality | 12Methodological Issues in Observational Studies, @NyytiSaarimaki
- 13. Observational Studies – Prospective - Retrospective | 13Methodological Issues in Observational Studies, @NyytiSaarimaki
- 14. Cohort Study • Follow-up time for subjects: •Not diseased • Exposure to outcome | 14Methodological Issues in Observational Studies, @NyytiSaarimaki
- 15. Case-Control Study • Divide the groups based on • outcome • Investigate retrospectively the exposures • Outcome to exposure | 15Methodological Issues in Observational Studies, @NyytiSaarimaki
- 16. Future Steps • Step 1: Literature review • Methods currently used in empirical software engineering to prove causality • Empirical studies claiming to use observational studies • Step 2: How to apply observational studies in SE • Step 3: Evaluation of the proposal • Expert assessment • Replication of MSR existing studies with our proposal • Step 4: Proposing guidelines for conducting and reporting observational studies in SE | 16Methodological Issues in Observational Studies, @NyytiSaarimaki
- 17. Proving causality in empirical software engineering without running controlled experiments by applying observational methodologies adopted from epidemiology
Notas del editor
- - Collaboration Sira Vegas and Natalia Juristo
- The investigator “observes” and assesses the strength of the relationship between an exposure and outcome.
- MSR= mining software repositories