A presentation about the added value of combining qualitative and quantitative methods. It begins with a brief discussion of qualitative research and how it is distinct from yet shares basic principles with quantitative research, followed by a discussion of four important ways mixed methods -- integrating qualitative and quantitative -- adds value to our research efforts, and then a discussion of mixed methods research -- what it is, typologies, alternatives to typologies, and the use of diagrams.

1. The Added Value of
Mixed Methods Research
Margaret R. Roller, MA
March 2017

2. Mixed Methods – March 2017 ©Margaret R. Roller 1
 1 – Qualitative Research
 2 – What is the Added Value?
 3 – Mixed Methods Research
 4 – Resources to Learn More About
Mixed Methods & Develop New Skills
Overview

3. Mixed Methods – March 2017 ©Margaret R. Roller 2
Qualitative Research

4. Mixed Methods – March 2017 ©Margaret R. Roller 3
Qualitative Research
Qualitative research gives us knowledge about the
nature of human experience & social processes.
 By going beyond the obvious or expedient to more
deeply understand behavior & attitudes.
 By looking for the answer to any single question or
objective in a host of related questions.
Qualitative research is about making connections.
 One facet of something in a particular context adds
meaning to another facet in a different context.
Qualitative research keeps us connected to the
society in which our research is conducted.

5. Mixed Methods – March 2017 ©Margaret R. Roller 4
Qualitative Research

6. Mixed Methods – March 2017 ©Margaret R. Roller 5
Qualitative Research
Yet qualitative & quantitative research share
many of the same priorities.
 Coverage & sampling
 Mitigating bias
• Checking assumptions
• Interviewer training
 Mitigating nonresponse
• Maximizing cooperation
 Transparency
 Telling a story

7. Mixed Methods – March 2017 ©Margaret R. Roller 6
Qualitative Research

8. Mixed Methods – March 2017 ©Margaret R. Roller 7
What is the Added Value?

9. Mixed Methods – March 2017 ©Margaret R. Roller 8
What is the Added Value?
1. Deepens our commitment to the research question
by expanding our methodological framework.
 Diversifies capabilities
• Gives researchers looking for funding a competitive
advantage via more innovative designs.
 Rigor
• “...perhaps the most important lesson we learned was
that rigor meant not being beholden to a particular
method, but rather letting the questions point to the
methods even if it meant combining methods.”
(Weitzman & Levkoff, 2000)

10. Mixed Methods – March 2017 ©Margaret R. Roller 9
What is the Added Value?
2. Deepens our commitment to the research question
by expanding our understanding of attitudes &
behavior.
Health
 Cultural nuances among dementia caregivers, e.g., social
stigma of dementia (Weitzman & Levkoff, 2000)
 Gap between knowledge & self-care in heart failure
patients (Vaughan Dickson et al., 2011)
 What “fear of falling” means, e.g., isolation & dependency
(Jellesmark, et al. 2012)
 Motivations for prescription drug abuse (Rigg & Ibañez, 2010)

11. Mixed Methods – March 2017 ©Margaret R. Roller 10
What is the Added Value?
Education
 Procrastination & motivation among students with
learning disabilities (Klassen et al., 2008)
Environment
 Nuances of how people talk about doubt in climate
change, e.g., media “emphasizing ‘bad’ news for
financial gain.” (Capstick & Pidgeon, 2014)
Conservation
 Conservation adoption decision process among
farmers, e.g., importance of communication, rapport, &
incentives (Nyanga, 2012)
Food Safety
 Gap between knowledge & behavior (Meysenburg et al., 2014).

12. Mixed Methods – March 2017 ©Margaret R. Roller 11
What is the Added Value?
3. Deepens our commitment to the research question
by embracing the most deserving segments &
issues in society.
 Underrepresented, marginalized
• Children
• Ethnic & racial minority groups
• Those with physical &/or mental disabilities
• Lesbian, gay, bisexual, transgender
 “Wicked problems,” e.g., social injustice, climate
change
 Sensitive issues

13. Mixed Methods – March 2017 ©Margaret R. Roller 12
What is the Added Value?
4. Deepens & expands our research skills, advancing
our professional development.
 Sharpening conceptual thinking, focusing on the
research question rather than method.
 Learning to identify a broader set of questions/issues
outside the scope of a mono-method design.
 Working collaboratively across methods.
 Gaining new hands-on knowledge of the strengths &
limitations of qualitative research.
 Becoming “a researcher.”

14. Mixed Methods – March 2017 ©Margaret R. Roller 13
Mixed Methods Research

15. Mixed Methods – March 2017 ©Margaret R. Roller 14
Mixed Methods - Definition
“An approach to research in the social,
behavioral, and health sciences in which the
investigator gathers both quantitative
(closed-ended) and qualitative (open-ended)
data, integrates the two, and then draws
interpretations based on the combined
strengths of both sets of data to understand
research problems.” (Creswell, 2015)

16. Mixed Methods – March 2017 ©Margaret R. Roller 15
Mixed Methods – Purpose or Rationale
Triangulation, corroboration of results from
different methods.
Clarification, enhancement of results from one
method with the results from another method.
Development, using the results of one method
to inform or develop another method.
Expansion, broadening the scope of inquiry
using different methods for different aspects.

17. Mixed Methods – March 2017 ©Margaret R. Roller 16
Mixed Methods – Typologies
Qualitative QuantitativeIntegration

18. Mixed Methods – March 2017 ©Margaret R. Roller 17
Mixed Methods - Typologies
Sequential
 Exploratory
 Explanatory
Concurrent (convergent)

19. Mixed Methods – March 2017 ©Margaret R. Roller 18
Qualitative
Data collection & analysis
Quantitative
Data collection & analysis
Integrated
Interpretation
Sequential -
Exploratory
Mixed Methods - Typologies
Build/
Develop

20. Mixed Methods – March 2017 ©Margaret R. Roller 19
Sequential -
Exploratory
Mixed Methods - Typologies
How people talk about their skepticism of
climate change (Capstick & Pidgeon, 2014)
Ensuring culture appropriateness to study
dementia caregivers (Weitzman & Levkoff, 2000)
Identifying forms of client violence directed
at social workers (Enosh et al., 2015)

21. Mixed Methods – March 2017 ©Margaret R. Roller 20
Qualitative
Data collection & analysis
Quantitative
Data collection & analysis
Integrated
Interpretation
Sequential -
Explanatory
Mixed Methods - Typologies
Explain/
Clarify

22. Mixed Methods – March 2017 ©Margaret R. Roller 21
Sequential -
Explanatory
Mixed Methods - Typologies
Expand on factors related to
procrastination among students with
learning disabilities (Klassen, et al., 2008)
Better understand motivations to abuse
prescription drugs & segment variations
(Rigg & Ibañez, 2010)
Understand “fun” & “healthy” as well as
gender differences (Willenberg, et al., 2010)

23. Mixed Methods – March 2017 ©Margaret R. Roller 22
Qualitative
Data collection & analysis
Quantitative
Data collection & analysis
Integrated
Interpretation
Merge
Compare & contrast
Concurrent
(convergent)
Mixed Methods - Typologies

24. Mixed Methods – March 2017 ©Margaret R. Roller 23
Concurrent
(convergent)
Mixed Methods - Typologies
Compare, contrast, & understand
contextual factors related to conservation
adoption (Nyanga, 2012)
Compare, contrast, & enrich findings
related to knowledge, cognitive function, &
self-care among heart failure patients
(Vaughan Dickson, et al., 2011)
Enrich findings related to food preparers’
knowledge of food safety compared to
their behavior (Meysenburg, et al., 2010)

25. Mixed Methods – March 2017 ©Margaret R. Roller 24
Mixed Methods - Typologies
These basic typologies obscure the fact that our
research designs are rarely this simple &
straightforward.

26. Mixed Methods – March 2017 ©Margaret R. Roller 25
Qualitative
Data collection & analysis
Qualitative
Data collection & analysis
Quantitative
Data collection & analysis
Integrated
Interpretation
Mixed Methods - Typologies
Build/
Develop
Explain/
Clarify

27. Mixed Methods – March 2017 ©Margaret R. Roller 26
Qualitative
Data collection
& analysis
Qualitative
Data collection
& analysis
Quantitative
Data collection
& analysis
Quantitative
Data collection
& analysis
Quantitative
Pilot study
Integrated
Interpretation
Mixed Methods - Typologies
Build/Develop
Build/Develop

28. 27

29. Mixed Methods – March 2017 ©Margaret R. Roller 28
Mixed Methods - Typologies
Some have proposed the idea of “dimensions” to
accommodate the variations that can exist in
mixed methods designs.

30. 29Leech, N. L., & Onwuegbuzie, A. J. (2007). A typology of mixed methods research designs. Quality & Quantity, 43(2), 265–275.

31. Mixed Methods – March 2017 ©Margaret R. Roller 30
Mixed Methods - Typologies
Others have proposed different approaches,
such as Joseph Maxwell’s “interactive” model,
emphasizing the web or network of factors that
impact research design.

32. 31
Web of contextual factors influencing a mixed methods research design

33. Mixed Methods – March 2017 ©Margaret R. Roller 32
Diagrams
Play an Important Role
in Mixed Methods

34. 33

35. 34

36. 35

37. 36

38. 37

39. 38

40. Mixed Methods – March 2017 ©Margaret R. Roller 39
Resources to
Learn More About Mixed Methods
& Develop New Skills

41. Mixed Methods – March 2017 ©Margaret R. Roller 40
Resources
There are many mixed methods texts. Here are a few:

42. Mixed Methods – March 2017 ©Margaret R. Roller 41
Resources
Mixed methods journals & reports:
 Journals
• Journal of Mixed Methods Research
• International Journal of Multiple Research Approaches
 Mixed Methods International Research Association
(MMIRA)
• Mertens, D. M., Bazeley, P., Bowleg, L., Fielding, N.,
Maxwell, J., Molina-Azorin, J. F., & Niglas, K. (2016). The
future of mixed methods: A five year projection to 2020.
 NIH – OBSSR
• Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith,
K. C. (2011). Best practices for mixed methods research in
the health sciences. Methods (Vol. 29).

43. Mixed Methods – March 2017 ©Margaret R. Roller 42
Resources
Mixed methods communities:
 MethodSpace (SAGE Publishing)
 LinkedIn
• Qualitative & Mixed Methods Research
• Bridges: Mixed Methods Network for Behavioral,
Social, & Health Sciences

44. Mixed Methods – March 2017 ©Margaret R. Roller 43
Resources
Professional development:
 U. of Michigan
• Michigan Mixed Methods Research program
• Summer Institute
 UNC-The Odum Institute
• Qualitative Research Summer Intensive
 U. of Alberta-IIQM & MMIRA
• Mixed Methods Webinar Series
 Southwest Workshop on Mixed Methods
Research
• U. of California/Riverside, October 2017

45. Mixed Methods – March 2017 ©Margaret R. Roller 44
Resources
Professional development: (continued)
 Mixed Methods Research Training Program for the
Health Sciences (sponsored by NIH)
• Joint initiative by Johns Hopkins & Harvard
• Directed at early- to mid-career researchers w/ PhD
• Application process closed for 2017 but possible
opportunity for 2018
 Qualitative research training
• Burke, RIVA, Emory U.
• Roller – https://researchdesignreview.com/qualitative-
workshops-training/

46. Mixed Methods – March 2017 ©Margaret R. Roller 45
References

47. Mixed Methods – March 2017 ©Margaret R. Roller 46
References
Capstick, S. B., & Pidgeon, N. F. (2014). What is climate change scepticism? Examination of the concept using a mixed
methods study of the UK public. Global Environmental Change, 24(1), 389–401.
Creswell, J. W. (2015). A concise introduction to mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand
Oaks, CA: Sage Publications.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand
Oaks, CA: Sage.
Enosh, G., Tzafrir, S. S., & Stolovy, T. (2015). The development of client violence questionnaire (CVQ). Journal of Mixed
Methods Research, 9(3), 273–290.
Fetters, M.D., Yoshioka, T., Greenberg, G.M, Gorenflo, D.W., Yeo, S.A. (2007). Advance consent in Japanese during
prenatal care for epidural anesthesia during childbirth. Journal of Mixed Methods Research, 1(4), 333-365.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation
designs. Educational Evaluation and Policy Analysis, 11(3), 255–274.
Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of Mixed Methods
Research, 7(2), 141–151.

48. Mixed Methods – March 2017 ©Margaret R. Roller 47
References
Hesse-Biber, S. N., & Johnson, R. B. (Eds.). (2015). The Oxford handbook of multimethod and mixed methods research
inquiry. Oxford University Press.
Jellesmark, A., Herling, S. F., Egerod, I., & Beyer, N. (2012). Fear of falling and changed functional ability following hip
fracture among community-dwelling elderly people: An explanatory sequential mixed method study. Disability and
Rehabilitation, 34(25), 2124–2131.
Klassen, R. M., Krawchuk, L. L., Lynch, S. L., & Rajani, S. (2008). Procrastination and motivation of undergraduates
with learning disabilities: A mixed-methods inquiry. Learning Disabilities Research & Practice, 23(3), 137–147.
Leech, N. L., & Onwuegbuzie, A. J. (2007). A typology of mixed methods research designs. Quality & Quantity, 43(2),
265–275.
Maxwell, J. A., & Loomis, D. (2003). Mixed methods design: An alternative approach. In A. Tashakkori & C. Teddlie
(Eds.), Handbook of mixed methods in social and behavioral research (pp. 241–271). Thousand Oaks, CA: Sage.
Morse, J. M. (2016). Mixed method design: Principles and procedures. Routledge.
Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 241–271).
Thousand Oaks, CA: Sage.

49. Mixed Methods – March 2017 ©Margaret R. Roller 48
References
Mertens, D. M., Bazeley, P., Bowleg, L., Fielding, N., Maxwell, J., Molina-Azorin, J. F., & Niglas, K. (2016). The future
of mixed methods: A five year projection to 2020.
Meysenburg, R., Albrecht, J. A., Litchfield, R., & Ritter-Gooder, P. K. (2014). Food safety knowledge, practices and
beliefs of primary food preparers in families with young children: A mixed methods study. Appetite, 73, 121–131.
Nyanga, P. H. (2012). Factors influencing adoption and area under conservation agriculture: A mixed methods
approach. Sustainable Agriculture Research, 1(2), 27–40.
Rigg, K. K., & Ibañez, G. E. (2010). Motivations for non-medical prescription drug use: A mixed methods analysis.
Journal of Substance Abuse Treatment, 39(3), 236–247.
Vaughan Dickson, V., Lee, C. S., & Riegel, B. (2011). How do cognitive function and knowledge affect heart failure
self-care? Journal of Mixed Methods Research, 5(2), 167–189.
Weitzman, P. F., & Levkoff, S. E. (2000). Combining qualitative and quantitative methods in health research with
minority elders: Lessons from a study of dementia caregiving. Field Methods, 12(3), 195–208.
Willenberg, L. J., Ashbolt, R., Holland, D., Gibbs, L., MacDougall, C., Garrard, J., … Waters, E. (2010). Increasing
school playground physical activity: A mixed methods study combining environmental measures and children’s
perspectives. Journal of Science and Medicine in Sport, 13(2), 210–216.

50. Mixed Methods – March 2017 ©Margaret R. Roller 49
Thank You!
Margaret R. Roller
rmr@rollerresearch.com
804.693.3208
www.rollerresearch.com
www.researchdesignreview.com