all subcategories of Quantitative research design

1. An introduction to
quantitative data collection
Course: Research Methodology
Student: Zahra Bayani
instructor: Dr. Omid Mazandarani

2. Outline:
1. Quantitative research design: definition and categorization
2. classification of quantitative research design
3. Experimental research design, objectives and steps
4. Ethical issues in Experimental Research
5. Correlational research design, objectives and steps
6. Ethical issues in Correlational Research
7. Survey design, objectives and steps
8. Ethical issues in Survey Research
9. Conclusion
10. References
Fall, 2017

3. Research design
To understand educational research, we will explore
some distinguishing features that are the research
designs, that we can use to collect, analyze, and
interpret data using quantitative and qualitative
research.

4. In this presentation we will discuss about
three research designs frequently used in
quantitative educational research:
◆ Experimental Designs
◆ Correlational Designs
◆ Survey Designs

5. Experimental
Research
Designs

6. Experimental design:
An experimental design is the traditional approach to conducting
quantitative research. In an experiment, you test an idea to
determine whether it influences an outcome or dependent
variable. You first decide on an idea with which to “experiment,”
assign individuals to experience it and then determine whether
those who experienced the idea performed better on some
outcome than those who did not experience it(Creswell, 2015).

7. When Do You Use an Experiment?
You use an experiment when you
want to establish possible cause and
effect between your independent and
dependent variables.

8. WHAT ARE KEY CHARACTERISTICS OF EXPERIMENTS?
1. Random assignment
Random assignment is the process of assigning individuals at
random to groups or to different groups in an experiment.
2. Control over extraneous variables through:
Pretests and Posttests
Covariates
Covariates are variables that the researcher controls for using statistics and that
relate to the dependent variable.
Matching of Participants
Homogeneous Samples

9. Blocking Variables
A blocking variable is a variable the researcher controls before the
experiment starts by dividing the participants into subgroups.
3. Manipulation of the treatment conditions:
Treatment Variables
Conditions
Intervening in the Treatment Conditions
4. Outcome measures
5. Group comparisons
6. Threats to validity(Creswell, 2013)

10. WHAT ARE THE TYPES OF EXPERIMENTAL DESIGNS:
◆ Between Group Designs
• True experiments (pre- and posttest, posttest only)
• Quasi-experiments (pre- and posttest, posttest only)
• Factorial designs
◆ Within Group or Individual Designs
• Time series experiments (interrupted, equivalent)
• Repeated measures experiments
• Single subject experiments

11. 1. True Experiments
In true experiments, the researcher randomly assigns
participants to different conditions of the experimental variable.
Individuals in the experimental group receive the experimental
treatment, whereas those in the control group do not. After
investigators administer the treatment, they compile average (or
mean) scores on a posttest. One variation on this design is to
obtain pretest as well as posttest measures or observations. When
experimenters collect pretest scores, they may compare net
scores (the differences between the pre- and posttests).

12. 2. Quasi-Experiments
In education, many experimental situations occur in which
researchers need to use intact groups. This might happen
because of the availability of the participants or because the
setting prohibits forming artificial groups. Quasi-
experiments include assignment, but not random assignment
of participants to groups. This is because the experimenter
cannot artificially create groups for the experiment.

13. 3. Factorial Designs
Factorial designs represent a modification of the between
group design in which the researcher studies two or more
categorical, independent variables, each examined at two or
more levels. The purpose of this design is to study the
independent and simultaneous effects of two or more
independent treatment variables on an outcome (Perry, 2011).

14. Within-Group or
Individual Designs

15. 1.Time Series experiments
• The interrupted time series design: This procedure consists
of studying one group, obtaining multiple pretest measures for a
period of time, administering an intervention, and then measuring
outcomes (or posttests) several times.
• The equivalent time series design: in which the investigator
alternates a treatment with a posttest measure. The data analysis
then consists of comparing posttest measures or plotting them to
discern patterns in the data over time.

16. 2. Repeated measure
In a repeated measures design, all participants in a single
group participate in all experimental treatments, with each
group becoming its own control. The researcher compares a
group’s performance under one experimental treatment with
its performance under another experimental treatment. The
experimenter decides on multiple treatments (as in factorial
designs) but administers each separately to only one group.

17. 3. Single-Subject Designs
(Also called N of 1 research, behavior analysis, or within-
subjects research) involves the study of single individuals,
their observation over a baseline period, and the
administration of an intervention. This is followed by
another observation after the intervention to determine if the
treatment affects the outcome.

18. Different types of single-subject design:
1. A/B design:
An A/B design consists of observing and
measuring behavior during a trial period
(A), administering an intervention, and
observing and measuring the behavior
after the intervention (B).

19. 2. Multiple Baseline Design
It is a frequently used single-subject design.
In this design, each participant receives an
experimental treatment at a different, so that
treatment diffusion will not occur among
participants. Researchers choose this design
when the treatment cannot be reversed and
doing so would be unethical or injurious to
participants.

20. 3. Alternating Treatments
A final type of single-subject design is the
alternating treatment. An alternating
treatment design is a single-subject design
in which the researcher examines the relative
effects of two or more interventions and
determines which intervention is the more
effective treatment on the outcome (Perry, 2011).

21. What are the steps in
conducting experimental
research?

22. Step 1:
Decide if an Experiment Addresses Your Research Problem
Step 2:
Form Hypotheses to Test Cause-and-Effect Relationships
Step 3:
Select an Experimental Unit and Identify Study Participants
Step 4:
Select an Experimental Treatment and Introduce It

23. Step 5:
Choose a Type of Experimental Design
Step 6:
Conduct the Experiment
Step 7:
Organize and Analyze the Data
Step 8:
Develop an Experimental Research Report
(Creswell, 2015)

24. HOW DO YOU EVALUATE EXPERIMENTAL
RESEARCH?
 The experiment has a powerful intervention.
 The treatment groups are few in number.
 Participants will gain from the intervention.
 The researcher derives the number of participants per group in some
systematic way.
 An adequate number of participants were used in the study.
 The researcher uses measures and observations that are valid, reliable,
and sensitive.
 The researcher controls for extraneous factors that might influence the
outcome.
 The researcher addresses threats to internal and external validity.
(Nunan, 1992)

25. Ethical Issues in Experimental
Research
Ethical issues in conducting experiments relate to withholding the
experimental treatment from some individuals who might benefit from
receiving it, the disadvantages that might accrue from randomly
assigning individuals to groups. This assignment overlooks the
potential need of some individuals for beneficial treatment. Ethical
issues also arise as to when to conclude an experiment, whether the
experiment will provide the best answers to a problem, and
considerations about the stakes involved in conducting the experiment.

26. Correlational
Research
Designs

27. Correlational research
 In correlational research designs, investigators use the correlation
statistical test to describe and measure the degree of association (or
relationship) between two or more variables or sets of scores.
 A correlation is a statistical test to determine the tendency or
pattern for two (or more) variables or two sets of data to vary
consistently.

28.  In the case of only two variables, this means that two variables
share common variance, or they co-vary together. To say that two
variables co-vary has a somewhat complicated mathematical basis.
Co-vary means that we can predict a score on one variable with
knowledge about the individual’s score on another variable.
 The statistic that expresses a correlation statistic as a linear
relationship is the product–moment correlation coefficient. The
statistic is calculated or two variables (rxy) by multiplying the z
scores on X and Y for each case and then dividing by the number of
cases minus one.

29. When Do You Use Correlational
Research?
 You use this design when you seek to relate two or
more variables to see if they influence each other.
 You also use this design when you know and can
apply statistical knowledge based on calculating
the correlation statistical test.

30. WHAT ARE THE TYPES OF
CORRELATIONAL DESIGNS?
• The two primary correlation designs are:
 Explanatory
 prediction

31. 1. Explanatory design:
An explanatory research design is a correlational design in
which the researcher is interested in the extent to which two
variables (or more) co-vary, that is, where changes in one
variable are reflected in changes in the other. Explanatory
designs consist of a simple association between two
variables or more than two.

32. Common characteristics of explanatory design:
 The investigators correlate two or more variables.
 The researchers collect data at one point in time.
 The investigator analyzes all participants as a single group.
 The researcher obtains at least two scores for each
individual in the group—one for each variable.
 The researcher reports the use of the correlation statistical
test (or an extension of it) in the data analysis.
 Finally, the researcher makes interpretations or draws
conclusions from the statistical test results (Dornyei, 2007).

33. 2. The Prediction Design
• Instead of simply relating variables—two variables at a time
or a complex set, in a prediction design, researchers seek to
anticipate outcomes by using certain variables as predictors. The
purpose of a prediction research design is to identify variables
that will predict an outcome or criterion. In this form of research,
the investigator identifies one or more predictor variable and a
criterion (or outcome) variable. A predictor variable is a
variable used to make a forecast about an outcome in
correlational research.

34. WHAT ARE THE KEY CHARACTERISTICS
OF CORRELATIONAL DESIGNS?
As suggested by the explanatory and prediction
designs, correlation research includes specific
characteristics:
 Displays of scores (scatterplots and matrices)
 Associations between scores (direction,
form, and strength)
 Multiple variable analysis (partial
correlations and multiple regression)

35. 1. Displays of Scores
Scatterplots
A plot helps to assess association between two scores for
participants. A scatterplot (or scatter diagram) is a pictorial
image displayed on a graph of two sets of scores for
participants.
A Correlation Matrix
Correlation researchers typically display correlation coefficients
in a matrix. A correlation matrix presents a visual display of the
correlation coefficients for all variables in a study.

36. 2. Associations between Scores
What Is the Direction of the
Association?
In a positive correlation (indicated by a “1”
correlation coefficient) the points move in the same
direction;
In a negative correlation (indicated by a “–”
correlation coefficient), the points move in the
opposite direction; If scores on one variable do not
relate in any pattern on the other variable, then no
linear association exists.

37. What Is the Form of the Association?
Correlational researchers identify the form of
the plotted scores as linear or nonlinear:
• Linear Relationship depicts a positive
linear relationship of scores, where low (or
high) scores on one variable relate to low (or
high) scores on a second variable.

38. • A negative linear relationship results,
where low scores on one variable relate to
high scores on the other variable. Low
scores on depression, for example, might
be associated with high scores on use of
the Internet, suggesting a negative
relationship.

39. • An uncorrelated relationship of scores:
In this distribution, the variables are
independent of each other. A particular score
on one variable does not predict or tell us any
information about the possible score on the
other variable.

40. What Is the Degree and Strength of
Association?
Degree of association means that the association
between two variables or sets of scores is a
correlation coefficient of –1.00 to +1.00, with 0.00
indicating no linear association at all. This
association between two sets of scores reflects
whether there is a consistent, predictable association
between the scores ( Gravetter & Wallnau, 2007 ).

41. 3. Multiple Variable Analysis
In many correlation studies, researchers predict outcomes
based on more than one predictor variable. Thus, they need
to account for the impact of each variable.
Two multiple variable analysis approaches are:
• partial correlations
• multiple regression

42. 1. Partial Correlations
In many research situations, we study three, four, or five variables
as predictors of outcomes. The type of variable called a mediating
or intervening variable “stands between” the independent and
dependent variables and influences both of them. This variable is
different from a control variable that influences the outcome in an
experiment. We use partial correlations to determine the amount
of variance that an intervening variable explains in both the
independent and dependent variables.

43. 2. Multiple Regression
Multiple correlation is a statistical procedure for examining the
combined relationship of multiple independent variables with a single
dependent variable.
• A beta weight is a coefficient indicating the magnitude of
prediction for a variable after removing the effects of all other
predictors. The coefficient of a beta weight identifies the strength of
the relationship of a predictor variable of the outcomes.

44. Meta-Analysis
• In another extension of correlation research, authors
integrate the findings of many research studies in a meta-
analysis by evaluating the results of individual studies
and deriving an overall numeric index of the magnitude of
results (Perry, 2011).

45. Steps for correlational research:
Step 1: Identify Individuals to Study
Step 2: Identify Two or More Measures for Each
Individual in the Study
Step 3: Collect Data and Monitor Potential Threats
Step 4: Analyze the Data and Represent the Results
Step 5: Interpret the Results

46. Ethical Issues in Conducting Correlational
Research
Ethical issues arise in many phases of the correlational
research process. In data collection, ethics relate to adequate
sample size, lack of control, and the inclusion of as many
predictors as possible. In data analysis, researchers need a
complete statement of findings to include effect size and the
use of appropriate statistics. Analysis cannot include making
up data. In recording and presenting studies, the write-up
should include statements about relationships rather than
causation, a willingness to share data, and publishing in
scholarly outlets (Creswell, 2015).

47. Survey
Research
Designs

48. Survey research design
• Survey research designs are procedures in quantitative
research in which investigators administer a survey to a
sample or to the entire population of people to describe
the attitudes, opinions, behaviors, or characteristics of the
population. Survey designs differ from experimental
research in that they do not involve a treatment given to
participants by the researcher.

49. When Do You Use Survey Research?
 You use survey research to describe trends, such
as community interests in school .
 You also use survey research to determine
individual opinions about policy issues, such as
whether students need a choice of schools to
attend.
 Surveys help identify important beliefs and
attitudes of individuals, such as college students’
beliefs about what constitutes abusive behaviors in
dating relationships.

50. WHAT ARE THE TYPES OF SURVEY
DESIGNS?
• Despite the many applications of surveys
today, there are still only two basic types of
research surveys:
• Cross sectional
• Longitudinal

51. 1. Cross-Sectional Survey Designs
• The most popular form of survey design used in education is a
cross-sectional survey design. In a cross-sectional survey design, the
researcher collects data at one point in time. This design has the
advantage of measuring current attitudes or practices and also
provides information in a short amount of time.

52. Several features of Cross-sectional designs:
 A cross-sectional study can examine current attitudes, beliefs, opinions, or
practices.
 Another cross-sectional design compares two or more educational groups in
terms of attitudes, beliefs, opinions, or practices.
 A cross-sectional design can measure community needs of educational
services as they relate to programs, courses, school facilities projects, or
involvement in the schools or in community planning.
 Some cross-sectional designs evaluate a program, such as a survey that
provides useful information to decision makers.
 A final type of cross-sectional design is a large-scale assessment of students
or teachers, such as a statewide study or a national survey involving
thousands of participants.

53. 2. Longitudinal Survey Designs
• A longitudinal survey design involves the survey
procedure of collecting data about trends with the same
population, changes in a cohort group or subpopulation, or
changes in a panel group of the same individuals over time.
Thus, in longitudinal designs, the participants may be
different or the same people (Dornyei, 2007).

54. Several types of longitudinal
designs:
Trend Studies
Cohort Studies
Panel Studies

55. 1. Trend Studies
• In some surveys, researchers aim to study changes within
some general population over a period of time. This form of
longitudinal research is called a trend study. Trend studies
are longitudinal survey designs that involve identifying a
population and examining changes within that population
over time.

56. 2. Cohort Studies
• A cohort study is a longitudinal survey design in which a
researcher identifies a subpopulation based on some
specific characteristic and then studies that subpopulation
over time. All members of the cohort must have the
common characteristic, such as being 18 years old in the
year 2001.

57. 3. Panel Studies
• A panel study is a longitudinal survey design in which the
researcher examines the same people over time. One
disadvantage of a panel design is that individuals may be
difficult to locate, the advantage to this type of study, however,
is that the individuals studied will be the same each time,
allowing the researcher to determine actual changes in specific
individuals (Creswell, 2015)

58. WHAT ARE THE KEY CHARACTERISTICS
OF SURVEY RESEARCH?
 Sampling from a population
 Collecting data through questionnaires or
interviews
 Designing instruments for data collection
 Obtaining a high response rate

59. WAYS OF COLLECTING DATA IN SURVEY
RESEARCH:
Questionnaires
Interviews

60. Questionnaire
• A questionnaire is a form used in a survey design that
participants in a study complete and return to the researcher.
The participant chooses answers to questions and supplies
basic personal or demographic information.

61. An interview survey
• An interview survey, is a form on which the researcher
records answers supplied by the participant in the study. The
researcher asks a question from an interview guide listens
for answers or observes behavior, and records responses on
the survey.

62. Major types of questionnaires and interviews:
 Mailed questionnaires
 Web-based questionnaires
 One-on-one interviews
 Focus group interviews
 Telephone interviews

63. 1. Mailed Questionnaires
• A mailed questionnaire is a form of data collection in
survey research in which the investigator mails a
questionnaire to members of the sample. Researchers might
develop their own questionnaire, modify an existing one, or
use one that they have located in the literature. A mailed
questionnaire is a convenient way to reach a geographically
dispersed sample of a population.

64. 2. Web-Based Surveys or
Questionnaires
• A Web-based questionnaire is a survey instrument for
collecting data that is available on the computer. Several
software programs are available for designing, gathering,
and analyzing survey data with sample questions and forms
e.g., Qualtrix at http://www.qualtrics.com/survey-
software/ or Survey Monkey at
http://www.surveymonkey.com.

65. 3. One-on-One Interviews
In one-on-one interviewing in survey research,
investigators conduct an interview with an individual in the
sample and record responses to closed-ended questions. The
process involves developing or locating an instrument and
training the interviewer(s) in good interview procedures.
One-on-one interviews are useful for asking sensitive
questions and enabling interviewees to ask questions or
provide comments that go beyond the initial questions.

66. 4. Focus Group Interviews
• In quantitative focus group interviews in survey
research, the researcher locates or develops a survey
instrument, convenes a small group of people (typically a
group of 4 to 6) who can answer the questions, and
records their comments on the instrument. Focus groups
provide for interaction among interviewees, collection of
extensive data, and participation by all individuals in a
group.

67. 5. Telephone Interviews
• In telephone interview surveys, the researcher records
the participants’ comments to questions on instruments
over the telephone. The researcher develops or locates an
instrument, obtains the telephone numbers of participants
in the sample, conducts the telephone calls, and asks the
participants to answer questions on the instrument.
Telephone interviews allow the researcher easy access to
interviewees who are geographically dispersed (Krueger, 1994).

68. Types of survey questions:
Personal, Attitudinal, and Behavioral Questions
Sensitive Questions
Open- and Closed-Ended Questions
Semi-closed-ended questions

69. Pilot Testing the Questions
A pilot test of a questionnaire or interview survey is a procedure
in which a researcher makes changes in an instrument based on
feedback from a small number of individuals who complete and
evaluate the instrument. The participants in the pilot test provide
written comments directly on the survey, and the researcher
modifies or changes the survey to reflect those concerns.

70. WHAT ARE THE STEPS IN CONDUCTING SURVEY
RESEARCH?
Step 1. Decide if a survey is the best design to use
Step 2. Identify the research questions or hypotheses
Step 3. Identify the population, the sampling frame, and the sample
Step 4. Determine the survey design and data collection procedures
Step 5. Develop or locate an instrument
Step 6. Administer the instrument
Step 7. Analyze the data to address the research questions or hypotheses
Step 8. Write the report

71. Potential Ethical Issues in Survey Research
• Ethical issues in survey research involve engaging in good
practices. During data collection, attention needs to be given to
using appropriate incentives and delivering on benefits
guaranteed. The survey data collection procedure cannot put
data collectors at risk for their safety. Safety applies to the
respondents or participants as well. Confidentiality of their
responses needs to be protected, along with minimizing links
between data respondents and participants.
• IDs linked to responses can be an effective means of
protecting individual identity. Also, the researcher has an
obligation to destroy survey instruments after the conclusion of
the study.

72. References:
Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches (4th
ed.). Thousand Oaks, CA: Sage.
Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and
qualitative research (5th ed.). Pearson
Dornyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative and mixed
methodologies. Oxford: Oxford University Press
Gravetter, F. J., & Wallnau, L. B. (2007). Statistics for the behavioral sciences (7th ed.). Belmont, CA:
Thomson Learning.
Krueger, R. A. (1994). Focus groups: A practical guide for applied research (2nd ed.). Thousand Oaks,
CA: Sage.
Nunan, D. (1992). Research methods in language learning. Cambridge University Press.
Perry, F. L. (2011). Research in applied linguistics: Becoming a discerning consumer (2nd ed.). New
York, NY: Routledge