1. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 1
QUANTITATIVE RESEARCH METHODS
SAMPLE OF
ANOVA-BASED PROCEDURES
Prepared by
Michael Ling

2. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 2
Problem
A consultant that develops training courses for users of a particular software wants to examine how
different characteristics of training courses influence knowledge gained about the software. Specifically,
the consultant is interested in the effects of two variables—(i) mode of delivery (face-to-face training
versus video-based training) and (ii) the provision of summary notes (provided versus not provided) on
knowledge about the software. The consultant recruits 160 individuals to participate in an experiment
designed to examine this issue. First, the consultant administers an intelligence (IQ) test to each
participant as intelligence is a factor that is known to influence how much knowledge individuals gain
from training courses. Each participant is then randomly allocated to one of four conditions (cells) that
are formed by crossing the two levels of the mode of delivery factor (i.e., whether the person gets face-
to-face training or video training; labelled “face-to-face” versus “video”) with the two levels of the
provision of summary notes factor (i.e., whether the person is provided with summary notes or not;
labelled “notes” versus “no notes”). The four conditions differ in the following way:
1. In the face-to-face/notes condition, participants spend three hours with an instructor who trains
them on how to use the software and then they receive notes that summarise the main points.
2. In the video/notes condition, participants spend three hours watching a video that trains them
how to use the software and then they receive notes that summarise the main points.
3. In the face-to-face/ no notes condition, participants spend three hours with an instructor who
trains them how to use the software however they do not receive any notes.
4. In the video/ no notes condition, participants spend three hours watching a video that trains
them how to use the software however they do not receive any notes.
The day following training each participant completes a knowledge test that assesses their
knowledge of how to use the software.
Dataset
The data from the experiment has been entered into the SPSS file SoftTrain.sav. The file contains the
following variables:
 id = A code used to identify each participant
 knowledge = The participants score on the knowledge test completed after training (out of 50).
 mode_of_delivery = The mode of delivery condition that the participant was in; video = 0, face-to-
face = 1
 provision_of_notes = Whether the participant received summary notes; no notes = 0, notes = 1
 IQ = The participants score on the intelligence test completed prior to training (out of 1000).

3. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 3
Instructions:
The researcher is interested in examining the main effects and interaction effect of mode of delivery and
provision of summary notes on knowledge (as measured by the score on the knowledge test). Open the
file and conduct the appropriate data analysis. Once you have analysed the data, write a report (no
longer than 2-3 double-spaced pages of text with 12pt font) in which you:
1. Describe how you analysed the data
2. Report the results of your data analysis, including an interpretation of what it means.
Solution
A 2x2 ANOVA/ANCOVA procedure was selected for the analysis because the
experiment consisted of (i) two independent factors (‘mode of delivery’ and ‘provision of notes’)
each of which had 2 levels and (ii) a single dependent variable (knowledge). In the early stage of
our analysis, testing was carried out to find out whether the intelligence (IQ) factor could be
considered as a covariate.
Testing IQ as a covariate
Treating the intelligence factor (IQ) as if it were a covariate, we used the tests of
Between-subjects Effects to study any changes in F ratios and error variance (Table 1, 2). We
found the total error variance was reduced to 5899.813 from 6081.175, which indicated that IQ
was, to a certain extent, related to the dependent variable. However, the F ratios were both
reduced for provision of notes (from 22.547 to 20.937) and mode of delivery (from 11.259 to
10.391), which was often an indication that the covariate (IQ) was not only correlated with the
dependent variable, but also with the between-groups factors. The covariate (IQ) not only
partitioned variance away from the error variance, but also from the variance due to the between-
groups factor. By removing the effect of IQ from the analysis, we had removed the true effects of

4. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 4
the between-group factors on the dependent variable. The effect of IQ was not limited to the
dependent variable alone.
In addition, the relationship between the covariate and the dependent variable was tested
and found to be nonlinear, as shown by the scatterplots (Figure 1, 2). The violation of this
assumption of ANCOVA reduced the power of the ANCOVA to find significant differences. As
a result, the use of IQ as a covariate had been ruled out from subsequent analysis.
Testing the basic assumptions of ANOVA
Data were analyzed to determine whether the basic assumptions of ANOVA procedure
were satisfied. The descriptive statistics were as shown in Table 3. The independence
assumption was satisfied as the participants were randomly allocated to one of the four groups.
The scale of measurement assumption was satisfied as the dependent variable (knowledge) was
an interval score from 0 to 50. As Levene’s Test for equality of error variance was non-
significant (Sig > 0.05), thus the assumption of homogeneity of variance had not been violated
(Table 4).
The skewness and kurtosis of the two groups in the mode of delivery factor were shown
in Table 5. Likewise, the skewness and kurtosis of the two groups in the provision of notes
factor were shown in Table 6. Their values lied between -1 and +1 which indicated that each of
the four groups was approximately normally distributed. This was also supported by the non-
significance of the Shapiro-Wilk statistics in the two groups of mode of delivery (Table 7) and
the two groups of provision of notes (Table 8). The box plots showed that there were no
outliners in the four groups (Figure 3, 4). Normality of the sampling distribution was attained.

5. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 5
Main and interaction effects
To investigate the main and interaction effects, we used ANOVA to examine four
sources of variability: (i) variability due to main effect of mode of delivery; (ii) variability due to
main effect of provision of notes; (iii) variability due to interaction effect of mode of delivery
and provision of notes; and (iv) within group (or error) variability.
As shown in Table 3, participants provided with notes (n=80, M = 29.3375, SD =
6.31844) achieved significantly higher scores than participants provided with no notes (n=80, M
= 24.6500, SD = 6.67908). The main effect of provision of notes factor was statistically
significant, F(1, 156) = 22.547, p < .005 (Table 1). Partial eta-square of the provision of notes
factor showed that it accounted for 12.6% of the overall variability (Table 1), which was a
‘medium’ effect.
As shown in Table 3, participants in face-to-face delivery mode (n=80, M = 28.6500, SD
= 6.84937) achieved significantly higher scores than participants in video delivery mode (n=80,
M = 25.3375, SD = 6.57179). The main effect of mode of delivery factor was also statistically
significant, F(1, 156) = 11.529, p < .05 (Table 1). Partial eta-square of the delivery factor
showed that it accounted for 6.7% of the overall variability (Table 1), which was a ‘medium’
effect.
The interaction effect of the mode of delivery and the provision of notes was
statistically significant, F (1, 156) = 4.053, p < 0.05 (Table 1). Partial eta-square of the
interaction effect showed that it accounted for 2.5% of the overall variability (Table 1), which
was a ‘small’ effect.
Nevertheless, with respect to the effect of mode of delivery on knowledge test, the profile
plot showed that it was an ordinal interaction. The mode of delivery had a stronger effect on the
knowledge test of the participants when they were not provided with notes than when they were
provided with notes (Figure 5).

6. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 6
With respect to the effect of provision of notes on knowledge test, the profile plot showed
that it was an ordinal interaction. The provision of notes had a stronger effect on the knowledge
test of the participants when they were trained in video mode than when they were trained in
face-to-face mode (Figure 6).

7. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 7
Appendix
Table 1: Tests of Between-Subjects Effects (with no covariate)
Dependent Variable:knowledge test
Source
Type III Sum of
Squares df Mean Square F Sig.
Partial Eta
Squared
Corrected Model 1475.819a
3 491.940 12.620 .000 .195
Intercept 116586.006 1 116586.006 2990.773 .000 .950
provision_of_notes 878.906 1 878.906 22.547 .000 .126
mode_of_delivery 438.906 1 438.906 11.259 .001 .067
provision_of_notes *
mode_of_delivery
158.006 1 158.006 4.053 .046 .025
Error 6081.175 156 38.982
Total 124143.000 160
Corrected Total 7556.994 159
a. R Squared = .195 (Adjusted R Squared = .180)
Table 2: Tests of Between-Subjects Effects (with IQ as covariate)
Dependent Variable:knowledge test
Source
Type III Sum of
Squares df Mean Square F Sig.
Partial Eta
Squared
Corrected Model 1657.180a
4 414.295 10.884 .000 .219
Intercept 1897.936 1 1897.936 49.863 .000 .243
IQ 181.362 1 181.362 4.765 .031 .030
provision_of_notes 796.945 1 796.945 20.937 .000 .119
mode_of_delivery 395.530 1 395.530 10.391 .002 .063
provision_of_notes *
mode_of_delivery
203.173 1 203.173 5.338 .022 .033
Error 5899.813 155 38.063
Total 124143.000 160
Corrected Total 7556.994 159
a. R Squared = .219 (Adjusted R Squared = .199)

8. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 8
Table 3: Descriptive Statistics
Dependent Variable:knowledge test
provision of notes factor mode of delivery factor Mean Std. Deviation N
no notes video 22.0000 5.59762 40
face-to-face 27.3000 6.68024 40
Total 24.6500 6.67908 80
notes video 28.6750 5.77078 40
face-to-face 30.0000 6.83130 40
Total 29.3375 6.31844 80
Total video 25.3375 6.57179 80
face-to-face 28.6500 6.84937 80
Total 26.9938 6.89407 160
Table 4: Levene’s Test of Equality of Error Variancea
Dependent Variable:knowledge test
F df1 df2 Sig.
.527 3 156 .664
Tests the null hypothesis that the error variance
of the dependent variable is equal across
groups.

9. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 9
Table 5: Descriptives of ‘Mode of Delivery’
mode of delivery factor Statistic Std. Error
knowledge test video Mean 25.3375 .73475
95% Confidence Interval for
Mean
Lower Bound 23.8750
Upper Bound 26.8000
5% Trimmed Mean 25.3611
Median 25.0000
Variance 43.188
Std. Deviation 6.57179
Minimum 9.00
Maximum 42.00
Range 33.00
Interquartile Range 8.75
Skewness .075 .269
Kurtosis .026 .532
face-to-face Mean 28.6500 .76578
95% Confidence Interval for
Mean
Lower Bound 27.1257
Upper Bound 30.1743
5% Trimmed Mean 28.5417
Median 29.0000
Variance 46.914
Std. Deviation 6.84937
Minimum 12.00
Maximum 49.00
Range 37.00
Interquartile Range 9.00
Skewness .181 .269
Kurtosis .427 .532

10. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 10
Table 6: Descriptives of ‘Provision of notes’
provision of notes factor Statistic Std. Error
knowledge test no notes Mean 24.6500 .74674
95% Confidence Interval for
Mean
Lower Bound 23.1636
Upper Bound 26.1364
5% Trimmed Mean 24.5000
Median 24.5000
Variance 44.610
Std. Deviation 6.67908
Minimum 9.00
Maximum 46.00
Range 37.00
Interquartile Range 8.75
Skewness .410 .269
Kurtosis .762 .532
notes Mean 29.3375 .70642
95% Confidence Interval for
Mean
Lower Bound 27.9314
Upper Bound 30.7436
5% Trimmed Mean 29.3333
Median 29.0000
Variance 39.923
Std. Deviation 6.31844
Minimum 12.00
Maximum 49.00
Range 37.00
Interquartile Range 7.75
Skewness .040 .269
Kurtosis .554 .532

11. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 11
Table 7: Tests of Normality for ‘mode of delivery’
mode of delivery factor
Kolmogorov-Smirnova
Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
knowledge test video .100 80 .046 .990 80 .774
face-to-face .059 80 .200*
.988 80 .684
a. Lilliefors Significance Correction
*. This is a lower bound of the true significance.
Table 8: Tests of Normality for ‘provision of notes’
provision of notes factor
Kolmogorov-Smirnova
Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
knowledge test no notes .092 80 .094 .983 80 .392
notes .066 80 .200*
.990 80 .765
a. Lilliefors Significance Correction
*. This is a lower bound of the true significance.

12. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 12
Figure 1: Scatterplot between IQ and Knowledge test; mode of delivery
as independent variable
Figure 2: Scatterplot between IQ and Knowledge test; provision of notes
as independent variable.
.

13. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 13
Figure 3: Effect of mode of delivery on knowledge test
Figure 4: Effect of provision of notes on knowledge test

14. ANOVA-BASED PROCEDURES July 2014 updated
Prepared by Michael Ling Page 14
Figure 5: Effect of mode of delivery on knowledge test
Figure 6: Effect of mode of delivery on knowledge test