Researchers use several tools and procedures for analyzing quantitative data obtained from different types of experimental designs. Different designs call for different methods of analysis. This presentation focuses on:
T-test
Analysis of variance (F-test), and
Chi-square test
2. The experimental design determines the
statistical test to be used to analyze the data.
There are several tools and procedures for
analyzing quantitative data obtained from
different types of experimental designs.
Different designs call for different methods of
analysis. This presentation focuses on:
1. T-test
2. Analysis of variance (F-test), and
3. Chi-square test
Experimental Design and Statistics
3. The Logic of Significance Testing
The results of an inferential
statistical test informs whether
the results of an experiment
would occur frequently or
rarely by chance.
Inferential statistical test with
small p values occur frequently
by chance (accept the null
hypothesis), whereas large
values occur rarely by chance
(reject the null hypothesis).
4. The Logic of Significance Testing
P value is the probability at
which the null hypothesis will
be rejected when it is true.
Traditionally statisticians say
that any event that occurs by
chance 5 times or fewer in 100
occasions is a rare event. (i.e.,
.05 level of significance).
5. T- Test
When the means of two independent groups are
to be compared, we can use the T- test. This test
can help determine how confident we can be that
the differences between two groups as a result of
the treatment is not due to chance. The
researcher calculates a t-value using the sample
mean and standard deviation and compares the
calculated t-value against a tabulated value. If null
hypothesis is rejected, we can say that the
difference between the two groups is significant.
6. Example for t-test
A researcher compares performances of two randomly
selected groups learning French. The two groups,
follow up their frontal lessons with practice sessions.
the Experimental Group gets practice sessions with
the aid of the computer.
the Control group has practice sessions with a
teacher.
The researcher investigates the effects of the
computer practice session on students’ achievement
in French.
7. Hypotheses testing
210 μμ:H
211 μμ:H
Mean scores of the two
groups are equal
2
ˆˆ
μˆ-μˆ
2
1
1
21
nn
tcal
Suppose, upon calculation, the researcher finds the
tcal = 1.99
The researcher can use t-test to test the hypothesis
8. Example for t-test
Upon comparison with the table value of
T at p=0.05, it is found that tcal > ttab
This means the null hypothesis is
rejected and the differences between
two groups are significantly different.
The result is reported as t=1.99, p=.05
9. One-Way ANOVA
When there are more than two groups, the
appropriate procedure is ‘ANOVA’ where we
need to analyze the variability within groups and
variability between groups. The test we do in
ANOVA is the F-test
10. One-Way ANOVA
3210 μμμ:H
samethearemeanspopulationtheofallNot:1H
The “F-test”
groupswithinyVariabilit
groupsbetweenyVariabilit
F
12. Example
Step 1) calculate the sum of
squares between groups:
Mean for group 1 = 62.0
Mean for group 2 = 59.7
Mean for group 3 = 50.3
Mean for group 4 = 65.4
Grand mean= 59.85
SSB = [(62-59.85)2 + (59.7-59.85)2 + (50.3-59.85)2 + (65.4-59.85)2 ] x n per
group= 19.65x10 = 1266.6 , n= number of observations in each group
TM1 TM2 TM3 TM4
60 50 40 57
67 52 45 67
42 43 43 54
67 67 55 67
56 67 46 69
62 59 61 69
64 67 45 68
59 64 52 65
72 63 53 70
71 65 63 68
14. Step 3) Fill in the ANOVA table
3 1266.6 422.2 7.38 0.001
36 2060.6 57.2
Source of variation d.f. Sum of squares Mean Sum of
Squares
F-statistic p-value
Between
Within
Total 39 3327.2
F value=
𝑉𝑎𝑟𝑖𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑔𝑟𝑜𝑢𝑝/𝑑𝑓
𝑉𝑎𝑟𝑖𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑤𝑖𝑡ℎ𝑖𝑛 𝑔𝑟𝑜𝑢𝑝/𝑑𝑓
15. Factorial ANOVA (Two way)
When two factors or more than two factors are
involved.(age, gender. level of competence)
The aim is to test if there is also an interaction
between teaching method and gender/age/etc.
18. Using the between, within and
interaction sum of squares, we create
the ANOVA table and calculate F-
Statistic associated with main effects
and interaction effects
Hypothesis test procedure will be the
same as before
19. Chi Square (Χ2)
Where o = observed frequencies, and
e = expected frequencies
The most obvious difference between the chi-square
tests and the other hypothesis tests we have
considered (t and ANOVA) is the nature of the data.
For chi-square, the data are frequencies rather than
numerical scores.
Chi Squared is used to observe the difference
between what we actually observe and what we
expect to find if the null hypothesis is true.
The chi-square statistic is calculated as follows
20. Data was collected on citizen’s viewpoints about building of the
2012 Olympic venue at Stratford and tried to find if viewpoints
changed according to the perspectives of different groups.
Through a survey/questionnaire, 20 responses from each
category of local person were collected, about the usefulness of
the new Olympic developments. The statement posed:
‘The 2012 Olympic Games development will be of benefit to the
whole community of Stratford, east London.’
1 2 3 4
Strongly agree Agree Disagree Strongly disagree
Case Study: Using chi-squared to
analyse questionnaire responses
21. Results of the survey.
Category (type) Frequency of negative
responses (Observed
values: o)
Business owner 4
School student 6
Adult male resident 14
Adult female resident 10
Senior citizen 16
20 people responded from each category and only the
frequency of negative response, i.e. those who either
disagreed or strongly disagreed with the statement.
22. The expected data (e) is the mean negative frequency of response,
calculated by adding up the observed data (o) and then dividing
by the number of categories, i.e. 5. This gives an expected
frequency of 10 for each category.
Business
owner
School
student
Adult
male
resident
Adult
female
resident
Senior
citizen
Total
o 4 6 14 10 16 50
e 10 10 10 10 10 50
o - e -6 -4 4 0 6 --------
(o – e)² 36 16 16 0 36 --------
(o – e)²
e
3.6 1.6 1.6 0 3.6 --------
x² 3.6 1.6 1.6 0 3.6 10.4
23. Interpreting the Chi-Squared Value
Calculated chi-square value= 10.4
4 degrees of freedom.
Critical values for 4 df are:
Confidence level 0.10
90%
0.05
95%
0.01
99%
0.005
99.5%
Critical value 7.78 9.49 13.28 14.86
To reject the null hypothesis (Hₒ), chi-squared score must be
greater than the critical value at the 0.05 level of significance.
Since 10.4 is higher than the 0.05 level of significance- which
is 9.49, we can reject the null hypothesis (Hₒ).
24. Summary of different
statistical procedures
Different types of data analysis are appropriate for
different types of research problems.
Qualitative: data collection of procedures of a low
level of explicitness.
Descriptive: use different types of descriptive
statistics (frequencies, central tendencies, and
variabilities).
Correlational analysis: examination of the
relationships between variables.
Multivariate procedures: more complex
relationships, dealing with a numbers of variables at a
time.
25. Experimental research
procedures for analyzing data from experimental
research:
T-test: helps examine whether the differences
between two samples are statistically significant.
One way analysis of variance: examines
differences between more than two groups;
Factorial analysis of variance: analyzing the
effect of a treatment under more complex
conditions.
Chi square: compare frequencies observed in a
sample with some theoretically expected frequencies.
26. Using Computer for Data
Analysis
The most popular is SPSS and its
updated version. (the researchers are
advised to find out which packages are
available when preparing the research
proposal.
27. Different phases in performing
computer data analysis
Phase 1: prepare the data collection tools
with a coding system integrated into the
procedures.
Phase 2: the data are transferred to coding
sheets. (Example see next slide)
Phase 3: the data are transferred to the
computer database. (with professional helps)
Phase 4: choose an appropriate program for
the analysis. (experts advise is encouraged)
Phase 5: get results.
29. Caution for the researcher during
the data analysis
Should have a “feel” for the results and
to use intuition. (false results, or error)
Keep a close watch on the results
(sensible).
Understand the statistics used for the
data analysis.
Acquaint themselves with the specific
statistical procedures.
Notas del editor
T- test, f- test etc, are inferential tests. If we conduct an experiment say, a 100 times, and a certain result occurs 95 times, then we can be confident in the result of the experiment. But this inference is drawn from the statistical test constructed based on only 1 experiment
T- test, f- test etc, are inferential tests. If we conduct an experiment say, a 100 times, and a certain result occurs 95 times, then w aan be confident in the result of the experiment. But this inference is drawn from the statistical test constructed based on only 1 experiment
P value is a chance of error in an hypothesis test
T test tells if thre is any significant diff or not between the 2 grps
The researcher administers same test for both groups and compares avg scores across 2 grps.
T cal is based on the sample mean and SD values. N is the sample size.
T- tab- 1.96
T- tab- 1.96
When there are more than two groups, the appropriate procedure is ‘ANOVA’ where we need to analyze the variability within groups and variability between groups. The test we do in Anova is the F-test
4 groups of 10 students each are taught by diff methods, then they given a same test and the following scores are obtained.
Variability between groups is captured by sum of squares between groups (SSB)
Variability within groups is captured by sum of squares within groups
The df- number of independent observations in the sample that are available to estimate the population parameter is called the degree of Freedom or df. For example, if we have 4 observations and the mean is 10, then it is possible to assign values to four observations but the fifth one will be fixed depending the mean. So in this case the df is 4-1=3.
Which means perhaps some teaching methods may be more suited for males or for females . We can know if males and females are responding differently to different methods but from the test we can not necessarily tell which method may be more suited for who .
Two factors or more than two factors are involved. (age, level of competence) Here what we aim to test is if there is also an interaction between teaching method and gender. Which means perhaps some teaching methods may be more suited for males or for females . We can know if males and females are responding differently to different methods but from the test we can not necessarily tell which method may be more suited for who .
Here
What is main effect – effect of teaching method or effect of gender.
What is interaction effect – whether boys or girls perform differently across different teaching methods
Thus it can be said that Chi Square is the data analysis procedure which helps the researcher compare the data displayed as frequencies in the categories being observed. Males/females, natives/non-natives, monolinguals/bilinguals, etc, are examples of categories.
Frequency refers to the number of observations in the categories used to classify the data.
example from a piece of human geography research into urban development relating to the 2012 Olympics site.
The researcher was particularly interested in those who felt negatively about the Olympic developments (disagreed or strongly disagreed)