2. Validity and Reliability
Both are properties of a measure
Both address the question: ‘How good is this measure?’
First approximation:
Reliability = Dependability
Validity = Truthfulness or Accuracy
Two important aspects in order to approve and validate quantitative research
The importance of measuring the accuracy and consistency of research instruments
(especially the questionnaires) known as validity and reliability, respectively.
3. validity
Validity is the ability of an instrument to measure what it is
designed to measure.
Validity is the extent to which a test measures what it claims to
measure.
A valid test measures accurately what it is intended to measure.
It is vital for a test to be valid in order for the results to be
accurately applied and interpreted.
4. Definition of Validity
“Validity is the degree to which the researcher has
measured what he has set out to measure.”
(Smith, 1991)
5. Why Validity?
Validity is done mainly to answer the following questions:
Is the research investigation providing answers to the
research questions for which it was undertaken?
If so, it providing these answers using appropriate
methods and procedures?
6. Bases of Validity in Quantitative Research
Controllability
Replicability
Predictability
Generalizability
Fragmentation
Randomization of sample
Neutrality
Objectivity
Observability
Inference
Manipulation of variables
7. Factors Affecting Validity
o History
o Maturation
o Changing the instruments
o Statistical regression
o Differential selection
o Experimental mortality
o Failure to complete protocol
o Changes
8. Types of Validity
A. Face validity - it is the extent to which the measurement method appears
‘on its face’ to measure the construct of interest.
B. Content validity - it is the extent to which the measurement method
covers the entire range of relevant behaviors, thoughts and feelings that
define the construct being measured.
C. Criterion validity - it is the extent to which people’s scores are
correlated with other variables or criteria that reflect the same construct.
9. Cont.
D. Construct validity - it is commonly used in social research
when no clear criterion exists for validation purpose.
E. External validity – it helps to answer the question: can the
research be applied to the real world? It is the extent to
which the results of a research study can be generalized to
different situations, different groups of people, different
settings, different conditions, etc.
10. Cont.
D. Internal validity – it is a way to measure: was the research
done right? It is related to how many confounding variables
you have in your study.
E. Population validity - it refers to the extent to which the
findings can be generalized to other populations of people.
F. Ecological validity - it refers to the extent to which the
findings can be generalized mainly the present situation.
11. reliability
Reliability is the degree of in the measurements made by a research
instrument.
It is the extent to which an experiment, test, or any measuring
procedure shows the same result on repeated trials.
A questionnaire is said to be reliable if we get same/similar answers
repeatedly.
It can be measured by estimating correlation coefficient.
12. Definition of Reliability
“How consistent the results are when the experiment is
repeated a number of times under same methodological
conditions, then instrument is said to be reliable.”
(Joppe, 2000)
13. Bases of Reliability in Quantitative Research
1) Demonstrability
2) Stability & Replicability
3) Parallel forms
4) Objectivity
5) Verification of data &
analysis
6) Answering research
questions
7) Meaningfulness to the
research
8) Parsimony
9) Generalizability
10) Accuracy
11) Neutrality
14. Factors Affecting Reliability
i. The length of the scale
ii. The expression of the items in the
scale
iii. Insufficiency
iv. Misunderstanding
v. Homogeneity of the group
vi. The duration of the scale
vii.The objectivity in scoring
viii.The coordination in making
measurement
ix. The explanation of the scale
x. The characteristics of the items of
scale
xi. Difficulty of the scale
15. Types of Reliability
There are 4 general classes of reliability estimates,
each of which estimates reliability in a different
way. They are:
1. Inter-Rather Reliability
2. Test-Retest Reliability
3. Parallel-Forms Reliability
4. Internal Consistency Reliability
16. 1) Test - Retest Reliability
Test-retest reliability is a measure of
reliability obtained by administering
the same test twice over a period of
time to a group of individuals. The
scores from Time 1 and Time 2 can
then be correlated in order to evaluate
the test for stability over time.
17. 2) Inter - Rater Reliability
Inter-rater reliability is a
measure of reliability used to
assess the degree to which
different judges or raters
agree in their assessment
decisions.
18. 3) Parallel-Forms Reliability
In this reliability, two
equivalent tests are given to
students a short time part. If
the forms are parallel, then
the tests produce the same
observed results.
19. 4) Internal Consistency Reliability
It is a measure of reliability used to evaluate the degree
to which different test items that probe the same
construct produce similar results. There are a wide
variety of internal consistency measures that can be used.
20. Cont.
a) Average Inter-Item Correlation:- it is a subtype of internal
consistency reliability. It is obtained by taking all of the
items on a test that probe the same construct (e.g., reading
comprehension), determining the coefficient for each pair of
items, and finally taking the average of all of these
correlation coefficients.
21. Cont.
b) Split-Half Reliability:- it begun by “splitting in half” all items of a test
that are intended to probe the same area of knowledge in order to form
2 sets of items. The entire test is administered to a group of individuals,
the total score for each set is computed, and finally the split-half
reliability is obtained by determining the correlation between the two
total set scores.
Reliability = 2r/1+r
r = the actual correlation b/w the 2 halves of the instrument.
22. Cont.
d). Cronbach’s Alpha ;-
This’s equivalent to the
average of all possible spilt-
half estimates’ although
that’s not how we compute
it. Just keep in mind that this
is equivalent to the average
of all possible split-half
correlation we would never
actually calculate it that
way.
23. Relationship between validity &
reliability
Both are closely related.
A test cannot be considered valid unless the measurements resulting from it are reliable.
Likewise, results from a test can be reliable and not necessarily valid.
A reliable test may not be valid at all (e.g. writing test).
To make tests reliable, we must be wary of reducing their validity.
A good test has a balance of validity and reliability.
Reliability is necessary but not sufficient for validity.
A measure can be (highly) reliable, but not (highly) valid.
If a measure is valid, it must also be reliable.
