UNDP/OECD/UNESCO IIEP Presented by Simone Bloem, OECD Directorate for Education, Programme for Co-operation with Non-member economies at the Regional Conference on Fighting Corruption in Education in Eastern Europe and Central Asia, 24-25 November 2011, Bratislava, Slovakia

1. Education outcomes and
contextual data: the potential of
international surveys like PISA
Simone Bloem
OECD Directorate for Education
Programme for Co-operation with Non-member economies

2. Outline
• Overview of international surveys:
PISA TIMMS PIRLS
• Relevance of PISA for assessing integrity of
education systems
• School system characteristics that are positively
associated with performance, equity and integrity
• Next steps
2

3. International Surveys
• OECD’s Programme for
PISA International Student Assessment
• IAE’s Trends in International
TIMMS Mathematics and Science Study
• IAE’s Progress in International
PIRLS Reading Literacy Study
3

4. OECD: PISA
Triennial survey of key competencies of 15-
year-old students
Main subjects tested: reading (2000, 2009),
mathematics (2003) and science (2006)
Tool to compare the efficiency and equity of
countries’ education systems in an international
perspective

5. A map of PISA countries and economies
5

6. IAE: TIMMS
Assessment of mathematics and
science competencies of 4th- and 8th-
grade students
Survey conducted every four years:
1995, 1999, 2003, and 2007
Participation of more than 60
countries, diverse in terms of
geographical location and economic
development

7. IAE: PIRLS
Assessment of students’ reading literacy at the
end of primary school, in some countries
students are assessed in fifth and sixth grade
PIRLS is conducted every five years : 2001, 2006
and 2011
55 countries from all around the world have
participated in PIRLS 2011
PIRLS 2011 has been extended to include
prePIRLS —less difficult and designed to test
basic reading skills that are a prerequisite for
PIRLS

8. Reading Mathematics Science
Mean score Mean score Mean score
Mean S.E. Mean S.E. Mean S.E.
OECD
Estonia 501 (2.6) 512 (2.6) 528 (2.7)
Poland 500 (2.6) 495 (2.8) 508 (2.4)
Hungary 494 (3.2) 490 (3.5) 503 (3.1)
OECD average 493 (0.5) 496 (0.5) 501 (0.5)
Latvia 484 (3.0) 482 (3.1) 494 (3.1)
Slovenia 483 (1.0) 501 (1.2) 512 (1.1)
Czech Republic 478 (2.9) 493 (2.8) 500 (3.0)
Slovak Republic 477 (2.5) 497 (3.1) 490 (3.0)
Croatia 476 (2.9) 460 (3.1) 486 (2.8)
Lithuania 468 (2.4) 477 (2.6) 491 (2.9)
Russian Federation 459 (3.3) 468 (3.3) 478 (3.3)
Serbia 442 (2.4) 442 (2.9) 443 (2.4)
Bulgaria 429 (6.7) 428 (5.9) 439 (5.9)
Romania 424 (4.1) 427 (3.4) 428 (3.4)
Montenegro 408 (1.7) 403 (2.0) 401 (2.0)
Albania 385 (4.0) 377 (4.0) 391 (3.9)

9. Background data
Students background questionnaire
• socio-economic background, learning habits, attitudes towards reading, involvement and
motivation
School principals questionnaire
• demographic school characteristics, school policies, quality of learning environment
Parents questionnaire
• PISA for selected countries (optional), PIRLS
• home learning environment, parental support, school choice
Teachers Questionnaire
• For TIMMS and PIRLS only
• Professional background, teaching methods, training and professional development
Curriculum questionnaires for National Research Coordinators
• TIMMS and PIRLS only
PIRLS/TIMMS 2011 Encyclopedia
• provides a profile of each country’s education system

10. International surveys deliver…
International
indicators and
benchmarks
Analysis in the
Analysis in the
national and
international
regional
context
context
Evidence-
based,
effective
education
policies
10

11. Integrity and PISA
• Starting point to tackle
corruptive practices is to
evaluate the integrity of
an educational system
• We are assuming that
systems with high level
of integrity are in
Successful
general those systems school system
that perform well – well
in terms of student • In PISA, successful school
achievement and systems are defined as those PISA
providing equal that perform above the OECD
opportunities. average in reading and in
which students’ socio-economic • Allows identifying
background has a smaller school characteristics
impact on reading performance that are shared by
Integrity than it is the case in a typical successful school
OECD country. systems
• The term ―successful school • PISA is used as a first
system‖ refers o two distinct point of reference in
areas: achievement and order to assign
equity education policy areas
of integrity

12. School characteristics related to
performance and/or equity
Low levels of
differentiation
Grade repetition
Student
transference
Ability grouping

13. Low differentiation of the school system
Differentiation : Access
• Unequal access to schools and
classrooms creates demand
situations for scarce places which
may lead to corruption, in
particular when perceived quality
of schools differs greatly

14. School characteristics related to
performance and/or equity
Low levels of
Accountability
differentiation
Grade
repetition The existence of
standards-based
external
examinations
Student
transference
And how they are
used
Ability grouping
Monitoring of
lessons by
inspectors

15. Accountability
Standards-based external examinations
• Educational achievement is made observable and transparent
• Facilitate the monitoring of performance of students, teachers,
and schools
• Can be considered as best practice to avoid opportunistic
behaviour by teachers and principals
• Teachers cannot “get away” by skipping content areas or
employ very easy tests to have a positive teacher performance
evaluation
• Since teachers do not know exam questions and areas
covered in the test beforehand, incentives for corruption are
nonexistent, e.g. selling exam questions in advance
External final exams
• Improve the signal effect of school leave certificates and final
examinations marks which are of particularly importance for
higher educational institutions or future employers and may
reduce corruption, e.g. fraud at university admission or
favouritism, political affiliation in employment

16. School characteristics related to
performance and/or equity
Low levels of School
Accountability
differentiation autonomy
Grade repetition
The existence of Autonomy to
standards-based make decisions on
external curricula and
examinations assessments
Student
transference
And how they are used
Ability grouping
Monitoring of lessons
by inspectors

17. School autonomy
School autonomy
• School autonomy has a positive effect on student
achievement, but only when appropriate accountability
practices are in place
• Schools with more autonomy may have more
incentives to opportunistic behaviour, in particular when
there is a high degree of information asymmetry about
school behaviour, e.g. parents, governing boards or
central planning authorities have insufficient information
on what is happing in the school
• “Without external exams, the advantage of superior local
knowledge must weighed against the disadvantage of
opportunistic behaviour, and the net effect of school
autonomy depends on the relative size of these two partial
effects.” Wößmann et al. (2008: 34)

18. School characteristics related to
performance and/or equity
Low levels of School Spending on
Accountability
differentiation autonomy education
Grade
The existence of Teacher salaries
repetition Autonomy to
standards-based make decisions
external on curricula and
examinations assessments
Student Minimum resource
transference requirements
And how they are
used
Ability grouping
Monitoring of
lessons by
inspectors

19. Spending on education
High teacher salaries
• Low teacher salaries are considered as one reason of corruption
• Reflect the social status of the teaching profession in a society
• Suggests commitment to the rules and guidelines of teaching in a country
and satisfaction of profession
• Reduced incentives for corrupt and unethical behaviour, e.g. private tutoring
or absenteeism
Availability of sufficient human and material resources
• The eventuality of corruptive behaviour is greater in schools that lack
material, e.g. teachers might feel compelled to ask for money to provide
students with learning materials, such as textbooks and laboratory equipment
or principals may illegally rent out school facilities
• Great inequity between schools in terms of resources may lead to high
demand for scarce places in ―better‖ schools. This demand may lead to
corruption occurrence

20. School characteristics related to
performance and/or equity
Low levels of School Spending on Learning
Accountability
differentiation autonomy education environment
Grade Autonomy to Teacher Disciplinary
repetition The existence of salaries climate
standards- make
based external decisions on
examinations curricula and
assessments Minimum Student and
Student
resource teacher
transference
requirements behaviour
And how they are
used Teacher-
Ability
student
grouping
relations
Monitoring of
lessons by
inspectors

21. Learning environment
Quality of learning
environment
• High quality of the learning environment,
approximated by indicators such as good
teacher-student relations, positive student
and teacher behaviour and good
disciplinary climate, are important to
ensure a high level of moral and
commitment in schools and classrooms.
• This can be considered as crucial elements
of an intact school system that prevent
various corruptive or unethical practices.

22. Next steps
PISA and corruption items
• Integration of corruption-related items in the
national PISA students’, parents’ and/or school
questionnaire, e.g. questions related to private
tutoring, gifts to teachers, cheating
• Raise awareness of the problem of corruption in the
education sector on an international level
• Open up new opportunities to research corruption
empirically
• Make corruption in education empirically measurable and
allow to relate corruption to educational outcomes

23. Thank you for your
attention!