1. Seeing your education system in the mirror of other systemsExamples from the OECD Jakarta, May 11 2010 Andreas SchleicherEducation Policy Advisor of the OECD Secretary-General

2. Domain 1 Individual learner LevelA LevelB Instructional settings LevelC Schools, other institutions Country or system LevelD Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour Socio-economic background of learners Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate The learning environment at school Community and school characteristics Output and performance of institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies

3. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

4. A world of change – highereducation Expenditure per student at tertiary level (USD) Cost per student Graduate supply Tertiary-type A graduation rate

5. A world of change – highereducation Expenditure per student at tertiary level (USD) United States Cost per student Finland Japan Graduate supply Tertiary-type A graduation rate

6. A world of change – highereducation Expenditure per student at tertiary level (USD) Australia Finland United Kingdom Tertiary-type A graduation rate

7. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate

8. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate

9. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate

10. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate

11. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate

12. A world of change – highereducation What about international students? Expenditure per student at tertiary level (USD) United States Australia A A United Kingdom Finland A Tertiary-type A graduation rate

13. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

14. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

15. Components of the private net present value for a male with higher education 27K$ 56K$ 170K$ 105K$ 35K$ 26K$ 367K$ Net present value in USD equivalent

16. Public cost and benefits for a male obtaining post-secondary education Public costs Public benefits Net present value, USD equivalent (numbers in orange shownegative values) USD equivalent

17. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

18. Spending choices on secondary schoolsContribution of various factors to upper secondary teacher compensation costsper student as a percentage of GDP per capita (2004) Percentage points

20. Incentives for teachers to enhance classroom practice

22. Instructional leadership

23. Innovation and knowledge management

24. Staff development, quality assurance and quality improvement

25. Support for other schools or networks, “system leaders”

26. Relationships between teachers, heads, supervisors, inspectors and advisors etc.Infrastructure Teaching and learning Strategies Funding Definition of expected standards Competencies and content, relevance and links to “real world” Alignment of curricula with standards Quality of materials to support work of teachers Assessment approaches around the standards Assessment for learning Assessment of learning and monitoring Standards and intervention in inverse proportion to success Leadership and school ethos Devolution of first line responsibility for the quality of educational provision to the point of delivery Local leadership and teacher development Feedback systems which provide information about effectiveness and trends in outcomes Performance indicators, stakeholder surveys, professional evaluation Stakeholder involvement Human capital High quality learning for every student Teaching quality and professional development State and regional context Effective classroom practice Organisation of schooling Autonomy and accountability Curriculum and assessment

27. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

28. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

29. How the demand for skills has changedEconomy-wide measures of routine and non-routine task input (US) Mean task input as percentiles of the 1960 task distribution The dilemma of schools: The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource (Levy and Murnane)

30. OECD’s PISA assessment of the knowledge and skills of 15-year-olds Coverage of world economy 83% 77% 81% 85% 86% 87%

31. Example Mathematics The real world The mathematical World Making the problem amenable to mathematical treatment A mathematical model A model of reality Understanding, structuring and simplifying the situation Using relevant mathematical tools to solve the problem A real situation Validating the results Mathematical results Real results Interpreting the mathematical results

32. High science performance Average performanceof 15-year-olds in science – extrapolate and apply Poland 2000 … 18 countries perform below this line Low science performance

33. High science performance Average performanceof 15-year-olds in science – extrapolate and apply High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

34. High science performance Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

35. Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics 20

36. Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics Variation of performance within schools Variation of performance between schools OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.

39. Explain phenomena scientifically

41. Knowledge about scienceAttitudes -Interest in science -Support for scientific enquiry -Responsibility Students demonstrate ability to compare and differentiate among competing explanations by examining supporting evidence. They can formulate arguments by synthesising evidence from multiple sources. Students can point to an obvious feature in a simple table in support of a given statement. They are able to recognise if a set of given characteristics apply to the function of everyday artifacts.

42. Top and bottom performers in science These students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations… These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position… Large prop. of poor perf. Large proportion of top performers 20

43. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD

44. Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada)after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1) Odds ratioCollege entry School marks at age 15 PISA performance at age 15

45. Relationship between test performance and economic outcomesAnnual improved GDP from raising performance by 25 PISA points Percent addition to GDP

46. Increase average performance by 25 PISA points (Total 115 trillion $) bn$

47. Public and private schools % Score point difference Public schools perform better Private schools perform better

48. School autonomy, standards-based examinations and science performanceSchool autonomy in selecting teachers for hire PISA score in science

49. Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model School principal’s positive evaluation of quality of educational materials(gross only) Schools with more competing schools(gross only) Schools with greater autonomy (resources)(gross and net) School activities to promote science learning(gross and net) One additional hour of self-study or homework (gross and net) One additional hour of science learning at school (gross and net) School results posted publicly (gross and net) Academically selective schools (gross and net) but no system-wide effect Schools practicing ability grouping (gross and net) One additional hour of out-of-school lessons (gross and net) 20 Each additional 10% of public funding(gross only) School principal’s perception that lack of qualified teachers hinders instruction(gross only) Effect after accounting for the socio-economic background of students, schools and countries Measured effect OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a

50. Paradigm shifts

51. High policy value A real-time assessment environment that bridges the gap between formative and summative assessment . Quick wins Must haves Examine individual, institutional and systemic factors associated with performance Extending the range of competencies through which quality is assessed Monitor educational progress Measuring growth in learning Low feasibility High feasibility Establish the relative standing of students and schools Assuming that every new skill domain is orthogonal to all others Money pits Low-hanging fruits Low policy value

52. High ambitions and universal standards Rigor, focus and coherence Great systems attract great teachers and provide access to best practice and quality professional development

53. Challenge and support Strong support Poor performance Improvements idiosyncratic Strong performance Systemic improvement Lowchallenge Highchallenge Poor performance Stagnation Conflict Demoralisation Weak support

55. Principals who manage ‘a building’, who have little training and preparation and are accountable but not empowered

56. Attracting, recruiting and providing excellent training for prospective teachers from the top third of the graduate distribution

57. Attracting and recruiting teachers from the bottom third of the graduate distribution and offering training which does not relate to real classrooms

58. Incentives, rules and funding encourage a fair distribution of teaching talent

59. The best teachers are in the most advantaged communitiesHuman capital

61. Seniority and tenure matter more than performance; patchy professional development; wide variation in quality

62. Teachers and the system expect every child to succeed and intervene preventatively to ensure this

63. Wide achievement gaps, just beginning to narrow but systemic and professional barriers to transformation remain in placeHuman capital (cont…)

64. Relatively few teachers participate in the kinds of professional development which they find has the largest impact on their work Comparison of teachers participating in professional development activities and teachers reporting moderate or high level impact by types of activity Figure 3.15

65. High ambitions Devolved responsibility,the school as the centre of action Accountability and intervention in inverse proportion to success Access to best practice and quality professional development

66. Local responsibility and national prescription Towards system-wide sustainable reform National prescription Schools today The industrial model, detailed prescription of what schools do Schools tomorrow? Building capacity Finland today Every school an effective school Schools leading reform

67. Strong ambitions Devolvedresponsibility,the school as the centre of action Integrated educational opportunities From prescribed forms of teaching and assessment towards personalised learning Accountability Access to best practice and quality professional development

68. High science performance Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Early selection and institutional differentiation High degree of stratification Low degree of stratification Low average performance Large socio-economic disparities Low average performance High social equity Low science performance

69. www.oecd.org; www.pisa.oecd.org All national and international publications The complete micro-level database email: pisa@oecd.org Andreas.Schleicher@OECD.org … and remember: Without data, you are just another person with an opinion Thank you !