Talk at Birkbeck

1. THE ROLE OF ‘OPPORTUNITY TO
LEARN’ IN THE GEOMETRY
CURRRICULUM
A multilevel comparison of six countries
Birkbeck, 6 June 2019
Christian Bokhove
University of Southampton
United Kingdom

2. Who am I ?
• Christian Bokhove
• From 1998-2012 teacher maths and computer science,
secondary school Netherlands
• PhD from Utrecht University
• Associate Professor at University of
Southampton
• Maths education
• Technology use
• Research methods
• Large-scale assessment

3. enGasia project
• England – Geometry - Asia
• British Academy
• 3 years International Partnership
• England, Japan, Hong Kong

4. enGasia project
1. Compare geometry education in England,
Japan and Hong Kong.
2. Two digital resources (electronic books) will be
designed. They are then implemented in
classrooms in those countries.
3. The methodology will include a more qualitative
approach based on lesson observations and a
quasi-experimental element.

5. Background
• International largescale assessments
like PISA and TIMSS for secondary mathematics
• English government looks to Asia
• Focus on ‘geometry’ or ‘space and shapes’ as at face
value there seem to be a lot of curriculum differences.
Even larger gap.
UK/ENG JAP HK KOR SGP USA
PISA 2012 475 (494) 558 (536) 567 (561) 573 (554) 580 (573) 463 (481)
TIMSS 2015 514 (518) 598 (586) 602 (594) 612 (606) 617 (621) 500 (518)
TIMSS 2011 498 (507) 586 (570) 597 (586) 612 (613) 609 (611) 485 (509)
TIMSS 2007 513 (513) 584 (570) 580 (572) 600 (597) 590 (593) 480 (508)

6. Curriculum changes
For example Key Stage 3
(11-13 yr olds)
“understand, from their experience of constructing them,
that triangles satisfying SSS, SAS, ASA and RHS are
unique, but SSA triangles are not” (DfEE, 1999, p. 38)
“use the standard conventions for labelling the sides and
angles of triangle ABC, and know and use the criteria for
congruence of triangles” (DfE, 2013, p.8)

7. Describe different parts project
1. Curriculum effects
2. Item analysis
3. Design of digital books
4. Effects of the books

8. Describe different parts project
1. Curriculum effects
2. Item analysis
3. Design of digital books
4. Effects of the books
Mainly about this

9. 1. Curriculum effects
Understand the role of curricular elements in mathematics
and science achievement, with a particular emphasis on
geometry education, at lower secondary level within and
across selected countries in the East and West.
1. Align different frameworks into one framework
2. Use largescale assessment data to explore its
application

10. Dynamic model
• Dynamic model of
educational
effectiveness, as
developed by
Creemers and
Kyriakides (2008)
The first implication for our theoretical lens is that
we will adopt a multilevel approach in our study.

11. Opportunity to Learn
• Carroll (1963)
• Schmidt and others:
greater OTL in
mathematics was
related to higher student
achievement in
mathematics.
• But many definitions
(e.g. see Scheerens et
al., 2017).
• Interaction with SES
We propose that we focus on variables regarding
‘opportunity to learn’ (OTL) in our study. In doing so
we should include controls for SES and proxies for
quality of instruction.

12. TIMSS curriculum framework
• Intended
• Implemented
• Attained
• Contrary to PISA
has a curriculum
focus.
We use TIMSS 2011 data in this study because of
its curriculum focus.

13. DOC framework
Dynamic model Opportunity to learn Curriculum - TIMSS
National OTL
School and
classroom OTL
Student OTL

14. DOC framework
Dynamic model Opportunity to learn Curriculum - TIMSS
National level Curriculum content coverage Intended curriculum
Classroom
(teacher)
and school
• Instructional hours in the
classroom
• Curriculum content coverage
• Curriculum content
preparation
• Degree and experience
teacher
Implemented curriculum
Student Time spent on mathematics
Socio-Economic Status
Attained curriculum

15. Using the model: research questions
I. How much of the variance in student achievement is
explained by student- and classroom-level OTL
curriculum factors within and across the six countries?
II. How much are these OTL curriculum factors related to
geometry achievement at grade eight in England,
Japan, Hong Kong SAR, Korea, Singapore and the
USA?

16. Secondary data analysis
• TIMSS 2011
• Complex sampling design (Rutkowski et al, 2010)
• Weights
• Plausible Values
• Multilevel models
• Multilevel models in HLM 6.08.

17. Variables
Dependent variables:
• 5 Plausible Values for Geometry achievement
Student level variables:
• Home Economic Resources as proxy for SES
• Weekly time for homework
Classroom (teacher) level variables:
• Classroom SES and Homework time
• OTL measures: percentage geometry content coverage and
mathematics instructional hours per week
• Teacher: edulevel, years experience, teachers prepared to teach
geometry

18. Results

19. Some interesting things to note
• SES England, Japan, USA comparable. Singapore, Hong
Kong slightly lower. Korea much higher.
• English, Japanese, Korean students less time homework.
• Japan lowest number of mathematics instruction hours,
USA highest.
• Geometry taught highest in Japan and Korea, lowest
England and Singapore.
• English teachers feel most prepared to teach geometry,
Japanese teachers least.
And more…

20. Multilevel tables (highlighted)

21. Some interesting things to note
• Japan and Korea very little variance at classroom level:
homogeneous.
• At student level SES positive predictor in Japan, Korea,
England, USA. But not Singapore and Hong Kong, likely
more homogeneity.
• OTL predictors mixed picture. For example:
• Homework differential effect student and class level
• Geometry content coverage not predictor
• Most teacher quality variables not predictor
• Instructional hours in Japan

22. Conclusions 1
I. How much of the variance in student achievement is
explained by student- and classroom-level OTL
curriculum factors within and across the six countries?
Differs between countries, might reveal
heterogeneity.

23. Conclusions 2
II. How much are these OTL curriculum factors related to
geometry achievement at grade eight in England,
Japan, Hong Kong SAR, Korea, Singapore and the
USA?
Geometry content coverage and teacher
preparedness no predictors. Teacher variables only
here and there. Instructional hours important for
Japan. Weekly time spent on homework differential
effects but not always intuitive.
In short: it’s complex

24. Describe different parts project
1. Curriculum effects
2. Item analysis
3. Design of digital books
4. Effects of the books

27. Which of these is the reason
that triangle PQR is a right
angle triangle?
A. 32 + 42 = 52
B. 5 < 3 + 4
C. 3 + 4 = 12 – 5
D. 3 > 5 – 4

28. Describe different parts project
1. Curriculum effects
2. Item analysis
3. Design of digital books
4. Effects of the books

29. 3. Design of digital books
• Based on design principles
• ‘Widgets’ (needs to cater for diversity of features)
• Interaction (includes feedback, like Intelligent Tutoring Systems)
• Interoperability (different parts of the book should work together eg
graph element and formula element)
• Storing results (teachers should be able to see how students used
the books)
• Standards (needs to work on any device)
• Authorability (should be able to modify content)

30. This time
we took a
real
textbook

31. And we made
two sections
with geometry
into digital
textbooks

32. Demo

33. Digitalbooks

34. With feedback and answer checking
34

35. 35

36. https://youtu.be/0KxOqT7iB38

37. Parts of the project
1. Curriculum effects
2. Item analysis
3. Design of digital books
4. Effects of the books Still looking at this
One challenge here was that the local
conditions in the three countries couldn’t
really be the same and this caused
methodological issues. It’s a set of mixed
data from different parts of the books. For
example in England we focused Mental
Rotation Skills.

38. Discussion
• Interplay SES and OTL
(curriculum time)
…what can we address?
• Definitions of OTL
• Academic Learning Time?
• Quality of instruction?)
• Role shadow education
• Not the final word: these
analyses need to be
complemented with more
detailed, qualitative data
about the curriculum. For
example low scores Japan
on the task to the right; they
only did Pythagoras one year
later.
Which of these is the reason
that triangle PQR is a right
angle triangle?
A. 32 + 42 = 52
B. 5 < 3 + 4
C. 3 + 4 = 12 – 5
D. 3 > 5 – 4

39. Other projects
https://bokhove.net/aera-2019/
https://www.slideshare.net/cbokhove/mental-rotation-skills-90673428

40. Thank you
• C.Bokhove@soton.ac.uk
• University of Southampton
• Twitter: @cbokhove
• Website: www.bokhove.net
• British Academy
IPM-2014 PM130271 project