Different performance monitoring models are appropriate for different funders. JET's Godwin Khoza speaks at the Tshikululu Social Investments Serious Social Investing 2013 workshop.

1. BSSIP Monitoring and Evaluation
Framework
13 March 2013

2. Evaluation of the Bojanala Systemic Improvement Project
Profiles Baselines Follow up
Investment
Decisions
Progress Impact Sustainability X
Teacher RBAT &
Diagnostic tests
Teacher
Standardised tests
Learner
performance
Annual OPR
2

3. Results of the RBAT on Math Literacy
100%
90%
80%
70%
60%
Score (%)
RBAT: July 2010
50%
Diag 1: January 2011
40%
Diag 2: September 2011
30% Diag 3: October 2012
20%
10%
0%
ML1 ML2 ML3 ML4 ML5 ML6 ML7 ALL
Teacher Identifier
3

4. Uses of the tests
• Designing the teacher development interventions
• Adjusting teacher development interventions
• Personal target setting by teachers
• Increasing the demand for teachers among teachers, e.g. on
mentorships
• Management awareness and decisions about teacher
development dynamics
4

5. BSSIP FET TEACHER DEVELOPMENT DOSAGE: 2010/2011
Q1 Q2 Q3 Q4
YEAR
INTERVENTION DOSAGE M1 M2 M3 TOTAL M4 M5 M6 TOTAL M7 M8 M9 TOTAL M10 M11 M12 TOTAL
TOTAL
CONTENT DEVELOPMENT 12 12 12 36
SELF-DIRECTED LEARNING 0
ONSITE MENTORING AND
SUPPORT PER TEACHER
2 2 2 6
PROFESSIONAL LEARNING
CLUSTERS
0
TOTAL HOURS PER TERM
PER TEACHER
14 14 14 42
TOTAL HOURS PER YEAR
PER TEACHER
84
5

6. BSSIP FET TEACHER DEVELOPMENT DOSAGE: 2012
Q1 Q2 Q3 Q4
YEAR
INTERVENTION DOSAGE M1 M2 M3 TOTAL M4 M5 M6 TOTAL M7 M8 M9 TOTAL M10 M11 M12 TOTAL
TOTAL
CONTENT DEVELOPMENT 19 19 19 19 19 19 19 19 76
SELF-DIRECTED LEARNING 12 12 12 36 12 12 12 36 12 12 12 36 12 12 24 132
ONSITE MENTORING AND
0 6 6 12 6 6 12 6 6 12 6 6 0 12 48
SUPPORT PER TEACHER
PROFESSIONAL LEARNING
2 2 4 2 2 4 2 2 4 2 2 4 16
CLUSTERS
TOTAL HOURS PER TERM
70 88 88 74 320
PER TEACHER
6

7. Challenges in implementing the tests
• Negative perceptions about teacher testing
• Weak standardisation – because the course content covered
during the year varies. Only difficulty level and cognitive
demand levels were standardised
• Vacancies and attrition (27 – 23 teachers )
7

8. FP Numeracy
Participation rate:
2010: 36 teachers (81.8%)
2011: 24 teachers (43.2%)
Overall average:
2010: 52.9%
2011: 54.1%
8

9. FP Numeracy continued
9

10. FP Numeracy
Highest scores:
Basic operations and money
Struggle: Manipulating digits to get largest difference
Middle scores:
Counting, number concept and place value
Fractions, techniques, patterns and flow diagrams, maps and grids
Struggle: Writing ordinal number in words, number pattern involving digital time,
Completion of number pattern up to seventh term
Lowest scores:
Shapes and objects
Time, mass, volume and length, area and perimeter
Graphs and data interpretation
Struggle: Problems where hours needs to be converted to days
Definitions for mass, weight and capacity
Evaluating the information in a graph critically
Listing outcomes for a simple trial
Reflection of a object
Drawing a rectangle with a specific perimeter on a grid
10 using tiling to calculate area

11. Evaluation of the Bojanala Systemic Improvement Project
Need for cost-
Profiles Baselines Follow up benefit analysis
Investment
Decisions
Progress Impact Sustainability X
Teacher RBAT &
Diagnostic tests
Teacher
Standardised tests
Learner
performance
Annual OPR
11