SlideShare una empresa de Scribd logo
1 de 16
 
The reliability of educational assessments Dylan Wiliam www.dylanwiliam.net Ofqual Annual Lecture, Coventry: 7 May 2009
The argument ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Dealing with uncertainty in society ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
The very first high-stakes assessment… ,[object Object],[object Object],[object Object]
Reliability ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Uncertainty in assessing English ,[object Object]
Uncertainty in assessing mathematics ,[object Object]
Measures of reliability ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Test length and reliability From To Just about the only way to increase the reliability of a test is to make it longer, or narrower (which amounts to the same thing). 0.70 0.75 0.80 0.85 0.90 0.95 0.70 1.0 0.75 1.3 1.0 0.80 1.7 1.3 1.0 0.85 2.4 1.9 1.4 1.0 0.90 3.9 3.0 2.3 1.6 1.0 0.95 8.1 6.3 4.8 3.4 2.1 1.0
Reliability is not what we really want ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Reliability v consistency ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Uncertainty in assessment at A-level ,[object Object],[object Object]
Here’s the table that got me into trouble… ,[object Object]
AERA, APA, NCME Standards (4 e/d,1999) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object]

Más contenido relacionado

Destacado

Dylan Wiliam's slides - 2013 11-21 bcssa
Dylan Wiliam's slides - 2013 11-21 bcssaDylan Wiliam's slides - 2013 11-21 bcssa
Dylan Wiliam's slides - 2013 11-21 bcssa
Hélène Galdin-O'Shea
 

Destacado (12)

A level reform and university entrance - A view from the Regulator
A level reform and university entrance - A view from the RegulatorA level reform and university entrance - A view from the Regulator
A level reform and university entrance - A view from the Regulator
 
Regulatory IT Systems (RITS) Roadshow Presentation
Regulatory IT Systems (RITS) Roadshow PresentationRegulatory IT Systems (RITS) Roadshow Presentation
Regulatory IT Systems (RITS) Roadshow Presentation
 
Fiona Pethick Speech To PAA\ VQ-SET National Conference
Fiona Pethick Speech To PAA\ VQ-SET National ConferenceFiona Pethick Speech To PAA\ VQ-SET National Conference
Fiona Pethick Speech To PAA\ VQ-SET National Conference
 
Getting to grips with RITS
Getting to grips with RITSGetting to grips with RITS
Getting to grips with RITS
 
What should we want from GCSEs & A levels?
What should we want from GCSEs & A levels?What should we want from GCSEs & A levels?
What should we want from GCSEs & A levels?
 
GCSE Science Score Seminar
GCSE Science Score SeminarGCSE Science Score Seminar
GCSE Science Score Seminar
 
Maintaining standards of qualifications for Summer 2010
Maintaining standards of qualifications for Summer 2010Maintaining standards of qualifications for Summer 2010
Maintaining standards of qualifications for Summer 2010
 
SSAT National Headteacher Steering Group
SSAT National Headteacher Steering GroupSSAT National Headteacher Steering Group
SSAT National Headteacher Steering Group
 
Achieving Quality
Achieving QualityAchieving Quality
Achieving Quality
 
Maintaining Standards
Maintaining StandardsMaintaining Standards
Maintaining Standards
 
Dylan Wiliam's slides - 2013 11-21 bcssa
Dylan Wiliam's slides - 2013 11-21 bcssaDylan Wiliam's slides - 2013 11-21 bcssa
Dylan Wiliam's slides - 2013 11-21 bcssa
 
the Independent Regulator
the Independent Regulatorthe Independent Regulator
the Independent Regulator
 

Similar a Dylan Wiliam's presentation at Ofqual's Chief Regulator's Report event

Quantitative MethodsChoosing a Sample.pptxChoosing a Samp.docx
Quantitative MethodsChoosing a Sample.pptxChoosing a Samp.docxQuantitative MethodsChoosing a Sample.pptxChoosing a Samp.docx
Quantitative MethodsChoosing a Sample.pptxChoosing a Samp.docx
amrit47
 
The Developmental Coordination Disorder Questionnaire
The Developmental Coordination Disorder QuestionnaireThe Developmental Coordination Disorder Questionnaire
The Developmental Coordination Disorder Questionnaire
Mandy Cross
 
Descriptive statistics research survey analysis (part 2)
Descriptive statistics research   survey analysis (part 2)Descriptive statistics research   survey analysis (part 2)
Descriptive statistics research survey analysis (part 2)
Ghivitha Kalimuthu
 
Descriptive statistics research survey analysis (part 2)
Descriptive statistics research   survey analysis (part 2)Descriptive statistics research   survey analysis (part 2)
Descriptive statistics research survey analysis (part 2)
Ghivitha Kalimuthu
 
Interventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scaleInterventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scale
Matt Williams
 
Interventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scaleInterventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scale
Matt Williams
 

Similar a Dylan Wiliam's presentation at Ofqual's Chief Regulator's Report event (20)

The Error of Our Ways
The Error of Our WaysThe Error of Our Ways
The Error of Our Ways
 
How to judge approximations? Pitfalls of statistics
How to judge approximations? Pitfalls of statisticsHow to judge approximations? Pitfalls of statistics
How to judge approximations? Pitfalls of statistics
 
A New Generation of Assessments: 3 Things You Need to Know
A New Generation of Assessments: 3 Things You Need to KnowA New Generation of Assessments: 3 Things You Need to Know
A New Generation of Assessments: 3 Things You Need to Know
 
Copie de PRESENTATION_ RELIABILITY _ VALIDITY.pptx
Copie de PRESENTATION_ RELIABILITY _ VALIDITY.pptxCopie de PRESENTATION_ RELIABILITY _ VALIDITY.pptx
Copie de PRESENTATION_ RELIABILITY _ VALIDITY.pptx
 
Quantitative MethodsChoosing a Sample.pptxChoosing a Samp.docx
Quantitative MethodsChoosing a Sample.pptxChoosing a Samp.docxQuantitative MethodsChoosing a Sample.pptxChoosing a Samp.docx
Quantitative MethodsChoosing a Sample.pptxChoosing a Samp.docx
 
EBP-2.ppt
EBP-2.pptEBP-2.ppt
EBP-2.ppt
 
The Reliability Programme: Leading the way to better tests and assessments
The Reliability Programme: Leading the way to better tests and assessmentsThe Reliability Programme: Leading the way to better tests and assessments
The Reliability Programme: Leading the way to better tests and assessments
 
1 1 data
1 1 data1 1 data
1 1 data
 
Testing in language programs (chapter 8)
Testing in language programs (chapter 8)Testing in language programs (chapter 8)
Testing in language programs (chapter 8)
 
Meaning and Importance of Statistics
Meaning and Importance of StatisticsMeaning and Importance of Statistics
Meaning and Importance of Statistics
 
Fulcher standardized testing
Fulcher standardized testingFulcher standardized testing
Fulcher standardized testing
 
lecture5.ppt
lecture5.pptlecture5.ppt
lecture5.ppt
 
The Developmental Coordination Disorder Questionnaire
The Developmental Coordination Disorder QuestionnaireThe Developmental Coordination Disorder Questionnaire
The Developmental Coordination Disorder Questionnaire
 
Validity & reliability
Validity & reliabilityValidity & reliability
Validity & reliability
 
Chapter 11
Chapter 11Chapter 11
Chapter 11
 
anchoring-heuristic Decision Making
anchoring-heuristic Decision Makinganchoring-heuristic Decision Making
anchoring-heuristic Decision Making
 
Descriptive statistics research survey analysis (part 2)
Descriptive statistics research   survey analysis (part 2)Descriptive statistics research   survey analysis (part 2)
Descriptive statistics research survey analysis (part 2)
 
Descriptive statistics research survey analysis (part 2)
Descriptive statistics research   survey analysis (part 2)Descriptive statistics research   survey analysis (part 2)
Descriptive statistics research survey analysis (part 2)
 
Interventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scaleInterventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scale
 
Interventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scaleInterventions for children’s dental anxiety: Validating a coping styles scale
Interventions for children’s dental anxiety: Validating a coping styles scale
 

Más de Ofqual Slideshare

Regulatory strategy, malpractice, event notifications and whistleblowing even...
Regulatory strategy, malpractice, event notifications and whistleblowing even...Regulatory strategy, malpractice, event notifications and whistleblowing even...
Regulatory strategy, malpractice, event notifications and whistleblowing even...
Ofqual Slideshare
 

Más de Ofqual Slideshare (20)

Ofqual Audit Approach: April 2015
Ofqual Audit Approach: April 2015Ofqual Audit Approach: April 2015
Ofqual Audit Approach: April 2015
 
The Ethics of Accountability in Education Assessment: Ofqual ethics symposium
The Ethics of Accountability in Education Assessment: Ofqual ethics symposiumThe Ethics of Accountability in Education Assessment: Ofqual ethics symposium
The Ethics of Accountability in Education Assessment: Ofqual ethics symposium
 
The Rational Teacher: Ofqual ethics symposium
The Rational Teacher: Ofqual ethics symposiumThe Rational Teacher: Ofqual ethics symposium
The Rational Teacher: Ofqual ethics symposium
 
Teacher Malpractice in Assessment: the International Context: Ofqual Ethics S...
Teacher Malpractice in Assessment: the International Context: Ofqual Ethics S...Teacher Malpractice in Assessment: the International Context: Ofqual Ethics S...
Teacher Malpractice in Assessment: the International Context: Ofqual Ethics S...
 
Teacher Ethics in Summative Assessment: Ofqual ethics symposium
Teacher Ethics in Summative Assessment: Ofqual ethics symposiumTeacher Ethics in Summative Assessment: Ofqual ethics symposium
Teacher Ethics in Summative Assessment: Ofqual ethics symposium
 
Next steps for the regulation of adult vocational qualifications: March 2015
Next steps for the regulation of adult vocational qualifications: March 2015Next steps for the regulation of adult vocational qualifications: March 2015
Next steps for the regulation of adult vocational qualifications: March 2015
 
Phil Beach's presentation to the Association of Colleges A level reform confe...
Phil Beach's presentation to the Association of Colleges A level reform confe...Phil Beach's presentation to the Association of Colleges A level reform confe...
Phil Beach's presentation to the Association of Colleges A level reform confe...
 
Assessment arrangements for reformed religious studies qualifications
Assessment arrangements for reformed religious studies qualificationsAssessment arrangements for reformed religious studies qualifications
Assessment arrangements for reformed religious studies qualifications
 
Preparing for validity audits
Preparing for validity auditsPreparing for validity audits
Preparing for validity audits
 
Westminster Forum: The future for Apprenticeships in England: Trailblazers, f...
Westminster Forum: The future for Apprenticeships in England: Trailblazers, f...Westminster Forum: The future for Apprenticeships in England: Trailblazers, f...
Westminster Forum: The future for Apprenticeships in England: Trailblazers, f...
 
Ofqual’s work on quality of marking
Ofqual’s work on quality of marking Ofqual’s work on quality of marking
Ofqual’s work on quality of marking
 
Securing Standards in Qualifications and Assessment (October 2014)
Securing Standards in Qualifications and Assessment (October 2014)Securing Standards in Qualifications and Assessment (October 2014)
Securing Standards in Qualifications and Assessment (October 2014)
 
Regulatory strategy, malpractice, event notifications and whistleblowing even...
Regulatory strategy, malpractice, event notifications and whistleblowing even...Regulatory strategy, malpractice, event notifications and whistleblowing even...
Regulatory strategy, malpractice, event notifications and whistleblowing even...
 
A level reform
A level reformA level reform
A level reform
 
New approaches to dealing with malpractice and maladministration - cskills pr...
New approaches to dealing with malpractice and maladministration - cskills pr...New approaches to dealing with malpractice and maladministration - cskills pr...
New approaches to dealing with malpractice and maladministration - cskills pr...
 
Malpractice - lessons learned - ncfe presentation
Malpractice - lessons learned - ncfe presentationMalpractice - lessons learned - ncfe presentation
Malpractice - lessons learned - ncfe presentation
 
Event notifications
Event notificationsEvent notifications
Event notifications
 
Malpractice and maladministration
Malpractice and maladministrationMalpractice and maladministration
Malpractice and maladministration
 
Whistle blowing
Whistle blowingWhistle blowing
Whistle blowing
 
Regulatory operations
Regulatory operationsRegulatory operations
Regulatory operations
 

Último

Spellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseSpellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please Practise
AnaAcapella
 
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
QucHHunhnh
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
ciinovamais
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
QucHHunhnh
 

Último (20)

Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
Spellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please PractiseSpellings Wk 3 English CAPS CARES Please Practise
Spellings Wk 3 English CAPS CARES Please Practise
 
On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
 
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual  Proper...General Principles of Intellectual Property: Concepts of Intellectual  Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
PROCESS RECORDING FORMAT.docx
PROCESS      RECORDING        FORMAT.docxPROCESS      RECORDING        FORMAT.docx
PROCESS RECORDING FORMAT.docx
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
 
Dyslexia AI Workshop for Slideshare.pptx
Dyslexia AI Workshop for Slideshare.pptxDyslexia AI Workshop for Slideshare.pptx
Dyslexia AI Workshop for Slideshare.pptx
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 
Unit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptxUnit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptx
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
Third Battle of Panipat detailed notes.pptx
Third Battle of Panipat detailed notes.pptxThird Battle of Panipat detailed notes.pptx
Third Battle of Panipat detailed notes.pptx
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
 

Dylan Wiliam's presentation at Ofqual's Chief Regulator's Report event

  • 1.  
  • 2. The reliability of educational assessments Dylan Wiliam www.dylanwiliam.net Ofqual Annual Lecture, Coventry: 7 May 2009
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10. Test length and reliability From To Just about the only way to increase the reliability of a test is to make it longer, or narrower (which amounts to the same thing). 0.70 0.75 0.80 0.85 0.90 0.95 0.70 1.0 0.75 1.3 1.0 0.80 1.7 1.3 1.0 0.85 2.4 1.9 1.4 1.0 0.90 3.9 3.0 2.3 1.6 1.0 0.95 8.1 6.3 4.8 3.4 2.1 1.0
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.