SlideShare una empresa de Scribd logo
1 de 17
Descargar para leer sin conexión
Ottawa Conference/CCME
C. Touchie, A. Gotzmann, C. Streefkerk,
A. De Champlain,
Medical Council of Canada
Putting the Pieces Together
to Create a Defensible
Blueprint:
A Subject Matter Expert Based
Process
April 2014
Ottawa, Canada
1
Credentialing Examinations
• Validity:
– Associated with the interpretation assigned to
test scores NOT with the score or with the test
itself
– Kane’s validity argument:
• Scoring
• Generalization
• Extrapolation  the content of the test is
matched appropriately to the
demands of practice
• Decision
2
Practice Analysis
• Practice Analysis Methods:
– Critical Incident Technique
– Task Inventory
– Professional Practice Model
• Professions: dynamic and social
2
Purpose of Blueprinting
• … is to assure the public that physicians
licensed to practice medicine have the
required knowledge, skills and attitudes
for safe and effective patient care.
• Only those who meet this standard are
qualified to enter professional practice
4
Decision points
• What we are assessing:
– Core foundational competencies expected of all
physicians regardless of discipline for the
purpose of fulfilling one of the requirements of
licensure
• Decision point 1
– Assessments leading to a decision for entry into
residency
• Decision point 2
– Assessments leading to a decision for entry into
independent practice
5
Practice Analysis
• Factors to Consider
1. Information required
2. Best sources of information
3. Strategy for obtaining that information
4. Strategy to put it all together
6Modified from M. Raymond, AERA 2014
1. Information Required
7
Focus of practice
descriptors
Level of
specificity
Attributes to be
measured
Current Issues Future of medical
education
Broad and
complex
Assessment needs
Supervising
PGY-1
Transition between
MD and residency
Entrustable
professional
activities
Supervisory needs
Incidence and
Prevalence
Common clinical
problems
Specific
presentations
(age, setting)
Frequency
National Survey MCC objectives Specific KSAs Importance of KSAs at
2 decision points
2. Sources of Information
8
Sources of Information
Current Issues Review of literature, medical educators
Supervising
PGY-1
New PGY-1s, supervisors
Incidence and
Prevalence
Billing data (CIHI, provincial sources), published
literature
National Survey Physicians: recent LMCC recipients, clinical
teachers
Informed public
3. Data Collection Strategy
9
Data Collection Strategy
Current Issues Review of literature, iterative review by experts
Supervising
PGY-1
Consensus technique/Survey
Incidence and
Prevalence
Review of existing data
National Survey Physicians: Survey
Informed public: Survey and focus groups
4. Strategy to pull it all together
Blueprint and Test Specifications
Defined 2 candidates Common Blueprint
Test Specifications for
each decision point
Subject Matter Experts
12 Experts 3 day meeting
4 Reports
Current Issues
PGY-1
supervision
Incidence and
Prevalence
National
Survey
Who were the SMEs?
Blueprint
MRA Rep of
Council
Central
Examination
Committee
Objectives
Committee
Test
Committees
RCPSCCFPC
UGME
Deans
PGME
Deans
University
Rep of
Council
Results:
Proposed Common Blueprint
Dimensions of Care
Health
Promotion and
Illness
Prevention
Acute Chronic Psychosocial
Aspects
PhysicianActivities
Assessment/
Diagnosis
Management
Communication
Professional
Behaviors
12
Dimensions of Care
Health
Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 30±5
Management 20±5
Communication 30±5
Professional
Behaviors 20±5
Column Percent 20±5 30±5 30±5 20±5 100
Assessments leading up to
Decision 1: Entry into Residency
13
Dimensions of Care
Health
Promotion
and Illness
Prevention
Acute Chronic Psychosocial
Aspects
Row
Percent
PhysicianActivities
Assessment/
Diagnosis 25±5
Management 35±5
Communication 20±5
Professional
Behaviors 20±5
Column Percent 20±5 25±5 35±5 20±5 100
Assessments leading up to
Decision 2: Entry into Independent Practice
14
Topics / Knowing
Processes / Doing
Processes/Doing
Textbook Topics
Academic Disciplines
Organ System
Clinical Problem / Reason for Visit
Medical Specialty
Cognitive Processes or Skills
Competency (ACGME, CanMEDS)
Physician Task or Skill
Conclusions
Defensible Blueprint:
• Define assessment purpose
• Choose information sources to inform
appropriately
• Include judgments from stakeholders
Lessons learned:
• Careful planning
– Team
– Information gathered
– Process
16
Thank You!
Questions – Comments
ctouchie@mcc.ca
17

Más contenido relacionado

La actualidad más candente

Development and Evaluation of clinical practice guideline (CPG) in psychiatry
Development and Evaluation of clinical practice guideline (CPG) in psychiatryDevelopment and Evaluation of clinical practice guideline (CPG) in psychiatry
Development and Evaluation of clinical practice guideline (CPG) in psychiatryDiptadhi Mukherjee
 
Evidence based practice power
Evidence based practice powerEvidence based practice power
Evidence based practice powerMahmoud Shaqria
 
Point of-Care Resources & Tools SC
Point of-Care Resources & Tools SCPoint of-Care Resources & Tools SC
Point of-Care Resources & Tools SCImad Hassan
 
ICTS Case Studies
ICTS Case StudiesICTS Case Studies
ICTS Case StudiesGregg Sweet
 
Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1Imad Hassan
 
Clinical Practice Guidelines
Clinical Practice GuidelinesClinical Practice Guidelines
Clinical Practice Guidelinesssompur
 
EBP Made Easy
EBP Made EasyEBP Made Easy
EBP Made EasyCPDspot
 
Outcomes research
Outcomes researchOutcomes research
Outcomes researchTony Ngo
 
EuroBioForum 2013 - Day 1 | Katherine Payne
 EuroBioForum 2013 - Day 1 | Katherine Payne EuroBioForum 2013 - Day 1 | Katherine Payne
EuroBioForum 2013 - Day 1 | Katherine PayneEuroBioForum
 
EBM Therapy Appraisal Template F1
EBM Therapy Appraisal Template F1EBM Therapy Appraisal Template F1
EBM Therapy Appraisal Template F1Imad Hassan
 
Introduction to outcomes research
Introduction to outcomes researchIntroduction to outcomes research
Introduction to outcomes researchTienie Stander
 
September 2015 - How do hospitalists perceive effectiveness of QI strategies
September 2015 - How do hospitalists perceive effectiveness of QI strategiesSeptember 2015 - How do hospitalists perceive effectiveness of QI strategies
September 2015 - How do hospitalists perceive effectiveness of QI strategiesVandad Yousefi MD, CCFP, FHM
 
Enrolment of trial patients challenges & strategies
Enrolment of trial patients challenges & strategiesEnrolment of trial patients challenges & strategies
Enrolment of trial patients challenges & strategiesBhaswat Chakraborty
 
Week 12 ppt research
Week 12 ppt researchWeek 12 ppt research
Week 12 ppt researchShalee Belnap
 
Surgical Audit
Surgical AuditSurgical Audit
Surgical Auditakinbodeog
 
FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...
FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...
FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...MedicReS
 
SafeCare presentation 2013.06.07
SafeCare presentation 2013.06.07SafeCare presentation 2013.06.07
SafeCare presentation 2013.06.07Mike Johnson
 

La actualidad más candente (20)

Development and Evaluation of clinical practice guideline (CPG) in psychiatry
Development and Evaluation of clinical practice guideline (CPG) in psychiatryDevelopment and Evaluation of clinical practice guideline (CPG) in psychiatry
Development and Evaluation of clinical practice guideline (CPG) in psychiatry
 
Outcomes research
Outcomes researchOutcomes research
Outcomes research
 
Evidence based practice power
Evidence based practice powerEvidence based practice power
Evidence based practice power
 
Point of-Care Resources & Tools SC
Point of-Care Resources & Tools SCPoint of-Care Resources & Tools SC
Point of-Care Resources & Tools SC
 
ICTS Case Studies
ICTS Case StudiesICTS Case Studies
ICTS Case Studies
 
Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1Knowledge Translation: Practical Strategies for Success v1
Knowledge Translation: Practical Strategies for Success v1
 
Clinical Practice Guidelines
Clinical Practice GuidelinesClinical Practice Guidelines
Clinical Practice Guidelines
 
EBP Made Easy
EBP Made EasyEBP Made Easy
EBP Made Easy
 
Outcomes research
Outcomes researchOutcomes research
Outcomes research
 
Key Performance Indices ( KPI ) Ambulances @ ni ems 2020
Key Performance Indices ( KPI )  Ambulances  @ ni ems 2020Key Performance Indices ( KPI )  Ambulances  @ ni ems 2020
Key Performance Indices ( KPI ) Ambulances @ ni ems 2020
 
EuroBioForum 2013 - Day 1 | Katherine Payne
 EuroBioForum 2013 - Day 1 | Katherine Payne EuroBioForum 2013 - Day 1 | Katherine Payne
EuroBioForum 2013 - Day 1 | Katherine Payne
 
EBM Therapy Appraisal Template F1
EBM Therapy Appraisal Template F1EBM Therapy Appraisal Template F1
EBM Therapy Appraisal Template F1
 
Introduction to outcomes research
Introduction to outcomes researchIntroduction to outcomes research
Introduction to outcomes research
 
September 2015 - How do hospitalists perceive effectiveness of QI strategies
September 2015 - How do hospitalists perceive effectiveness of QI strategiesSeptember 2015 - How do hospitalists perceive effectiveness of QI strategies
September 2015 - How do hospitalists perceive effectiveness of QI strategies
 
Enrolment of trial patients challenges & strategies
Enrolment of trial patients challenges & strategiesEnrolment of trial patients challenges & strategies
Enrolment of trial patients challenges & strategies
 
Week 12 ppt research
Week 12 ppt researchWeek 12 ppt research
Week 12 ppt research
 
Surgical Audit
Surgical AuditSurgical Audit
Surgical Audit
 
Utilizing ebp for quality improvement
Utilizing ebp for quality improvementUtilizing ebp for quality improvement
Utilizing ebp for quality improvement
 
FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...
FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...
FDA 2013 Clinical Investigator Training Course: The Physician as Clinician an...
 
SafeCare presentation 2013.06.07
SafeCare presentation 2013.06.07SafeCare presentation 2013.06.07
SafeCare presentation 2013.06.07
 

Destacado

“Primary health care: back to Alma-Ata in early 21st century"
“Primary health care: back to Alma-Ata in early 21st century"“Primary health care: back to Alma-Ata in early 21st century"
“Primary health care: back to Alma-Ata in early 21st century"Jean Jacques Bernatas
 
Primary Health Care, Objectives, Principles and Policy Directions
Primary Health Care, Objectives, Principles and Policy DirectionsPrimary Health Care, Objectives, Principles and Policy Directions
Primary Health Care, Objectives, Principles and Policy DirectionsHealth and Labour
 
Apostila Ilustração Biológica em Grafite
Apostila Ilustração Biológica em GrafiteApostila Ilustração Biológica em Grafite
Apostila Ilustração Biológica em GrafiteRogério Lupo
 
Health for all- primary health care- millennium development goals
Health for all- primary health care- millennium development  goalsHealth for all- primary health care- millennium development  goals
Health for all- primary health care- millennium development goalsAhmed-Refat Refat
 
Primary Health Care
Primary Health CarePrimary Health Care
Primary Health CareDoc Lorie B
 

Destacado (7)

Declaracion de alma
Declaracion de almaDeclaracion de alma
Declaracion de alma
 
“Primary health care: back to Alma-Ata in early 21st century"
“Primary health care: back to Alma-Ata in early 21st century"“Primary health care: back to Alma-Ata in early 21st century"
“Primary health care: back to Alma-Ata in early 21st century"
 
Primary Health Care, Objectives, Principles and Policy Directions
Primary Health Care, Objectives, Principles and Policy DirectionsPrimary Health Care, Objectives, Principles and Policy Directions
Primary Health Care, Objectives, Principles and Policy Directions
 
Apostila Ilustração Biológica em Grafite
Apostila Ilustração Biológica em GrafiteApostila Ilustração Biológica em Grafite
Apostila Ilustração Biológica em Grafite
 
Health for all- primary health care- millennium development goals
Health for all- primary health care- millennium development  goalsHealth for all- primary health care- millennium development  goals
Health for all- primary health care- millennium development goals
 
Primary Health Care
Primary Health CarePrimary Health Care
Primary Health Care
 
France Power Point
France Power PointFrance Power Point
France Power Point
 

Similar a Blueprinting update - 2014 Ottawa Conference

Evidence based practice. the process of decision making
Evidence based practice. the process of decision makingEvidence based practice. the process of decision making
Evidence based practice. the process of decision makingAmbreen Sadaf
 
Medical audit process
Medical audit processMedical audit process
Medical audit processsopi_1234
 
Clinical research.pdf
Clinical research.pdfClinical research.pdf
Clinical research.pdfSmriti Arora
 
NAC PRA update - 2014 Ottawa Conference
NAC PRA update - 2014 Ottawa ConferenceNAC PRA update - 2014 Ottawa Conference
NAC PRA update - 2014 Ottawa ConferenceMedCouncilCan
 
evidence based periodontics
 evidence based periodontics    evidence based periodontics
evidence based periodontics neeti shinde
 
Clinical Audit Overview
Clinical Audit OverviewClinical Audit Overview
Clinical Audit OverviewFastbleep
 
Using the Patient Activation Measure to improve quality of care for patients ...
Using the Patient Activation Measure to improve quality of care for patients ...Using the Patient Activation Measure to improve quality of care for patients ...
Using the Patient Activation Measure to improve quality of care for patients ...Ben Harris-Roxas
 
Evidence based orthodontics parag
Evidence based orthodontics paragEvidence based orthodontics parag
Evidence based orthodontics paragParag Deshmukh
 
CLINICAL AUDIT
CLINICAL AUDITCLINICAL AUDIT
CLINICAL AUDITS A Tabish
 
Evidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGY
Evidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGYEvidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGY
Evidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGYYasser Sami Abdel Dayem Amer
 
Evidence Based Practice - EBP
Evidence Based Practice - EBPEvidence Based Practice - EBP
Evidence Based Practice - EBPSaira Ambreen
 
Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...
Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...
Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...CHC Connecticut
 
Gbpca ncqa pcmh overview 02 25 13 final
Gbpca ncqa pcmh overview 02 25 13 finalGbpca ncqa pcmh overview 02 25 13 final
Gbpca ncqa pcmh overview 02 25 13 finalgbpca
 
Dr Ayman Ewies - Clinical audit made easy
Dr Ayman Ewies - Clinical audit made easyDr Ayman Ewies - Clinical audit made easy
Dr Ayman Ewies - Clinical audit made easyAymanEwies
 
Nursing process, deepani nanayakkara, nursing tutor, srilanka
Nursing process,  deepani nanayakkara, nursing tutor, srilanka Nursing process,  deepani nanayakkara, nursing tutor, srilanka
Nursing process, deepani nanayakkara, nursing tutor, srilanka Deepani Nanayakkara
 
Nursing process a simple guide for student nurses
Nursing process a simple guide for student nursesNursing process a simple guide for student nurses
Nursing process a simple guide for student nursesDeepani Nanayakkara
 

Similar a Blueprinting update - 2014 Ottawa Conference (20)

Evidence based practice. the process of decision making
Evidence based practice. the process of decision makingEvidence based practice. the process of decision making
Evidence based practice. the process of decision making
 
Medical audit process
Medical audit processMedical audit process
Medical audit process
 
Clinical research.pdf
Clinical research.pdfClinical research.pdf
Clinical research.pdf
 
NAC PRA update - 2014 Ottawa Conference
NAC PRA update - 2014 Ottawa ConferenceNAC PRA update - 2014 Ottawa Conference
NAC PRA update - 2014 Ottawa Conference
 
evidence based periodontics
 evidence based periodontics    evidence based periodontics
evidence based periodontics
 
Clinical pathway
Clinical pathwayClinical pathway
Clinical pathway
 
Clinical Audit Overview
Clinical Audit OverviewClinical Audit Overview
Clinical Audit Overview
 
Using the Patient Activation Measure to improve quality of care for patients ...
Using the Patient Activation Measure to improve quality of care for patients ...Using the Patient Activation Measure to improve quality of care for patients ...
Using the Patient Activation Measure to improve quality of care for patients ...
 
Evidence based orthodontics parag
Evidence based orthodontics paragEvidence based orthodontics parag
Evidence based orthodontics parag
 
CLINICAL AUDIT
CLINICAL AUDITCLINICAL AUDIT
CLINICAL AUDIT
 
Evidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGY
Evidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGYEvidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGY
Evidence-Based Clinical Practice Guidelines for OBSTETRICS AND GYNECOLOGY
 
Evidence Based Practice - EBP
Evidence Based Practice - EBPEvidence Based Practice - EBP
Evidence Based Practice - EBP
 
PCORI at Academy Health
PCORI at Academy HealthPCORI at Academy Health
PCORI at Academy Health
 
Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...
Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...
Advancing Team-Based Care: A Team Approach to Prevention and Chronic Illness ...
 
PCORI Methodology Committee Report
PCORI Methodology Committee ReportPCORI Methodology Committee Report
PCORI Methodology Committee Report
 
Gbpca ncqa pcmh overview 02 25 13 final
Gbpca ncqa pcmh overview 02 25 13 finalGbpca ncqa pcmh overview 02 25 13 final
Gbpca ncqa pcmh overview 02 25 13 final
 
Module 5 - Concise Analysis Method
Module 5 - Concise Analysis MethodModule 5 - Concise Analysis Method
Module 5 - Concise Analysis Method
 
Dr Ayman Ewies - Clinical audit made easy
Dr Ayman Ewies - Clinical audit made easyDr Ayman Ewies - Clinical audit made easy
Dr Ayman Ewies - Clinical audit made easy
 
Nursing process, deepani nanayakkara, nursing tutor, srilanka
Nursing process,  deepani nanayakkara, nursing tutor, srilanka Nursing process,  deepani nanayakkara, nursing tutor, srilanka
Nursing process, deepani nanayakkara, nursing tutor, srilanka
 
Nursing process a simple guide for student nurses
Nursing process a simple guide for student nursesNursing process a simple guide for student nurses
Nursing process a simple guide for student nurses
 

Más de MedCouncilCan

Peering through the Looking Glass: Towards a Programmatic View of the Qualify...
Peering through the Looking Glass: Towards a Programmatic View of the Qualify...Peering through the Looking Glass: Towards a Programmatic View of the Qualify...
Peering through the Looking Glass: Towards a Programmatic View of the Qualify...MedCouncilCan
 
A journey towards programmatic assessment
A journey towards programmatic assessmentA journey towards programmatic assessment
A journey towards programmatic assessmentMedCouncilCan
 
Pushing the Boundaries of Medical Licensing
Pushing the Boundaries of Medical Licensing Pushing the Boundaries of Medical Licensing
Pushing the Boundaries of Medical Licensing MedCouncilCan
 
Peeking behind the test: insights and innovations from the Medical Council of...
Peeking behind the test: insights and innovations from the Medical Council of...Peeking behind the test: insights and innovations from the Medical Council of...
Peeking behind the test: insights and innovations from the Medical Council of...MedCouncilCan
 
Séance d'orientation pour les candidats de l'examen de la CNE 2015
Séance d'orientation pour les candidats de l'examen de la CNE 2015Séance d'orientation pour les candidats de l'examen de la CNE 2015
Séance d'orientation pour les candidats de l'examen de la CNE 2015MedCouncilCan
 
2015 NAC candidate orientation presentation
2015 NAC candidate orientation presentation2015 NAC candidate orientation presentation
2015 NAC candidate orientation presentationMedCouncilCan
 
Surveying the landscape: An overview of tools for direct observation and asse...
Surveying the landscape: An overview of tools for direct observation and asse...Surveying the landscape: An overview of tools for direct observation and asse...
Surveying the landscape: An overview of tools for direct observation and asse...MedCouncilCan
 
Moving Towards Programmatic Assessment
Moving Towards Programmatic AssessmentMoving Towards Programmatic Assessment
Moving Towards Programmatic AssessmentMedCouncilCan
 
The New Blueprint: challenging the comfort zone
The New Blueprint: challenging the comfort zoneThe New Blueprint: challenging the comfort zone
The New Blueprint: challenging the comfort zoneMedCouncilCan
 
Séance d'orientation pour les candidats de l'examen de la CNE de 2014
Séance d'orientation pour les candidats de l'examen de la CNE de 2014Séance d'orientation pour les candidats de l'examen de la CNE de 2014
Séance d'orientation pour les candidats de l'examen de la CNE de 2014MedCouncilCan
 
2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentationMedCouncilCan
 
Orientation à l’intention des candidats au format ECOS (2014)
Orientation à l’intention des candidats au format ECOS (2014)Orientation à l’intention des candidats au format ECOS (2014)
Orientation à l’intention des candidats au format ECOS (2014)MedCouncilCan
 
2014 Candidate Orientation Presentation - Certification Examination in Family...
2014 Candidate Orientation Presentation - Certification Examination in Family...2014 Candidate Orientation Presentation - Certification Examination in Family...
2014 Candidate Orientation Presentation - Certification Examination in Family...MedCouncilCan
 
Présentation d'orientation de 2014 - EACMC, partie II
Présentation d'orientation de 2014 - EACMC, partie IIPrésentation d'orientation de 2014 - EACMC, partie II
Présentation d'orientation de 2014 - EACMC, partie IIMedCouncilCan
 
Présentation d'orientation - Examen de la CNE 2014
Présentation d'orientation - Examen de la CNE 2014Présentation d'orientation - Examen de la CNE 2014
Présentation d'orientation - Examen de la CNE 2014MedCouncilCan
 
2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation 2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation MedCouncilCan
 
Pre-Exam Orientation for Candidates taking the Certification Examination in F...
Pre-Exam Orientation for Candidates taking the Certification Examination in F...Pre-Exam Orientation for Candidates taking the Certification Examination in F...
Pre-Exam Orientation for Candidates taking the Certification Examination in F...MedCouncilCan
 
Pre-Exam Candidate Orientation
Pre-Exam Candidate OrientationPre-Exam Candidate Orientation
Pre-Exam Candidate OrientationMedCouncilCan
 
Practice Ready Assessment for IMG Physicians
Practice Ready Assessment for IMG PhysiciansPractice Ready Assessment for IMG Physicians
Practice Ready Assessment for IMG PhysiciansMedCouncilCan
 

Más de MedCouncilCan (20)

Peering through the Looking Glass: Towards a Programmatic View of the Qualify...
Peering through the Looking Glass: Towards a Programmatic View of the Qualify...Peering through the Looking Glass: Towards a Programmatic View of the Qualify...
Peering through the Looking Glass: Towards a Programmatic View of the Qualify...
 
A journey towards programmatic assessment
A journey towards programmatic assessmentA journey towards programmatic assessment
A journey towards programmatic assessment
 
Pushing the Boundaries of Medical Licensing
Pushing the Boundaries of Medical Licensing Pushing the Boundaries of Medical Licensing
Pushing the Boundaries of Medical Licensing
 
Peeking behind the test: insights and innovations from the Medical Council of...
Peeking behind the test: insights and innovations from the Medical Council of...Peeking behind the test: insights and innovations from the Medical Council of...
Peeking behind the test: insights and innovations from the Medical Council of...
 
Séance d'orientation pour les candidats de l'examen de la CNE 2015
Séance d'orientation pour les candidats de l'examen de la CNE 2015Séance d'orientation pour les candidats de l'examen de la CNE 2015
Séance d'orientation pour les candidats de l'examen de la CNE 2015
 
2015 NAC candidate orientation presentation
2015 NAC candidate orientation presentation2015 NAC candidate orientation presentation
2015 NAC candidate orientation presentation
 
Surveying the landscape: An overview of tools for direct observation and asse...
Surveying the landscape: An overview of tools for direct observation and asse...Surveying the landscape: An overview of tools for direct observation and asse...
Surveying the landscape: An overview of tools for direct observation and asse...
 
Moving Towards Programmatic Assessment
Moving Towards Programmatic AssessmentMoving Towards Programmatic Assessment
Moving Towards Programmatic Assessment
 
The New Blueprint: challenging the comfort zone
The New Blueprint: challenging the comfort zoneThe New Blueprint: challenging the comfort zone
The New Blueprint: challenging the comfort zone
 
The
The The
The
 
Séance d'orientation pour les candidats de l'examen de la CNE de 2014
Séance d'orientation pour les candidats de l'examen de la CNE de 2014Séance d'orientation pour les candidats de l'examen de la CNE de 2014
Séance d'orientation pour les candidats de l'examen de la CNE de 2014
 
2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation
 
Orientation à l’intention des candidats au format ECOS (2014)
Orientation à l’intention des candidats au format ECOS (2014)Orientation à l’intention des candidats au format ECOS (2014)
Orientation à l’intention des candidats au format ECOS (2014)
 
2014 Candidate Orientation Presentation - Certification Examination in Family...
2014 Candidate Orientation Presentation - Certification Examination in Family...2014 Candidate Orientation Presentation - Certification Examination in Family...
2014 Candidate Orientation Presentation - Certification Examination in Family...
 
Présentation d'orientation de 2014 - EACMC, partie II
Présentation d'orientation de 2014 - EACMC, partie IIPrésentation d'orientation de 2014 - EACMC, partie II
Présentation d'orientation de 2014 - EACMC, partie II
 
Présentation d'orientation - Examen de la CNE 2014
Présentation d'orientation - Examen de la CNE 2014Présentation d'orientation - Examen de la CNE 2014
Présentation d'orientation - Examen de la CNE 2014
 
2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation 2014 NAC candidate orientation presentation
2014 NAC candidate orientation presentation
 
Pre-Exam Orientation for Candidates taking the Certification Examination in F...
Pre-Exam Orientation for Candidates taking the Certification Examination in F...Pre-Exam Orientation for Candidates taking the Certification Examination in F...
Pre-Exam Orientation for Candidates taking the Certification Examination in F...
 
Pre-Exam Candidate Orientation
Pre-Exam Candidate OrientationPre-Exam Candidate Orientation
Pre-Exam Candidate Orientation
 
Practice Ready Assessment for IMG Physicians
Practice Ready Assessment for IMG PhysiciansPractice Ready Assessment for IMG Physicians
Practice Ready Assessment for IMG Physicians
 

Último

Research Methodology and Tips on Better Research
Research Methodology and Tips on Better ResearchResearch Methodology and Tips on Better Research
Research Methodology and Tips on Better ResearchRushdi Shams
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...Nguyen Thanh Tu Collection
 
PHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdf
PHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdfPHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdf
PHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdfSumit Tiwari
 
Auchitya Theory by Kshemendra Indian Poetics
Auchitya Theory by Kshemendra Indian PoeticsAuchitya Theory by Kshemendra Indian Poetics
Auchitya Theory by Kshemendra Indian PoeticsDhatriParmar
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...Nguyen Thanh Tu Collection
 
BBA 205 BE UNIT 2 economic systems prof dr kanchan.pptx
BBA 205 BE UNIT 2 economic systems prof dr kanchan.pptxBBA 205 BE UNIT 2 economic systems prof dr kanchan.pptx
BBA 205 BE UNIT 2 economic systems prof dr kanchan.pptxProf. Kanchan Kumari
 
The OERs: Transforming Education for Sustainable Future by Dr. Sarita Anand
The OERs: Transforming Education for Sustainable Future by Dr. Sarita AnandThe OERs: Transforming Education for Sustainable Future by Dr. Sarita Anand
The OERs: Transforming Education for Sustainable Future by Dr. Sarita AnandDr. Sarita Anand
 
2024.03.16 How to write better quality materials for your learners ELTABB San...
2024.03.16 How to write better quality materials for your learners ELTABB San...2024.03.16 How to write better quality materials for your learners ELTABB San...
2024.03.16 How to write better quality materials for your learners ELTABB San...Sandy Millin
 
The First National K12 TUG March 6 2024.pdf
The First National K12 TUG March 6 2024.pdfThe First National K12 TUG March 6 2024.pdf
The First National K12 TUG March 6 2024.pdfdogden2
 
Metabolism of lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptx
Metabolism of  lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptxMetabolism of  lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptx
Metabolism of lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptxDr. Santhosh Kumar. N
 
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdfArti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdfwill854175
 
Quantitative research methodology and survey design
Quantitative research methodology and survey designQuantitative research methodology and survey design
Quantitative research methodology and survey designBalelaBoru
 
Plant Tissue culture., Plasticity, Totipotency, pptx
Plant Tissue culture., Plasticity, Totipotency, pptxPlant Tissue culture., Plasticity, Totipotency, pptx
Plant Tissue culture., Plasticity, Totipotency, pptxHimansu10
 
THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...
THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...
THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...Subham Panja
 
AI Uses and Misuses: Academic and Workplace Applications
AI Uses and Misuses: Academic and Workplace ApplicationsAI Uses and Misuses: Academic and Workplace Applications
AI Uses and Misuses: Academic and Workplace ApplicationsStella Lee
 
30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc
30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc
30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.docdieu18
 
Dhavni Theory by Anandvardhana Indian Poetics
Dhavni Theory by Anandvardhana Indian PoeticsDhavni Theory by Anandvardhana Indian Poetics
Dhavni Theory by Anandvardhana Indian PoeticsDhatriParmar
 
3.14.24 The Selma March and the Voting Rights Act.pptx
3.14.24 The Selma March and the Voting Rights Act.pptx3.14.24 The Selma March and the Voting Rights Act.pptx
3.14.24 The Selma March and the Voting Rights Act.pptxmary850239
 

Último (20)

Research Methodology and Tips on Better Research
Research Methodology and Tips on Better ResearchResearch Methodology and Tips on Better Research
Research Methodology and Tips on Better Research
 
Least Significance Difference:Biostatics and Research Methodology
Least Significance Difference:Biostatics and Research MethodologyLeast Significance Difference:Biostatics and Research Methodology
Least Significance Difference:Biostatics and Research Methodology
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (FRIE...
 
PHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdf
PHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdfPHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdf
PHARMACOGNOSY CHAPTER NO 5 CARMINATIVES AND G.pdf
 
Auchitya Theory by Kshemendra Indian Poetics
Auchitya Theory by Kshemendra Indian PoeticsAuchitya Theory by Kshemendra Indian Poetics
Auchitya Theory by Kshemendra Indian Poetics
 
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...
HỌC TỐT TIẾNG ANH 11 THEO CHƯƠNG TRÌNH GLOBAL SUCCESS ĐÁP ÁN CHI TIẾT - HK2 (...
 
BBA 205 BE UNIT 2 economic systems prof dr kanchan.pptx
BBA 205 BE UNIT 2 economic systems prof dr kanchan.pptxBBA 205 BE UNIT 2 economic systems prof dr kanchan.pptx
BBA 205 BE UNIT 2 economic systems prof dr kanchan.pptx
 
The OERs: Transforming Education for Sustainable Future by Dr. Sarita Anand
The OERs: Transforming Education for Sustainable Future by Dr. Sarita AnandThe OERs: Transforming Education for Sustainable Future by Dr. Sarita Anand
The OERs: Transforming Education for Sustainable Future by Dr. Sarita Anand
 
2024.03.16 How to write better quality materials for your learners ELTABB San...
2024.03.16 How to write better quality materials for your learners ELTABB San...2024.03.16 How to write better quality materials for your learners ELTABB San...
2024.03.16 How to write better quality materials for your learners ELTABB San...
 
The First National K12 TUG March 6 2024.pdf
The First National K12 TUG March 6 2024.pdfThe First National K12 TUG March 6 2024.pdf
The First National K12 TUG March 6 2024.pdf
 
Metabolism of lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptx
Metabolism of  lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptxMetabolism of  lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptx
Metabolism of lipoproteins & its disorders(Chylomicron & VLDL & LDL).pptx
 
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdfArti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
 
Quantitative research methodology and survey design
Quantitative research methodology and survey designQuantitative research methodology and survey design
Quantitative research methodology and survey design
 
Plant Tissue culture., Plasticity, Totipotency, pptx
Plant Tissue culture., Plasticity, Totipotency, pptxPlant Tissue culture., Plasticity, Totipotency, pptx
Plant Tissue culture., Plasticity, Totipotency, pptx
 
THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...
THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...
THYROID HORMONE.pptx by Subham Panja,Asst. Professor, Department of B.Sc MLT,...
 
AI Uses and Misuses: Academic and Workplace Applications
AI Uses and Misuses: Academic and Workplace ApplicationsAI Uses and Misuses: Academic and Workplace Applications
AI Uses and Misuses: Academic and Workplace Applications
 
30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc
30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc
30-de-thi-vao-lop-10-mon-tieng-anh-co-dap-an.doc
 
ANOVA Parametric test: Biostatics and Research Methodology
ANOVA Parametric test: Biostatics and Research MethodologyANOVA Parametric test: Biostatics and Research Methodology
ANOVA Parametric test: Biostatics and Research Methodology
 
Dhavni Theory by Anandvardhana Indian Poetics
Dhavni Theory by Anandvardhana Indian PoeticsDhavni Theory by Anandvardhana Indian Poetics
Dhavni Theory by Anandvardhana Indian Poetics
 
3.14.24 The Selma March and the Voting Rights Act.pptx
3.14.24 The Selma March and the Voting Rights Act.pptx3.14.24 The Selma March and the Voting Rights Act.pptx
3.14.24 The Selma March and the Voting Rights Act.pptx
 

Blueprinting update - 2014 Ottawa Conference

  • 1. Ottawa Conference/CCME C. Touchie, A. Gotzmann, C. Streefkerk, A. De Champlain, Medical Council of Canada Putting the Pieces Together to Create a Defensible Blueprint: A Subject Matter Expert Based Process April 2014 Ottawa, Canada 1
  • 2. Credentialing Examinations • Validity: – Associated with the interpretation assigned to test scores NOT with the score or with the test itself – Kane’s validity argument: • Scoring • Generalization • Extrapolation  the content of the test is matched appropriately to the demands of practice • Decision 2
  • 3. Practice Analysis • Practice Analysis Methods: – Critical Incident Technique – Task Inventory – Professional Practice Model • Professions: dynamic and social 2
  • 4. Purpose of Blueprinting • … is to assure the public that physicians licensed to practice medicine have the required knowledge, skills and attitudes for safe and effective patient care. • Only those who meet this standard are qualified to enter professional practice 4
  • 5. Decision points • What we are assessing: – Core foundational competencies expected of all physicians regardless of discipline for the purpose of fulfilling one of the requirements of licensure • Decision point 1 – Assessments leading to a decision for entry into residency • Decision point 2 – Assessments leading to a decision for entry into independent practice 5
  • 6. Practice Analysis • Factors to Consider 1. Information required 2. Best sources of information 3. Strategy for obtaining that information 4. Strategy to put it all together 6Modified from M. Raymond, AERA 2014
  • 7. 1. Information Required 7 Focus of practice descriptors Level of specificity Attributes to be measured Current Issues Future of medical education Broad and complex Assessment needs Supervising PGY-1 Transition between MD and residency Entrustable professional activities Supervisory needs Incidence and Prevalence Common clinical problems Specific presentations (age, setting) Frequency National Survey MCC objectives Specific KSAs Importance of KSAs at 2 decision points
  • 8. 2. Sources of Information 8 Sources of Information Current Issues Review of literature, medical educators Supervising PGY-1 New PGY-1s, supervisors Incidence and Prevalence Billing data (CIHI, provincial sources), published literature National Survey Physicians: recent LMCC recipients, clinical teachers Informed public
  • 9. 3. Data Collection Strategy 9 Data Collection Strategy Current Issues Review of literature, iterative review by experts Supervising PGY-1 Consensus technique/Survey Incidence and Prevalence Review of existing data National Survey Physicians: Survey Informed public: Survey and focus groups
  • 10. 4. Strategy to pull it all together Blueprint and Test Specifications Defined 2 candidates Common Blueprint Test Specifications for each decision point Subject Matter Experts 12 Experts 3 day meeting 4 Reports Current Issues PGY-1 supervision Incidence and Prevalence National Survey
  • 11. Who were the SMEs? Blueprint MRA Rep of Council Central Examination Committee Objectives Committee Test Committees RCPSCCFPC UGME Deans PGME Deans University Rep of Council
  • 12. Results: Proposed Common Blueprint Dimensions of Care Health Promotion and Illness Prevention Acute Chronic Psychosocial Aspects PhysicianActivities Assessment/ Diagnosis Management Communication Professional Behaviors 12
  • 13. Dimensions of Care Health Promotion and Illness Prevention Acute Chronic Psychosocial Aspects Row Percent PhysicianActivities Assessment/ Diagnosis 30±5 Management 20±5 Communication 30±5 Professional Behaviors 20±5 Column Percent 20±5 30±5 30±5 20±5 100 Assessments leading up to Decision 1: Entry into Residency 13
  • 14. Dimensions of Care Health Promotion and Illness Prevention Acute Chronic Psychosocial Aspects Row Percent PhysicianActivities Assessment/ Diagnosis 25±5 Management 35±5 Communication 20±5 Professional Behaviors 20±5 Column Percent 20±5 25±5 35±5 20±5 100 Assessments leading up to Decision 2: Entry into Independent Practice 14
  • 15. Topics / Knowing Processes / Doing Processes/Doing Textbook Topics Academic Disciplines Organ System Clinical Problem / Reason for Visit Medical Specialty Cognitive Processes or Skills Competency (ACGME, CanMEDS) Physician Task or Skill
  • 16. Conclusions Defensible Blueprint: • Define assessment purpose • Choose information sources to inform appropriately • Include judgments from stakeholders Lessons learned: • Careful planning – Team – Information gathered – Process 16
  • 17. Thank You! Questions – Comments ctouchie@mcc.ca 17