Rutgers University's presentation to Colorado Community College System's CHAMP Advisory Committee meeting presentation August 25, 2014

1. CHAMP EVALUATION
Heather McKay
Director
Education and Employment Research Center
School of Management and Labor Relations

2. Who is EERC?
Education and Employment Research Center
School of Management and Labor Relations Rutgers
We conduct research and evaluation on workforce development
and education issues.
Related Projects:
• COETC Evaluation
• CHEO Evaluation
• Colorado Sectors Project Evaluation
• Lumina Foundation project on linking Higher Education and
the Workforce System

3. CHAMP
The Colorado Helps Advanced Manufacturing Program (CHAMP)
was created to increase the number of manufacturing degrees and
certificates that align with the industry’s competencies and skills
and to establish a pipeline of highly qualified advanced
manufacturing workers. CHAMP goals are:
• Build off Colorado’s existing and emerging manufacturing sector
partnerships and career pathway work to develop employer driven
curriculum
• Use technology to accelerate training and reach a broad audience
• Redesign the current Colorado Community College System model for credit
for prior learning to accelerate certification
• Develop stackable and latticed certificate with agreements between
participating community colleges and Metropolitan State University of
Denver

4. Evaluation
• Third Party Evaluation is required by the USDOL
• Involves:
- Outcomes evaluation
- Quasi-experimental design
- Comparison Cohort
- Outcomes reporting for schools and consortium
- Scorecard
- Process evaluation
- Focused look a various grant topics
- Examination of the project implementation
-Sustainability

5. Evaluation and Process
• Program Pathways
• Data collection from IR – at system level and non-system
schools – Happens twice a year.
• Surveys – students, faculty, employers
• Interviews – Project Lead and Navigator interviews coming
soon
• Site Visits – Likely Spring 2015
• Navigator monthly data collection
• Quarterly reports
• Review of project materials
• Attendance at meetings and events

6. Some of the Key Evaluation Focal Points
– Interactive Career Map.
• Process of developing the career map.
• Use of the career map by industry, buy faculty and staff at colleges,
and students.
– Portfolio Assessment for Colorado’s advanced manufacturing.
• Process for developing the portfolio including partners who were
involved, key decisions made on structure, and tools used.
• Use of Prior Learning Assessment (PLA) after implementation.
• Analysis of grades earned by students using PLA and its impact on
completion.
• Analysis of persistence and degree completion with comparison to
students who do not use PLA.

7. Key Evaluation Focal Points Cont.
– Credit for prior learning redesign.
• Process to develop protocol.
• Implementation of protocol at individual colleges.
– Employer driven certifications and curricula content.
• Process of revision for certifications and curricula.
• Role played by employers in the process.
• Analysis of outcomes through interviews with colleges, employers, and other
stakeholders.
• Student outcomes using a comparison cohort design.
• Wage data analysis.
– Online and hybrid course development.
• Development of online and hybrid courses.
• Student outcomes using a comparison cohort design.

8. Key Evaluation Focal Points Cont.
– Massive Open Online Courses (MOOCs).
• Process for developing MOOCs.
• Use of MOOC’s by staff, faculty and students at the colleges
• Retention and completion of MOOCs.
– Streamlined transfer and articulation to Metro State University of Denver
(MSUD).
• Development of articulation agreement with MSUD.
• Use of articulation agreement for student transfer.
• Tracking of students first semester after transfer, looking at grades and
retention.
– Navigators.
• Roles of navigators.
• Details of Navigator position at each of the colleges.
• Tracking of students who use Navigator services.

9. Key Evaluation Focal Points Cont.
– Internships.
• Development of internships.
• Use of internships.
• Outcomes of internships.
• Employment post internship.
– Professional Development.
• Methods of professional development.
• Use of professional development.
• Perceptions of faculty and staff of the professional
development.

10. Some Evaluation Products
• Year-one report – Fall 2014
• Scorecard annually – Spring 2015
• College case studies
• Navigator briefing report and updates
• Curriculum Redesign – outcomes reports annually and Final
report
• Faculty Development Briefing report
• CO manufacturing industry report

11. Some Key Team Members
Quantitative Team:
• Debra Borie-Holtz
• Joseph Rua
• James Lloyd
• Cristina Gherasimov
Qualitative Team:
• Renee Edwards
• Suzanne Michael
• Audrey Mattoon
• Dilafruz Nazarova

12. Website
You can find – contact information, staff information, and reports
from past projects.
http://smlr.rutgers.edu/eerc
My Contact Information:
Heather A. McKay
hmckay@work.Rutgers.edu
848-445-4735
THANK YOU!