This document outlines the key elements of an effective program review process for community colleges. It discusses integrating program review, planning, resource allocation, and assessment of student learning outcomes and institutional effectiveness. The presentation provides details on developing program missions aligned with institutional missions, identifying student and program outcomes, analyzing achievement and learning outcome data, identifying gaps, and using the results to plan program improvements and reallocate resources. The goal is to embed this systematic, ongoing process at all levels of the institution to continuously assess and improve programs, student achievement, and institutional quality.

1. Accrediting Commission
for
Community and Junior
Colleges
1

2. Elements of an Effective
Program Review for Integrated
Planning, Learning Outcomes,
and Assessment
Dr. Barbara Beno, President
Dr. Steve Maradian, Vice President
Accrediting Commission for Community and Junior Colleges
WASC
Academic Resource Conference
21 April 2009
2

3. Workshop Purpose: learn to
Construct an effective, integrated system of
program review, planning, and resource
allocation
Enable the institution to continually assess its
effectiveness
Identify and understand the component parts of
a viable program review process
Continued
3

4. Workshop Purposes
Continued
Embed the process at all levels of the institution
Create a culture of informed decision-making
and sustainable continuous quality improvement
Use results of this assessment to advance
effectiveness and educational quality of an
institutional program
4

5. Elements of Effective
Program Review
5

6. District Mission
Institutional Mission and
Expectations
Programmatic
Mission
Outcomes
Design
Identify Program Inputs
Gaps Changes
Allocate
Needed
Resources Process
Analysis of
Outcomes
Align
Implement Resources
Assessment
Program
6

7. District Mission
Institutional Mission and
Expectations
Programmatic
Mission
Outcomes
Design
Identify Program Inputs
Gaps Changes
Allocate
Needed
Resources Process
Analysis of
Outcomes
Align
Implement Resources
Assessment
Program
7

8. System Mission
and
Expectations
Link between district/system (d/s) and
college mission(s)
Functional map identifies roles and
responsibilities between college and d/s
College Review at campus and district level(s)
Mission
All data used to drive college improvements
and d/s understanding/support
College planning is linked with the d/s in
support of college mission(s) and activities
8

9. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Resources
Program
9

10. Institutional Mission Defines
Institution’s Broad Purpose
The Intended Student Population
A Commitment to Student Learning
(the institutional mission flows from the system mission)
All programs and services must align with this mission
The mission is regularly reviewed and revised as needed
The mission is central to planning and decision making
10

11. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Measure Implement Align
Outcomes Program Resources
11

12. Programmatic Mission
Program’s Purposes:
(E.g.,workforce training, transfer, general education,
pre-collegiate education, baccalaureate education, etc.)
Define Programmatic Student Learning Outcomes
The program’s mission must align with institutional mission
The mission is reviewed regularly and revised as needed
The mission is central to programmatic planning
and decision making
12

13. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Resources
Program
13

14. Outcomes
Student Achievement Student Learning
14

15. How are outcomes analyzed?
Analysis requires a judgment about whether the
outcomes are “good enough”
– Benchmarks of performance of similar colleges
– Benchmark of past institutional performance
(e.g., 5% increase over previous year’s)
– Externally imposed benchmarks – state governments,
professional associations, employers, etc.
– Targets or goals established by the institution
– Others?
Analysis requires valid data or information
15

16. Student Achievement Outcomes
• Course completion
• Retention term to term
• Progression to next course/level
• Program completion
• Degree/certificate completion
• Transfer
• Success/Scores on licensure exams
• Job placement
All data collected must be analyzed
16

17. Student Learning Outcomes
• Established by faculty at the course, program,
degree, and certificate levels;
• Faculty, administrators, and trustees play a role
• Authentic assessment designed to
determine what students actually learn;
Assessment data are collected and analyzed.
17

18. Relationship Among Course, Program,
and Institutional SLOs
Institutional Student Learning Outcomes: there is a broad set of SLOs which
describe outcomes students have achieved when they complete their
degree/certificate/program
Program Student Learning Outcomes: a set of SLOs students have achieved
upon completion of a sequence or cluster of courses in a program. These
contribute to institutional SLOs
Course Student Learning Outcomes: these are agreed-upon outcomes
students must achieve to complete the course and these contribute to program
and/or institutional SLOs
18

19. Development of Student Learning
Outcomes
Course Level:
Discipline Faculty
Program Level: programmatic
faculty in the disciplines and external
clients (employers, transfer
institutions, & the public)
Institutional Level:
Faculty, Academic Administrators
Trustees
Through the
Mission, Values, and
Commitment
19

20. SLO Assessment
Planning is critical for Commitment for all
an on-going systemic organizational levels
effort Celebrate
Document and use of accomplishments and
assessment data for improve student
improvement learning
Information sharing Don’t delay; it’s not a
across disciplines to fad
share best practices
20

21. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Resources
Program
21

22. Inputs
Existing Resources Students
22

23. Existing Resources
Staff
(number and capacity)
Facilities
Equipment
Funding
All continually aligned to course/program
23

24. Students
Who are they?
How well prepared are they?
What are their educational goals?
What are their program goals?
What are their support needs?
24

25. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Inputs
Changes
Allocate
Needed
Resources Process
Analysis of
Outcomes
Align
Implement Resources
Assessment Program
25

26. Process:
Pedagogy and Support Services
Course outlines
Course content, intended SLOs
Strategies for assessing student learning
Instructional support and services for students
Variable delivery modes & scheduling, etc.
26

27. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Align
Implement Resources
Assessment
Program
27

28. Ongoing Alignment
of Resources
(Human, Physical, Technology, & Financial)
• Sufficient and appropriate resources are provided to
meet program and institutional needs
• Resource allocation through established processes
is integrated with and informed by
institutional evaluation and planning
District/system support necessary to ensure
that program review leads to meaningful
improvements at the college
28

29. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Assessment Implement Align
Program Resources
29

30. Program Implementation
Scheduling and sequence of courses
Alignment with general education courses
30

31. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Inputs
Gaps Changes
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Resources
Program
31

32. Assessment
Gather meaningful student achievement data
Measure attainment of student learning outcomes
Data are qualitative and quantitative
Data are longitudinal where appropriate
Outcomes data are continually collected and analyzed
32

33. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Resources
Program
33

34. Analysis of Outcomes
See Slide 15
• Understanding the meaning of the data collected
• Judgments about what is good enough must be made
• Institutional and system governance groups
must be informed
Some colleges collect data through the
district/system; others collect data independently
34

35. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program
Gaps Changes Inputs
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Program Resources
35

36. Gap Analysis
Comparison of actual outcomes with intended
or targeted outcomes.
What worked to attain intended outcomes?
What part(s) of the program needs to be changed
to attain intended outcomes?
36

37. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program Inputs
Gaps Changes
Allocate
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment
Program Resources
37

38. Design Program Changes
To program and course outcomes
To inputs and processes
To human, physical, technological
and financial resources
Programmatic changes must be in line with mission
Planned changes must inform the process
of budget allocation
System consultation is pivotal
38

39. Institutional Mission
Programmatic Mission
Outcomes
Design
Program
Identify Changes Inputs
Gaps
Allocate
System role
Needed
Resources Process
Analysis of
Outcomes
Implement Align
Assessment Resources
Program
39

40. (Re)Allocate Needed Resources
• Results of program review are clearly and
consistently linked to the institutional planning
and resource (re)allocation processes
• Constituent groups on campus are
involved in the decision-making processes
• Board Governance decisions reflect institutional
priorities identified through assessment
District/system support necessary to ensure that
program review leads to meaningful
improvements at the college
40

41. Reallocate Needed Resources
•Program review and planning inform/direct
resource reallocation
District/system support necessary
to ensure that program review leads
to meaningful improvements at the college
41

42. Course
Level
Student
Learning
Outcomes
Program Level
Student Learning Outcomes
which meet Program goals
and intentions
Institutional Level Student Learning Outcomes
these are agreed-upon outcomes students must
achieve to complete the degree or certificate.
42

43. The Sustainable Continuous
Quality Improvement Level of
Program Review
Program review processes are ongoing, systematic,
and used to assess and improve student learning and
achievement; the Board is informed of successes and
needed improvements
The institution reviews and refines the program
review process to improve institutional effectiveness
The results of program review are used to continually
refine and improve program practices resulting in
improvements in student achievement and learning
43

44. A Culture of Assessment
Institutional – Professional
Commitment development
– Program review – Planning and resource
allocation
– Institutional
effectiveness – Involvement and
support from
– Information sharing
Board
– Systemic assessment President
planning Faculty
– Common assessment Staff
terminology Information technology
44

45. Institutional Mission
Programmatic Mission
Outcomes
Design
Identify Program Inputs
Gaps Changes
Allocate System role??
Needed
Resources Process
Analysis of
Outcome
Implement Align
Assessment Resources
Program
45

46. How Did We Get Here?
Accreditation Standards have required program
review and integrated planning since the 1990s
To those requirements, there is an expectation that
Student Achievement and SLO data become a part of
the program review and planning processes
46

47. What is Expected Now?
Accredited colleges are expected to be at the
Sustainable Continuous Quality Improvement Level
on the Commission’s Rubric for Evaluating
Institutional Effectiveness – Parts I and II (Program
Review and Planning)
47

48. What is Expected in the
Future?
Accredited colleges are expected to remain at the
Sustainable Continuous Quality Improvement Level
on the Commission’s Rubric for Evaluating
Institutional Effectiveness – Parts I and II (Program
Review and Planning)
48

49. What is Expected in the
Future?
By 2012, colleges are expected to be at the
Proficiency Level on the Commission’s Rubric Part III
– Student Learning Outcomes
ACCJC to release a more detailed “proficiency”
description in Fall 2010
Over time at this level, colleges will achieve the
fourth level (Sustainable Continuous Quality
Improvement)
49

50. Thoughts?
50

51. puzzle
Are all
pieces clearer?
51

52. An Exercise
Using Program Review Data to
Plan Improvements
52

53. Thank You
53