Creating outcomes and assessment measures -jarvis christian college

6/8/2012 Assessment Workshop OIRPA (ALLEN): 09/09

Strategic
Planning
Program
Assessment
Mission Process
Review

Institutional
Effectiveness
Processes

Program
Informed
Review
Budgeting
Process

Planning
Committee


 Assessment is a process of gathering and interpreting information to determine the
degree to which a program is meeting established goals and then using that
information to enhance the program.

 As a part of institutional effectiveness, assessment is required at all levels. There is one core
requirement and one comprehensive standard established by the Southern Association of
Colleges and Schools that specifically addresses assessment and student learning
outcomes.

Core Requirement 2.5

Comprehensive Standard 3.3.1


Follow-Up on Identify
Implemented Expected
Changes Outcomes

Identify
Interpret How Where
Results Will Inform Outcomes
Teaching/Learning are
Addressed

Determine
Methods and
Schedule of
Criteria to
Assessment
Assess
Outcomes

Identify the
Who Will Be
Expected
Assessed
Performance

Source: Maki, P. (2004). Assessing for Learning: Building A Sustainable Commitment Across the Institution (pp. 4-5). Sterling, VA: Stylus Publishing.

Formative assessment is utilized to Summative assessment is cumulative in
determine immediately whether students nature and is utilized to determine
have learned what the instructor whether students have met the course
intended. This type of assessment is goals or student learning outcomes at
intended to help instructors identify material the end of a course or program.
that needs to be clarified or re-taught and
should not be used to evaluate or grade
students. Results of formative assessment
can assist instructors in ascertaining
whether curriculum or learning activities
need to be modified during a class session
or before the next class meets.

Central Michigan University Provost’s Office. (n.d.) Formative and summative assessment. Central Michigan
University. http://www.provost.cmich.edu/assessment/toolkit/formativesummative.htm


• This level of assessment focuses on individual student performance within a course aimed at instructional improvement.
• When to use: when you want to know, as an instructor, about what and how students are learning.
Classroom
Formative

• This level of assessment focuses on the learning outcomes of a course and how well students achieve them
• When to use: when you want to provide feedback about students’ learning outcomes at the end of a course that can be used to
Course make revisions in course content, adjust the sequencing of courses, improve overall degree program curricula, etc.
Gen. Ed./Pro.

• This level of assessment focuses on the extent to which an academic program or educational support unit achieves its stated
student learning and/or program outcomes
Program • When to use: used to satisfy CS 3.3.1 and used to improve program curriculum to ensure student success
CS 3.3.1

• Assessment of how well an institution is meeting its objectives/goals
Institutional
CR 2.5


MATURE
SMART Outcomes Assessment
Methods


CREATING SMART OUTCOMES
• What is an outcome?
– Program Outcome: these are general and broad statements that focus on identifying
content or what students should learn, understand, or appreciate as a result of the program
(Maki, 2004). These are like goals or competencies.

– Process Outcome: The end result of what a program or process is to do, achieve, or
accomplish. Process outcomes can be as simple as completion of a task or
activity, although this is not as meaningful as it could be and does not provide information
for improvement. For example, the Department of English will increase the diversity of their
application pool by 50%. (Adapted from www.assessment.tamu.edu)

– Student Learning Outcome: Student Learning Outcomes (SLOs) are specific statements
that describe the learning that students are expected to achieve as a result of the program.
That is, active verbs are used to identify the “actions, behaviors, dispositions, and ways of
thinking or knowing that students should be able to demonstrate.” (Maki, 2004)


S • Specific

M • Measureable

A • Attainable

R • Realistic

T • Time-Bound
Drucker, P. F. (2006)



S •Specific

• Specific to your unit or program
M •Measureable
• Learning outcomes that describe the expected
abilities, knowledge, values and attitudes a student has based on their
learning
A •Attainable

• Expected outcomes that are critical to your unit or program
R •Realistic
• Data should create opportunity to make improvements

T •Time-Bound

Information Source: 2005 University of Central Florida 4A-1 UCF Academic Program Assessment Handbook September 2006 Information, Analysis, and Assessment



S •Specific

• The intended outcome should be one for which it is feasible to collect
M •Measureable accurate and reliable data.
• Learning outcomes: use Bloom’s Taxonomy -- Relies on active
verbs that identify what students should be able to
A •Attainable demonstrate, represent, or produce
(e.g., create, apply, formulate, etc.)
• Process outcomes: focus on how accurate, how economical, how
R •Realistic
satisfied, how prompt, how many, etc.
• Consider resources (e.g., staff, technology, assessment
T •Time-Bound support, institutional level surveys, etc.)
• Provide opportunities to triangulate data



S •Specific
Questions that can assist in defining aggressive outcomes
• How have the students’ experiences in the program contributed to their
M •Measureable
abilities, knowledge, values and attitudes? Ask:
o Cognitive skills: What does the student know?
A •Aggressive o Performance skills: What does the student do?
o Affective skills: What does the student care about?

R •Realistic • What are the college-level knowledge, abilities, values and attitudes
expected of graduates of the program?
• Can students achieve these outcomes, based on the current
T •Time-Bound
curriculum, as a result of the program? (Map to curriculum)



S •Specific

Determine what standards are expected from students in your program.
M •Measureable
Then ask, do I have the resources to do this? Consider human and
financial resources.
A •Aggressive

R •Realistic

T •Time-Bound




S •Specific

Can this outcome be achieved within the specified time-frame?
M •Measureable

A •Aggressive

R •Realistic

T •Time-Bound



S •Specific Poor: Students should know the historically important systems of
psychology.

Better: Students should understand the
M •Measureable
psychoanalytic, Gestalt, behaviorist, humanistic and cognitive
approaches to psychology.
Best: Students should be able to recognize and articulate the
A •Aggressive
foundational assumptions, central ideas and dominant criticisms of the
psychoanalytic, Gestalt, behaviorist, humanistic and cognitive
approaches to psychology. (Ex. From University of Central Florida)
R •Realistic

T •Time-Bound

6/8/2012 Assessment Workshop FITTS: 09/09

EVALUATE THESE OUTCOMES
S •Specific

Example 1: Students completing the undergraduate program in
Home Buying will have knowledge of Home Buying Principles.
M •Measureable

Example 2: All juniors and seniors will pass the Department of
Sociology’s distance learning course.
A •Aggressive

R •Results-Oriented

T •Time-Bound


S •Specific

M •Measureable

A •Aggressive

R •Results-Oriented

T •Time-Bound


MATURE
SMART
Assessment
Outcomes
Methods


CREATING MATURE ASSESSMENT METHODS
M •Match

A •Appropriate

T •Target

U •Useful

R •Reliable

E •Efficient



M •Match

Match the outcome with the appropriate assessment method.
A •Appropriate

Consider the ability of an assessment method to address specific
T •Target assessment questions, as well as its relevance and utility.

U •Useful

R •Reliable

E •Efficient



M •Match

• Which type of assessment method is appropriate?
A •Appropriate
• Direct: Achievement
• Indirect: Perception
T •Target
• Are the resources available to use this method?
U •Useful
• Would your department have control over the method of
assessment?
R •Reliable

E •Efficient



M •Match

Determine the desired level of performance.
A •Appropriate Benchmarking (using institutional data, peer institution data,
nationally accepted standard for the field or discipline)
T •Target
The criterion/benchmark for success should be stated in terms of
percentages, percentiles, averages or other quantitative measures.
U •Useful
Establish a reasonable benchmark. Avoid using absolutes such as
“100%”, “zero”, and “all” when establishing criteria.
R •Reliable

NOTE: If you have previously measured an outcome, it is helpful to use this
E •Efficient as the baseline for setting a target for next year.



M •Match

Can you make inferences about the progress towards the outcome?
A •Appropriate

Achieving high-quality assessment requires addressing issues identified
T •Target by Linn and Baker (1996) and Herman, Aschbacher, and Winters (1992)
such as:
•Does the selected method cover the curriculum objectives?
U •Useful
•Does it match the desired level of complexity?
•Can the results be generalized, and to what extent?
R •Reliable •Will information be gained that will be useful in improving programs?

E •Efficient



M •Match

A reliable assessment method is one that yields consistent responses
A •Appropriate
over time. (e.g., lack ambiguity, completion time, etc.)

T •Target

U •Useful

R •Reliable

E •Efficient



M •Match

Estimate the time required to develop, administer and evaluate
A •Appropriate
various assessment methods.

T •Target If possible, use two measures. Look for ways to conduct a deeper
analysis. Use qualitative and quantitative assessment measures
(qualitative: open-ended questions on surveys, focus groups, and
U •Useful
structured interviews)

R •Reliable

E •Efficient


CREATING MATURE ASSESSMENT METHODS: DIRECT METHODS
 Capstone course assignments  Observations in class by evaluator who is
 Capstone projects not the instructor
 Case studies, hypothetical situation  Peer evaluation of practical skills using
responses rubric
 Minute papers  Clinical practice or internship skill
 Course-embedded questions assessment
 Portfolio assignments  Standardized Examinations and Tests
 Standard rubric  National Test
 Research papers  State Test
 Performance appraisal of in class  Local Tests
exercises  Pre-post test
 Expert evaluation  Test-embedded questions (across several
course sections)
 Licensure Exams

CREATING MATURE ASSESSMENT METHODS: INDIRECT METHODS
 Cooperative Institutional Research  Advisory Board Surveys
Program (CIRP)  Alumni Surveys
 College Student Expectations  Assessment Surveys
Questionnaire (CSXQ)  Customer Surveys
 Institutional Priorities Survey  Employer Surveys
 National Survey of Student Engagement  External Peer Review Surveys
(NSSE)  Point of Service Surveys
 Your First College Year (YFCY)  Focus Groups
 Alumni Surveys  Interviews
 Employer Surveys  Case Studies
 Graduating Seniors and Graduates Surveys  Retention Data
 Non-Returning Student Survey
 Student Satisfaction Surveys
 Entering Freshmen Survey

Example Outcome 1: Students will be able to recognize and articulate
M •Match
the foundational assumptions, central ideas and dominant criticisms of
the psychoanalytic, Gestalt, behaviorist, humanistic and cognitive
A •Appropriate approaches to psychology. (Ex. From UCF)

T •Target
How can we assess this outcome?
U •Useful

R •Reliable

E •Efficient


M •Match

A •Appropriate

T •Target

U •Useful

R •Reliable

E •Efficient


Central Michigan University Provost’s Office. (n.d.) Formative and summative assessment. Central
Michigan University. http://www.provost.cmich.edu/assessment/toolkit/formativesummative.htm

Drucker, P. F. (2006). The Practice of Management (originally published in 1954). New York, NY:
HarperCollins.

Stanford University (2007). Analyzing Assessment Results: http://www.stanford.edu/dept/pres-
provost/irds/assessment/analyze.pdf

University of Central Florida (2005). UCF Academic Program Assessment Handbook February 2005
Information, Analysis, and Assessment: http://oeas.ucf.edu/doc/acad_assess_handbook.pdf

Maki, P. (2004). Assessing for Learning: Building A Sustainable Commitment Across the Institution (pp.
4-5). Sterling, VA: Stylus Publishing.


Mrs. Cleopatra Allen

•callen@wileyc.edu
•(903) 927-3295

Creating outcomes and assessment measures -jarvis christian college

Recomendados

Recomendados

Más contenido relacionado

Similar a Creating outcomes and assessment measures -jarvis christian college

Similar a Creating outcomes and assessment measures -jarvis christian college (20)

Último

Último (20)

Creating outcomes and assessment measures -jarvis christian college

Notas del editor