Using the recent AAC&U report A Crucible Moment: College Learning and Democracy’s Future and research arising from the Personal and Social Responsibility Inventory, participants will discuss current areas of research related to civic learning and identify areas for additional research. Discussion of best practices will focus on both indirect and direct measures of civic learning and campus engagement, including existing and available measures that can be adapted for use on participants’ campuses. Robert D. Reason, Associate Professor of Student Affairs and Higher Education, and Joshua J. Mitchell, doctoral student and Graduate Research Assistant in the Research Institute for Studies in Education, both at Iowa State University

1. Robert Reason Josh Mitchell Kathleen Gillon
Iowa State University
Upper Midwest Civic Engagement Summit
Dubuque, IA
May 29, 2013

2. • Introductions:
• Who we are
• Our interest in civic learning research and assessment
• What we hope to gain from this workshop
• Workshop Goals and Interests:
• Why Measure Civic Learning?
• What is Civic Learning?
• How do we Measure Civic Learning?

3. • General Workshop Structure:
• Assessing Civic Learning: Challenges and Opportunities
• Defining Civic Learning: What Works?
• A Crucible Moment
• History and Call to Action
• Key Findings and Recommendations
• Key recommendations for Researchers
• Campus-Based Assessment:
• Assessing Engagement
• Indirect vs. Direct Methods of Evaluation
• Connecting Assessment Models with Measurable Learning Outcomes
• Tools and Techniques

4. • Group Discussion - Major Issues in Assessing Civic Engagement on Campus
• What are some of the biggest obstacles in assessing civic learning and
engagement on your campus?
•
•
•
• What are some of the solutions found to address these challenges?
•
•
•

6. • Commissioned by The Department of Education, initiated by Martha
Kanter, Under Secretary of Education
• Partnership between Global Perspective Inventory, Inc. (GPI) and the
Association of American Colleges and Universities
• National Task Force on Civic Learning and Democratic Engagement
• A series of five national roundtables representing leaders from a
wide range of constituent groups inside and outside of higher
education
• Several open sessions at a number of national and regional
conferences
• A commissioned paper on what we know from research on the
impact of civic learning and democratic engagement on student
learning and development
6

8. KEY RECOMMENDATIONS FOR HIGHER
EDUCATION
1. Foster a CIVIC ETHOS across all parts of the campus and
educational culture.
2. Make CIVIC LITERACY a core expectation for all students.
3. Practice CIVIC INQUIRY across all fields of study.
4. Advance CIVIC ACTION through transformative
partnerships, at home and abroad.

9. WHY STUDY CIVIC LEARNING??

10. EVIDENCE SUPPORTING NEED FOR
COMPREHENSIVE CIVIC LEARNING
• U.S. ranked 139th in voter participation of 172
world democracies in 2007.
• Only 24% of graduating high school seniors
scored at the proficient or advanced level in civics
in 2010. (NCES, 2011)
• Less than ½ of 12th graders reported studying
international topics as part of a civics education.
(NCES, 2011)

11. STUDENTS VIEWS ON CIVIC LEARNING AND
HIGHER EDUCATION

12. FINDINGS RELATED TO CIVIC LEARNING
AND HIGHER EDUCATION
• A group of 14,000 college seniors surveyed in 2006-07
averaged just over 50% in a civic literacy exam.
(Intercollegiate Studies Institute, 2007)
• Just over 1/3 of college faculty surveyed strongly agreed
their campus actively promotes awareness of US or global
social, political, and economic issues.
• Only slightly more than one-third of students surveyed
felt strongly that faculty publicly advocated the need for
students to become active and involved citizens. (Dey et al.
2009)

13. A FOUNDATION PARTIALLY LAID
Curricular civic pathways
• Making civic literacy a core expectation for all students
in general education programs
• Integrating civic inquiry into a central field of study
Powerful civic pedagogies
• Intergroup and deliberative dialogue
• Service Learning and Volunteer Service
• Collective civic problem solving

14. THE CHALLENGE OF THE NEXT DECADES OF
CIVIC LEARNING
• From elective to expected
• From one time to progressive learning over time
• From individually oriented civic action to
collaboratively oriented action
• From some departments, divisions, and people to
everyone all over campus
• From parallel practices and programs to
integrated ones

15. PRIORITIES FOR FUTURE RESEARCH
 Disseminate existing assessment tools for measuring civic learning.
 Amass and publicize evidence that shows how civic learning, civic agency, and
democratic engagement result in increased retention and college success.
 Use the Civic Investment Plan matrix to identify specific action research projects
that could be initiated at one’s own institution.
 Establish standards in civic learning to serve as guidelines for measuring and
reporting progress.
 Sponsor and support further research on the impact of programs and
partnerships that foster civic learning and democratic engagement on learning
outcomes and student development.
 Include additional research questions in routinely administered higher education
surveys to explore how learning environments can enhance key civic
competencies
 Develop national civic indicators and report on levels of civic and democratic
knowledge, skills, values, and action achieved by high school and college
graduates.

16. FIVE AREAS IN WHICH MORE EVIDENCE ON
CIVIC LEARNING IS NEEDED:
• Existing data on civic outcomes is almost entirely based on student self-reports of
their behaviors, attitudes, and growth over time. Multiple measures are needed to
capture students’ ability to demonstrate civic competencies.
• Data on civic outcomes connected with the experiences of underserved students
(underrepresented minorities, first-generation, transfer, and low income).
• Greater evidence is needed on outcomes more closely associated with civic
competencies or capacity building, particularly for success in an increasingly global
and diversifying nation (e.g. collaborative problem-solving, deliberative dialogue,
teamwork).
• Research on outcomes or evidence related to global knowledge and skills is
particularly thin
• Though there is strong evidence linking civic learning experiences with improved
rates of retention and completion, greater evidence is needed on the ways in which
variations in civic learning (i.e. forms of practice and levels of intensity) impact
measures of student success.
- Excerpted from “A Brief Review of the Evidence on Civic Learning in
Higher Education, Finley, 2011

17. WHAT IS CIVIC LEARNING?

18. ASSESSING CIVIC LEARNING: TERMS
• Direct measures
• Indirect measures
• Constructs
• Self-reports

20. ASSESSING CIVIC LEARNING: MEASURABLE
OUTCOMES
• Knowledge, skills, and values (Saltmarsh, 2005)
• Knowledge, skills, and behaviors (Cogan, 1999)
• Knowledge, skills, dispositions, identity (Stokamer, 2011)
• Knowledge, skills, practice, and disposition (The Center
for Engaged Democracy, 2012)

21. KNOWLEDGE
• Understanding of topics related to democratic society;
systems and processes of democratic decision-making
and governance
- Civic knowledge is understanding of how democracy
operates

22. SKILLS
• Necessary abilities to understand and advocate for a
complex position
- includes intercultural communication, decision-making,
critical thinking

23. DISPOSITIONS
• Values and attitudes related to actions—the
inclination to engage in civic actions

24. CIVIC IDENTITY
• “Commitment to civic responsibility through efficacy in
action” (Stokamer, 2012, p. 51).
- Requires the incorporation of civic responsibility into
one’s self-concept

25. HOW DO WE ASSESS CIVIC LEARNING?

26. INDIRECT MEASURES
• Assume:
• that engagement in educationally-purposeful activities
result in learning outcomes
• that engagement with educationally-purposeful
campus climate supports learning outcomes

27. INDIRECT CLIMATE MEASURES
• National Survey of Student Engagement (http://nsse.iub.edu/)
• Benchmarks
• Active and Collaborative Learning
• Enriching Educational Experiences
• Supportive Campus Environment
• Global Perspectives Inventory (https://gpi.central.edu/index.cfm)
• The GPI reflects a global and holistic view of student learning and development
and the importance of the campus environment in fostering holistic student
development. The GPI measures how
• a student thinks,
• views herself as a person with a cultural heritage, and
• relates to others from other cultures, backgrounds and values.

28. GLOBAL PERSPECTIVE INVENTORY
Interpersonal -- Social Responsibility
• I think of my life in terms of giving back to society.
• I work for the rights of others.
• I put the needs of others above my own personal
wants.
• I consciously behave in terms of making a difference.
• Volunteering is not an important priority in my life.

29. PERSONAL & SOCIAL RESPONSIBILITY
INVENTORY (PSRI)
STRIVING FOR EXCELLENCE: developing
a strong work ethic and consciously doing one’s
very best in all aspects of college
CULTIVATING PERSONAL
AND ACADEMIC INTEGRITY: recognizing
and acting on a sense of honor, ranging from
honesty in relationships to principled engagement
with a formal academic honors code
CONTRIBUTING TO A LARGER
COMMUNITY: recognizing and acting on
one’s responsibility to the educational
community and the wider society, locally,
nationally, and globally
TAKING SERIOUSLY THE
PERSPECTIVES OF OTHERS: recognizing
and acting on the obligation to inform one’s
own judgment; engaging diverse and
competing perspectives as a resource for
learning, citizenship, and work
DEVELOPING COMPETENCE IN
ETHICAL AND MORAL REASONING
AND ACTION: developing ethical and moral
reasoning in ways that incorporate the other
four responsibilities; using such reasoning in
learning and in life
http://www.psri.hs.iastate.edu/

30. PERSONAL & SOCIAL RESPONSIBILITY
INVENTORY (PSRI)
Striving for Excellence
• Overall Climate for Excellence**
• Motivation to Develop a Strong Work
Ethic
• Communicating Expectations about
Excellence**
• Developing a Strong Work Ethic
Cultivating Academic Integrity
• General Climate for Academic
Integrity**
• Faculty Roles in Academic
Integrity**
• Developing Academic Integrity
Contributing to a Larger Community
• General Climate for Contributing to a
Larger Community**
• Advocating for Contributing to a
Larger Community**
• Developing a Commitment to
Contributing to a Larger Community
Taking Seriously the Perspectives of Others
• General Climate for Perspective
Taking**
• Advocating for Perspective Taking**
• Developing Perspective Taking
Refining Ethical and Moral Reasoning
• General Climate for Ethical and
Moral Reasoning**
• Sources of Support for Ethical and
Moral Reasoning**
**Indicates a factor score that is comparable to professionals’factor score. The factor is comprised of exact (or
analogous) items as the students’factor.

31. DIRECT MEASURES
Attempt to measure (directly) the outcomes
believed to be associated/comprise the construct of
“civic competency” or “civic learning.”

32. CIVIC ATTITUDES & SKILLS QUESTIONNAIRE
(MOELY ET AL., 2002)
Factor 1: Civic Action
I plan to do some volunteer work
I am committed to making a positive
difference.
Factor 2: Interpersonal and problem-
solving skills
I can listen to other people’s opinions.
I can successfully resolve conflicts with others.
I try to place myself in the place of others in
trying to assess their current situation.
I tend to solve problems by talking them out.
Factor 3: Political Awareness
I am aware of current events.
I plan to be involved in the political process.
Factor 4: Leadership Skills
I am a good leader.
I have the ability to lead a group of people.
I feel I can make a difference in the world.
Factor 5: Social Justice Attitudes
People are poor because they choose to be poor.
(Reverse scored)
In order for problems to be solved, we need to
change public policy.
We need to change people’s attitudes in order to
solve social problems.
Factor 6: Diversity Attitudes
I find it difficult to relate to people from a different
race or culture. (Reverse scored)
I enjoy meeting people who come from
backgrounds very different from my own.
Cultural diversity within a group makes the group
more interesting and effective.

33. ACTIVISM ORIENTATION SCALE
(CORNING & MYERS, 2002)
Conventional Activism
• Display a poster or bumper sticker with a
political message?
• Invite a friend to attend a meeting of a
political organization or event?
• Purchase a poster, t-shirt, etc. that endorses a
political point of view?
• Serve as an officer in a political
organization?
• Attend an informational meeting of a
political group?
• Organize a political event (e.g., talk, support
group, march)?
• Give a lecture or talk about a social or
political issue?
• Go out of your way to collect information on
a social or political issue?
• Campaign door-to-door for a political
candidate?
High-Risk Activism
• Engage in a political activity in which
you knew you would be arrested?
• Engage in a physical confrontation at a
political rally
• Engage in a political activity in which
you feared that some of your
possessions would be damaged?
• Engage in an illegal act as part of a
political protest?
• Engage in a political activity in which
you suspect there would be a
confrontation with the police or
possible arrest?
• Block access to a building or public
area with your body?

34. EXPECTED POLITICAL VOICE MEASURE
(SEIDER, GILLMOR, & RABINOWIC, 2012)
 Contact or visit a public official to express your opinion
 Contact a newspaper or magazine to express your opinion on an issue
 Call in to a radio or television talk show to express your opinion
 Write a comment on a political blog
 Take part in a protest, march, or demonstration
 Sign a written or email petition about a political or social issue
 NOT buy something because you dislike the conditions under which it was made
 Go door to door to campaign for a political cause
 Work together with others to solve a problem in the community where you live
 Run for a political position at the local, state, or national level

35. COMMUNITY SERVICE SELF-EFFICACY SCALE
(REEB ET AL., 2010)
 If I choose to participate in community service in the future, I will be able to make a meaningful
contribution.
 In the future, I will be able to find community service opportunities which are relevant to my
interests and abilities.
 I am confident that, through community service, I can help in promoting social justice.
 I am confident that, through community service, I can make a different in my community.
 I am confident that I can help individuals in need by participating in community service activities.
 I am confident that, in future community service activities, I will be able to interact with relevant
professionals in ways that are meaningful and effective.
 I am confident that, through community service, I can help in promoting equal opportunity for
citizens.
 Through community service, I can apply my knowledge in ways that solve “real-life” problems.
 By participating in community service, I can help people to help themselves.
 I am confident that I will participate in community service activities in the future.

36. OPEN ENDED QUESTIONS/NARRATIVE DATA
• Open ended question:
What experiences at this campus have helped you to further develop your
work ethic? Please describe 1-2 examples below.
• Narrative Data
• Just the sheer amount of work that I need to do to stay afloat in a class. And
the result of high grades, especially at such a rigorous institution, is
rewarding
• Work-load: sink or swim. Without a strong work ethic, you will truly drown.
• Well, the academic standards are such that to "get by" you have to have at
least a DECENT work ethic or be very passionate about your subject matter.
While I've struggled with procrastination, small class sizes and the
opportunity to work closely with professors has enabled me to grow as a
student and a person.

37. PURPOSES OF NARRATIVE DATA
Narrative data can…
1. Provide a Comprehensive Understanding of Civic Engagement
• Insight into Quantitative Measures
• Overall climate
• Recent situations
2. Allow for Engagement in Civic Literacy
• Key Recommendation for Higher Ed: Make CIVIC LITERACY a core
expectation for all students.
• Can students articulate their experiences related to civic learning?
3. Inform future assessment and practice

38. I am heavily involved in non-academic activities
(service organizations, academic organizations,
greek life, etc) and have taken on a number of
leadership roles during my time here. It is my
responsibility to be on top of my work so that I can
perform my leadership responsibilities for each
organization and also complete my coursework in
a timely fashion.
I am captain of the Mock Trial
team. My experience with
this organization has honed
me into being an even more
responsible, organized, and
motivated individual.
I am standards chair for my
sorority. As such, it is my job
to promote good academic
and social behavior among
our members.
Learning about events
like the community
garden and seeing all the
opportunities that our
institution offers have
helped me deepen my
commitment to
contributing to the
community.
Being a member of a student organization dedicated to non-
partisan discussion of politics has helped me understand the
need for civic engagement.
Our school has a culture of taking
on many projects, clubs,
leadership positions, volunteer
experiences, jobs, etc. but this is
different than work ethic. Students
here collect titles, but don't
necessarily spend the time to be
efficient or good at any of these.

39. I feel they are trying to put a gag on faculty
with unpopular opinions! When the
president received a vote of no confidence
from the faculty, instead of firing him, the
board of trustees helped the president to
commit a retribution firing of the only
untenured faculty member from the
department involved in uncovering the
administration's violation of shared
governance. WHAT A FIASCO! Complete lack
of integrity…This is why the past three board
of trustee meetings have been packed with
student protestors, but the board refuses to
respond to student demands for justice.
Unfortunately the
president of the school is
involved in many
despicable legal battles
against academic freedom
of the faculty. This has
deepened my
commitment to justice,
especially when big
institutions like this school
abuse their power.
I find it extremely disheartening and
disturbing that the President has
waged a campaign AGAINST faculty
free speech and shared governance.
The department and faculty do,
however, uphold, support and
educate us about academic integrity
in spite of the administration's abuse.
Students are actively discouraged from speaking any
view that is contrary to the senior administrations’.
When faculty try to advocate for students they are
retaliated against by the administration by being
denied chairships, tenure, and other faculty
esources. These denials have resulted in costly
litigation by the university, which drains scarce
financial resources.
Seeing the immoral
actions of the
President and the
Board of Trustees
with regard to
tenure and the
Board's inattention
to the academic
health and needs of
the University has
discouraged many
of us.

40. USING NARRATIVE DATA
• Read
• Don’t let the data sit on the shelf
• Analyze
• Thematic
• Cross-case
• Present
• How can we present narrative data in a way that is concise, meaningful. and
easy to share?

41. INFORMING FUTURE ASSESSMENT &
PRACTICE
• Using narrative data to guide additional assessment & practice
• Focus Groups
• Interviews
• Observations
• Texts & Websites

42. CIVIC LEARNING: TOOLS AND RESOURCES
Literature
• A Crucible Moment: College Learning and Democracy’s Future:
http://www.aacu.org/civic_learning/crucible/index.cfm
• A Brief Review of the Evidence on Civic Learning in Higher Education:
http://www.aacu.org/civic_learning/crucible/documents/CivicOutcomesBrief.pdf
• Promising Practices for Personal and Social Responsibility:
https://secure.aacu.org/source/Orders/index.cfm
• Making Progress: What We Know About the Achievement of Liberal Education
Outcomes: https://secure.aacu.org/source/Orders/index.cfm
• Diversity and Democracy, Fall 2012 Issue – Civic Engagement and Student Success: A
Resonant Relationship: http://www.diversityweb.org/diversitydemocracy/
Assessment Tools and Inventories
• Personal and Social Responsibility Inventory: http://www.psri.hs.iastate.edu/
• Global Perspective Inventory: https://gpi.central.edu/index.cfm
• AAC&U VALUE rubrics: http://www.aacu.org/value/index.cfm

43. FOR FURTHER INFORMATION ON THE PERSONAL AND SOCIAL RESPONSIBILITY
INVENTORY (PSRI), CONTACT
Robert Reason
PSRI Director / Associate Professor of Education
Iowa State University
rreason@iastate.edu http://www.psri.hs.iastate.edu/