Presentation about University of Michigan Pilot on Digital Badges for Co-Curricular Learning pilot. Presented to Mozilla Open Badges Research Community Call on May 21, 2014 (Notes available here: https://openbadges.etherpad.mozilla.org/research-calls-May21) Summary: This pilot project studied the recognition of undergraduate engineering students' co-curricular learning experiences using digital badges in one semester, Winter 2014. Using a web environment, students described and reflected upon their experiences in categories of competencies that leaders in industry and education have identified when evaluating the future needs of the global STEM workforce. The objectives of the project were to (1) deploy an online system that served to standardize the recognition of engineering co-curricular learning; (2) understand different motivations students have for seeking recognition for their co-curricular learning and whether digital badges satisfy those motivations; (3) maximize the perceived value of digital badges while minimizing undue burden on the student to collect evidence of their co-curricular learning; (4) examine how students discuss, discover, and share digital badges and their supporting evidence, with their peers and with potential employers; and (5) disseminate findings that inform the use of digital badges designed to represent the wide variety of skills that students can acquire through co-curricular opportunities in higher education.

1. Using !
Digital
Badges
to Recognize
Co-Curricular Learning
Steven Lonn, Ph.D.
Assistant Director, USE Lab & Library Learning Analytics Specialist
University of Michigan
Twitter: @stevelonn
1
http://www.wiscnews.com/news/local/article_76df7d0c-38d6-11e2-97ca-001a4bcf887a.html

2. Acknowledgements
2
• UM Third Century Program
• Cinda-Sue Davis, Darryl Koch, Debbie
Taylor, Joanna Woods
• Caitlin Holman
• Mallory Anderson
• Rob Soltesz
• Digital Badges Advisory Group
• Library Learning Technologies Incubation
Group (LTIG)
• Members of the USE Lab

3. http://www.ukces.org.uk/assets/ukces/images/
publications/ukces-stem-infographic-aw.jpg
http://www.adeccousa.com/engineering-and-technical/employer-
services/resource-center/PublishingImages/stem-jobs-infographic.png
http://www.thecampuscompanion.com/wp-content/uploads/2011/09/Microsoft-Stem-Infographic.jpg
STEM
Workforce Shortage
3

4. 4
Concerns Extend
Beyond Technical Skills

5. 5
• Successful college transition
program - beginning 7th year
• Overarching goal: increase
number of students pursing
and graduating with STEM
degrees.
• Increase diversity
• Provide students with
competencies acquired
outside of the
traditional classroom setting

6. 6
Students often have problems articulating their
skills and experiences with recruiters / employers
http://www.engin.umich.edu/college/departments/images/mpowered-startup-career-fair
http://www.engin.umich.edu/college/about/images/campus-resources/college-resources/engineering-career-resource-center/display-file/image/career-fair-013113.jpg

7. 7
Stephen JB Thomas from the Noun Project
How can students gain
recognition and connect
their co-curricular
learning with their
curricular experiences?

8. 7
Stephen JB Thomas from the Noun Project
How can students gain
recognition and connect
their co-curricular
learning with their
curricular experiences?
How can students
communicate their
learning with
employers?

9. 8
Initial funding:
Transforming Learning for a
Third Century (TLTC)
Quick Wins Grant
!
Awarded September 2013

10. Employers’ Council Brainstorm on
Desired Skills Beyond the Transcript
9
Subcommittee on Undergraduate Retention
• International Programs
• Leadership
• Project management skills
• Business Acumen
• Conflict Management/
Resolution
• Foreign Language
• GPA Progression
• Communication Skills
(written/ Spoken)
• Listening
• Negotiation skills
• Body Language (Cultural)
• Technical Abilities
• Social Skills
• Internships
• Intense Research projects
• Solid Design Project
• Team Skills
• Personal Branding
• Social Media
• Attendance
• Integrity
• Communication/
Acceptability/
Transparency/ Speed
(CAT)
• Meeting Commitments on
time
• Proficient Problem solving
capability
• Dress for Success
• Adaptability to different
cultures
• Confidence but not
arrogance

11. 10
Student
focus
groups

12. 11

13. 11
• Community
Service
• Cross-Cultural
Experiences
• Entrepreneurial
Mindset
• Ethics
• Intellectual
Curiosity
• Leadership
• Professional
Development
• Science &
Engineering
Research

14. 12
• Intellectual
Curiosity
• Leadership
• Professional
Development
• Science &
Engineering
Research
• Community
Service
• Cross-Cultural
Experiences
• Entrepreneurial
Mindset
• Ethics

15. 13
https://tubarks.files.wordpress.com/2012/04/blooms1.png
Beginner
Intermediate
Advanced
Bloom’s
Taxonomy
(revised)

16. 14
Beginner - Community Service
DESCRIPTION:!
Exposure to concepts of community service
!
CRITERIA:!
Attend three (3) different types of short-term community service events / opportunities or one (1) long-term community
service event / opportunity, is able to remember them, and has demonstrated that they understand the relevant themes
and connections
!
EXAMPLES: !
Detroit Center, Michigan Outreach, Global Citizenship Club, Net Impact, Edward Ginsburg Center, Project SERVE,
Project Community
!
EVIDENCE:!
1. Some evidence of attendance or participation for all three (3) activities (picture of you there, note from organizer,
screenshot, name on attendance sheet, etc.).
2. Description of the events / opportunities. Experiences should total at least 3 different days / events of community
service exposure.
3. Reflection on meaning, themes, connections, and items in reflection prompt (reflection should be brief, but
substantive).
!
REFLECTION PROMPT:!
Using examples from your experiences described above:
Community Service - what it is, what it means to you, how is it demonstrated & recognized, importance, relationship to
Engineering learning (why is it important to serve the community as an Engineer?)
!
Reviewer Notes (not seen by students)!
• Recognition of importance.
• Be able to identify community service opportunities versus work opportunities.
• What are the different types of community service opportunities?
• Which ones do you gravitate towards and why?
• What community needs are served by these opportunities?
• What affect does volunteering have on you & others?

17. 15
DESCRIPTION:!
Analysis and Application of community service concepts
!
CRITERIA:!
The earner is able to distinguish and describe various aspects of community service and demonstrate community service
organizational and troubleshooting skills within one (1) co-curricular context.
!
EXAMPLE: !
Become active in one volunteer organization (e.g., Project SERVE), gather information and help select required tasks for
identified problems, apply Engineering skills to community service organization needs and issues.
!
EVIDENCE:!
1. A short description of the context of your community service work.
2. A written description, video, or other form of analysis of the concepts surrounding community service.
• This analysis should also include a reflection on Engineering skills used in your community service work using the
questions in the reflection prompt.
!
REFLECTION PROMPT:!
Analysis of Community Service Concepts:
Using examples from your experiences described above: What is the goal of the community service organization? Are they
achieving that goal? How could that goal be expanded or the effort better realized? What is required to achieve that goal in
terms of people and resources? What are the necessary steps?
!
Application of Engineering Skills in Community Service:
Using examples from your experiences described above: How have you used your skills as an Engineer in your community
service work? Do you require additional skills? Which ones and why? How has your community service work informed your
future as an Engineer? Why?
!
Reviewer Notes (not seen by students)!
• Recognition and application of community volunteerism and application of Engineering skills in community contexts.
• Why did you choose to volunteer with this organization?
• What have you learned and what has the organization learned from you?
• How have your skills as an Engineer informed your community service work?
• What problems do you think community service can help address? Why should engineers participate in that effort?
Intermediate - Community Service

18. 16
DESCRIPTION:!
Evaluation and Creation of Community Service Concepts
!
CRITERIA:!
The earner is able to make informed judgments about community service ideas and defend the decisions by summarizing the
various options. Through the course of participating and/or helping to organize a volunteer organization, the earner generates,
reorganizes, and/or designs new approaches related to the organizational goals.
!
EXAMPLE: !
As the co-organizer of the Michigan Relay for Life event, the earner visited several other Relay for Life events and generated a
list of ideas that the Michigan event could build upon. The earner worked with the other co-organizers to prioritize, assign, and
carry out the generated ideas, greatly improving the quality of the annual event for all participants.
!
EVIDENCE:!
1. A short description of the context of your community service experience(s) and what aspects of the work (e.g., planning,
fundraising, etc.) you helped organize
2. An email or letter from one (or more) of your community service organization peers explaining how your participation was
beneficial and critical to the success of the organization, citing 2-3 concrete examples.
3. A written description, video, or other form of evaluation of your community service skills and experiences, answering all of
the questions in the reflection prompt in depth.
!
REFLECTION PROMPT:!
Evaluation of Community Service Skills:
What decisions have you made in your community service work? Why were these the most effective decisions at the time? What
were the alternatives? Can you explain and justify your reasoning?
!
Creation of New Community Service Skills:
Can you think of a new or modified structure, process, design, etc. that you initiated in your community service work? Why was
this change needed? Was it a success? Why / why not? How did your team members react? Why?
!
Reviewer Notes (not seen by students)!
• Compare and contrast community service approaches, goals, and decisions.
• Be able to describe the decision-making process and defend decisions made.
• With hindsight, explain how future decisions will be informed.
• Demonstration of the ability to identify, delegate, and carryout several aspects of community service work including planning,
funding, advertising, and logistical management.
Advanced - Community Service

19. 17
Student naming competition for
badge name in December 2013

20. 18
Winter 2014 pilot with 58
engineering freshman in
M-STEM Academies
!
Pre & Post Surveys
Focus Groups
Badge Evidence
System Use Data

21. Prior Familiarity with
Digital Badges
19
How familiar were you with
digital badges?
Somewhat
11%
A little
21%
Not at All
68%
How likely are you to apply for
a digital badge this term?
Very Likely
16%
Somewhat Likely
40%
Neither Likely nor Unlikely
32%
Somewhat Not Likely
9%
Not at All
4%

22. 20
Student designed by Lemon Liu from the Noun Project
Checklist designed by Rafael Farias Leão from the Noun Project
Selfie designed by Claire Jones from the Noun Project
Text designed by Julien Miclo from the Noun Project
Blog designed by Oriol Carbonell from the Noun Project
Video Player designed by Felipe Santana from the Noun Project
Outbox designed by Simple Icons from the Noun Project
Worker designed by James Fenton from the Noun Project
Profile designed by Ryan Beck from the Noun Project
earner
views
criteria
apply
for badge?
decides reflects
creates
evidence
provides evidence
to reviewer
does
evidence
meet
criteria?
review & resubmit
no
yes
issues
badge
earner decides
which badges to
share via Mozilla
Open Badges
Backpack
earner
displays
badges

23. Mblem System Requirements
21
• Review badge criteria for all levels
• Apply - Review - Return - Resubmit processes
• “Hidden” - Awarded badges
• All earned badges & evidence viewable by all others
in the program
• Promote sharing, recognition, and variety of
opportunities
• Reviewer and Earner email notifications
• Integration with Single Sign-on
• Integration with OpenBadges

24. 22
Mblem online
system launched
in January 2014
All levels,
awardees (and
evidence), and
number of
applicants visible
to all within the
learning
community

25. 23
Criteria
includes
examples,
required
evidence,
and reflection
prompts

26. 24
Everyone
within the
program can
view earned
badges &
evidence

27. Special Badges for
M-STEM Academies
25
Recognized as a representative of the M-STEM Academies!
!
The earner is enthusiastic and has evident passion about M-
STEM and the University of Michigan, and also presents
themselves well to their audience. Furthermore, the earner
attends M-STEM meetings when not required to do so, volunteers
to speak to corporate representatives and/or information
sessions, and remains in contact with M-STEM Coaches.
Recognized as making a lasting contribution to the M-STEM
Academies!
!
The earner makes a lasting contribution to the M-STEM
community. The M-STEM “Elders” deem this contribution a
substantial benefit to the entire M-STEM Academies that will
enhance the living and/or learning environment for all
participants. Evidence: A sustained commitment to the M-STEM
community that cumulates in a program, process, procedure, or
other significant contribution or benefaction.

28. 26
Export Examples
Mozilla
OpenBadges
Backpack

29. 26
Export Examples
Mozilla
OpenBadges
Backpack
Linked In
ePortfolio
Digital Resume
Facebook

30. Overall Statistics
27
Category Number
AwardedCommunity Service 7
Cross-Cultural Experiences 4
Entrepreneurial Mindset 1
Ethics 2
Intellectual Curiosity 1
Leadership 4
Professional Development 5
Science & Engineering
Research
3
M-STEM Family Legacy 3
M-STEM Representative 8
27 “Applied for”
Badges Awarded
(13 students)
11 “Special”
Badges Awarded
(9 students)
38 Total!
Badges Awarded!
(19 students)

31. Examples of Evidence
28

32. Why Didn’t More Students
Apply for Mblem Badges?
30
Category # Agree or Strongly Agree
Not enough time due to coursework 21
Did not have pictures or other evidence 19
I do not think that badges will help me get a job 19
I do not see the value of digital badges 17
Did not have time due to co-curricular activities 15
Not enough time due to work and/or family 13
Could not remember activities to qualify 9

33. How useful do you think badges will be in
supporting your future conversations with
potential employers?
31
Very Useful
14%
Somewhat Useful
23%
Neither
17%
Somewhat Not Useful
17%
Not Useful at All
29%
Didn’t Earn a Badge
Somewhat Useful
47%
Neither
26%
Somewhat Not Useful
5%
Not Useful at All
21%
Earned a Badge

34. How useful do you think badges will be in
supporting your future conversations with
potential employers?
32
Mblems are not universally
known so it may be time
consuming to explain what
it is and then talk about how
you achieved it.
These badges will allow for a
more clear and detailed
discussion of accomplishments,
backed by the review and
approval of M-STEM.

35. 33
Digital Badges and the
Value Proposition
!
• Implications for motivation
• Self-determination theory (Deci & Ryan, 2002)
• Autonomy
• Competence
• Relatedness
!
• What is the value proposition for employers?
• Distilling badges
• The casual glance vs. detailed review
!
!

36. 34
Future Plans:
!
Expand to all M-STEM Academies (Engineering &
LSA) students in
Fall 2014 (approx. 450 students)
!
Additional Potential Pilots:
UROP, Housing Digital Literacy Project, Digital
Media Commons, etc.
Partnerships with other universities:
Purdue, Penn State, Michigan State, others