Listen to session audio while manually viewing slides at: http://ofcoursesonline.com/?p=408. Presentation w/ Rudy McDaniel and Joseph Fanfarelli at 2014 Information Fluency Conference.
Stinkin' Badges: Why We Need 'Em and How to Use 'Em
1. Stinking Badges: Why We Need
Em’ and How to Use Em’
Rudy McDaniel, Joseph Fanfarelli, and Kelvin Thompson
Information Fluency Conference
University of Central Florida
February 27, 2014
2. Outline
•
•
•
•
Overview
Four Quick Case Studies of Badging Projects
Toward a Badge Design Taxonomy
Discussion / Q&A
• Note: these slides can be downloaded from:
http://goo.gl/ezz3DV or
http://tinyurl.com/badgesrock
3. Achievements and Badges
• Achievements, or earned tokens of
accomplishment, often encourage players to spend more
time within digital systems (esp. videogames) and to alter
their playing habits in order to unlock particular types of
challenges (e.g., find every coin in a given area or unlock a
particular puzzle within a certain amount of time).
• Badges, or visible markers of achievement, have now
made the transition from entertainment media to other
forms of scholarship and pedagogy, particularly in online
learning environments (Jindal, 2011; Bruckman, 2004;
Lindgren & McDaniel, 2011; Lindgren, McDaniel, &
Friskics, 2011).
6. How Do Badges Work?
• Badge = Task-reward system.
– Task - Can present a task to
complete.
– Reward – Can also serve as a reward
for completing the task.
• Rewards for completing goals can
be
– Internal to the system (e.g. Points).
– External to the system (e.g. Free or
discounted “stuff”).
– The badge, itself.
7. Badges Are Gaining Widespread Attention
• A few familiar names that are actively taking part in
badging:
urdue
8. What Makes Them So Interesting?
• Badges can serve as:
–
–
–
–
–
Goal setters
Motivators
Inspiration to Explore
Creativity Boosters
Progress Trackers
• Connect Badge Criteria to Course Objectives
• Expected vs. Unexpected Badges
– Foster different goals.
– Expected may help in achieving a specific purpose, while
Unexpected may hurt the purpose (and vice versa).
9. Implications for Education
• Motivate students to do their best work.
– Or additional work.
• Help students set goals for clearer routes to
success.
• More precisely and creatively track progress, in
comparison to the final grade in a course.
• Encourage students to implement creative
thinking to discover unexpected achievements.
• We will now discuss some of our
implementations.
10. Dumb(?) Things I’ve Done with Badges
•
•
•
•
Badges seen only by recipient
Badges not easily shareable
Badges as “back-handed compliments”
Badges for required activities
27. Course Structure
• 30 different modules to
choose from at 10 different
points in the semester.
• A back story involving a media
mogul recruiting new
students (the “dream job”
scenario) is released via four
different animations
throughout the course.
29. Assessment Approach
• Implementation of course technology and
curriculum with ~100 students in Fall 2010
and ~200 students in Fall 2011 was
successful
• Badges were added in Fall 2011 version of
the course
32. Assessing the Effects of Badges
• Several components of assessment including
student project analysis, focus groups, and
comparisons to other courses
• Focus here is on engagement and learning
surveys that asked specifically about the badges
• 206 students completed at least one survey
• 127 completed both pre- and post-surveys
33. Student Surveys
To what extent did you believe that having the ability to choose which module to
take throughout the course was a positive feature that helped you to learn?
Extremely
Positive
Mostly
Positive
Somewhat
Positive
Not at all
Positive
I found myself working harder on assignments/projects in order to acquire
achievements.
Strongly Disagree
1
2
3
4
5
6
7
Strongly Agree
In the last 8 weeks, how often have you discussed ideas from this course outside of
class?
Very Often
Often
Sometimes
Never
• Some questions adapted from the 2010 NSSE
36. Attitudes - Badges
Felt Achievement System Had
Positive Impact on Course
Motivated by Other Students
Receiving Achievements
7 = Strongly Agree
7 = Strongly Agree
7
7
6
6
5
5
4
4
3
3
2
2
1
1
0
0
Males
Females
Males
Females
37. Attitudes - Badges
Variable 1
Felt Achievement System
Was Positive (1 to 7)
Worked Harder To Receive
Achievements
(1 to 7)
Worked Harder To Receive
Achievements
(1 to 7)
Seeing Others Get
Achievements Was
Motivating (1 to 7)
Pearson
Correlation (r)
Significance
(p)
Discussed Ideas Outside of
Class (1 to 7)
.175
.040*
Commented On Other
Students’ Work (1 to 7)
.217
.010*
Amount Of Time Spent
Collaborating With Other
Students
.242
.004**
Total Number Hours Spend On
Course Per Week
.158
.064
Variable 2
• Post-Survey: Positive feelings about badge system was
correlated with other positive feelings in the course
38. Assessment Summary
• Importance of “framing” the achievement system at the
outset
• Interesting gender patterns – suggests badge systems may
be a productive means of targeting female learners
39. Case Study #4: Two Current UCF Courses
• 1 Semester, 158 Undergraduates
• 2 courses: Graphic Design (99 students) and Web
Design (59 students).
– 2 sections of each.
» 1 section of each has badges.
» 1 section of each does not.
Graphic
Design 1
Graphic
Design 2
Web Design Web Design
1
2
Badges
Yes
No
Yes
No
Students
49
50
30
29
40. Current Courses: Background
• Courses are meant to prepare students for the
Adobe Certified Expert (ACE) Exam in the
related software.
– Web Design = Dreamweaver
– Graphic Design = Photoshop
• Completely web-based 16 week courses.
• Balanced emphasis on Quizzes / Exams and
Project-based assignments.
41. Current Courses: Badges
• Badges are:
– Unexpected - No list of possible badges can be
found by students.
– Private – Students cannot see the badges others
have earned.
– Both Objective and Subjective.
• Objective – Named all layers within a Photoshop
project.
• Subjective – Helped a classmate succeed
– Awarded both automatically and by the instructor.
42. Method of Award
• Checkbox in gradebook for each badge
simplifies the process
43. Viewing Badges
• Badges can be found from the course
menu, like other important course
information.
45. Current Courses: The Goals
• Improve:
– Motivation
– Engagement
– Academic Performance
• Identify:
– Can number of achievements earned be used to
predict grades in a certification preparation
course?
46. Badging Observations
• Each stakeholder determines value
o Issuer, Earner, “Observer,” (Displayer)
• Potential value in each phase of badging:
o
Underlying data/record
o
Notification email
o
Claiming (“Save and Share”)
o
Making public
o
Linking to specific badges
47. Unanswered Questions to Ponder
• Why do badges appeal to some but not
others?
• Does badging really engage the unengaged?
• What is the right balance of automation and
personal attention for course badging?
• What is the relationship between badges
and formal credentials?
• What is the right balance of curricular and
co-curricular badging at an institution?
48. Toward a Taxonomy for Badge Design
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Subject (e.g., information literacy; educational tech; interdisciplinary; etc.)
Type (e.g., academic; professional development; etc.)
Level (e.g., undergraduate, graduate; etc.)
Tiers (e.g., single tier; multiple difficulty tiers; three cumulative tiers; etc.)
Issued By (e.g., single issuer/multiple; instructor; organization; etc.)
Scale (e.g., course-level; discipline-specific; institution-wide; public; etc.)
Population Size (i.e., to whom badges are available; e.g., 1230; 35; 217; etc.)
Badge Image Source (e.g., graphic designer; badge making template; etc.)
Platform (e.g., Purdue Passport; Credly; etc.)
Focus (e.g., core competencies; off-topic diversion/fun; secondary competencies; etc.)
Grades/Badges (e.g., badges only; badges = grades; grades & badges separate; etc.)
Fixed/Extensible (e.g., defined list of badges; new badges suggested/added on the fly; etc.)
Expected/Unexpected (e.g., published list (“a priori”); discovered Easter eggs; etc.)
Visibility (e.g., issuer; earner; bounded group (“class”); public; etc.)
Status (e.g., complete; interrupted; underway; planning; etc.)
Version 1.1
49. Thank You!
• Rudy McDaniel,
rudy@ucf.edu,
@rutang5
• Joey Fanfarelli,
joseph.fanfarelli
@ucf.edu
• Kelvin Thompson,
kelvin@ucf.edu,
@kthompso
Badge graphics courtesy of Matthew Dunn
Notas del editor
Reserved for Rudy’s Intro
Title Slide
Joey BeginThe concept of badging isn’t really new. It’s been around since at least the Napoleonic Era, when Napoleon Bonaparte issued ribbons to his military for exceptional service. We’ve also seen it in Boy and Girl Scouts for decades, and more recently, we’ve seen it in video games.
So, what is a badge? It’s actually two things in one – a task and a reward. It can present a task to be completed, it can be a reward for completing a task, or it can be both. In badging systems, we typically see three types of rewards: One type is Internal – something within the system, extra credit or points that have no value to the outside world. Next, we see external rewards – something outside of the system, discounts or free “stuff.” Finally, the badge may serve as the reward itself, similar to a trophy.
But badges have gained an unparalleled amount of attention in the past few years. NASA is looking to increase STEM education in grades 4-12, by creating badges geared toward teamwork and robotics. Purdue is currently beta testing its badging system as a method for showcasing a greater breadth of its students skills, moving beyond a simple complete/incomplete framework for each class. Other major players include the Smithsonian Institution, UC-Davis and the MacArthur Foundation.
But what makes them so interesting? When properly designed and implemented, badges serve a variety of beneficial purposes. They can serve as….It’s worth noting that achievements can be expected or unexpected, and this distinction is critical. Knowing which achievements exist can facilitate goal setting, but can reduce motivation. Unexpected achievements can increase motivation, while removing the ability to aid in goal setting. Careful design is essential when developing a badging system.
Joey EndSo what does this mean for the educational setting? Badges may motivate students to do their best work (or maybe even additional work!). They can help students be successful by better defining the path to success. They can more precisely track progress in comparison to the final grade in the course. With badges, you can state that a student implemented hydroponics, rather than simply saying a student received an “A” in Agriculture. Also, by implementing unexpected badges, we can encourage students to implement creative thinking in an attempt to discover the achievements. We will now discuss our experiences in teaching three instances of University courses, two that have been completed, and one that is currently in progress.
Kelvin Begin
Kelvin End
Joey BeginWe’re currently using badges in two university courses. Here is the breakdown. 158 Undergraduates are being included over one semester. There are two courses, graphic design with 99 students, and web design with 59. Two sections of each course, for a total of four sections, and one section of each course has badges.
The courses are meant to prepare students for the Adobe Certified Expert Exam, while teaching them the fundamentals of web and graphic design. They are completely web based and taught over the traditional 16 weeks. The courses have a balanced emphasis on quizzes and exams, and project-based assignments.
In the course, the badges are unexpected. The students do not know which badges exist. They are private. Students cannot see the badges that others have earned.They are both objective and subjective. For example…Several are awarded automatically, when a student attains a particular grade on a quiz or exam. Others are awarded by the instructor.
The interface simplifies the instructor award process. It is built directly into the gradebook. Since students can see there grades, the names of the badges had to be codified, broken down into abbreviations, as you see above. To award an achievement, the instructor clicks to place a checkmark in the appropriate cell. When they system next updates, the achievement will be awarded to the student, who will receive an e-mail.
Badges can be found right from the course menu, where all of the other important course information resides. This is the instructor view. As you can see, I haven’t earned any badges. For a student, this would be populated with the badges they have earned. From the instructor view, I can see every achievement that has been given, when it was given, and who it was given to.
When I click one of the badges, I can see the full description. This functionality is also present on the student side, allowing them to know why they achieved the badge. This description is also present in the notification e-mail they receive.
Joey EndOur current goals with the course are to improve motivation, engagement, and academic performance. We are also looking to see if the number of achievements earned can be used to predict grades in a certification preparation course. Concluding the semester, we will undergo data analysis to see how well these goals were achieved.