Discussion of the inversion of an Algorithms course: how it is motivated by learning theory; how the activities are organized; outcomes. This is an expanded version of an invited keynote talk for the "TechLogic" conference at the University of Hawaii at Manoa.

1. Putting Learning in its Place:
Design of an Inverted Technical
Course
Dan Suthers, Dept. of ICS

2. ICS 311 Algorithms
Understand, design and analyze the abstract
computational procedures that computer
programs are instances of
! Central in CS prerequisite structure
! Convergence of discrete math, programming
(2 semesters each prerequisite)
! Difficult, fast paced (2 chapters/week of CLRS)
! High repeat rate: graduation bottleneck
Not my area!

3. From Professing to Inverting
The transition
! Lectured to learn ...
! Predisposition: “I’m a hypocrite!”
! Carrot: course release to invert
! Opportunity: Webster 101 became available
Redesign drew on
! Prior experience teaching
! Learning Science, CSCL, TEL research
! Leslie's suggestions

4. Multidimensionality of Learning
Various theories capture multidimensionality
! Bloom's taxonomy: remember, understand,
apply, analyze, evaluate, create
! Merrill's Component Display Theory:
remember/use/find a fact/concept/
procedure/principle
Suthers' Taxonomy: Learning in Social Settings
! Agent: Individual, Group, Network/Community
! Processes: Acquisition, Intersubjective Meaning-
Making (Argumentation, Co-construction,
Negotiation), Creation of Cultural Capital,
Identity and Participation ...

5. Student Learning Objectives
! Familiarity with core concepts, notations,
issues, results
! Procedural skill in tracing algorithms, proving
properties (correctness), analyzing complexity
! Reasoning about conceptual issues and
abstracting to principles
! Creating and analyzing new algorithms for
applications
! Collaborative skills and ability to work on
teams

6. What is the place for each aspect of learning?
Technologies and associated modes of
interaction differ in affordances for activities
! What is the proper “place” for each learning
activity?
Classroom is our unique resource
! Many things can be done in a classroom:
what can be done best there?
! Important to viability of "brick & mortar"
setting (we aren't supposed to be here!)

7. Motivating the Inverted Classroom
Claim: classrooms are uniquely suited to
collaborative learning and problem solving
with support of educators and peers
Theory: Vygotskian ZPD/internalization
! Before class, self-study to prepare for the
intensive class experience
! In class, student achieves more than s/he
can alone, with assistance of instructor and
TA and in social interaction with others
! Out of class, practice to internalize what
was learned in the social setting.

8. Inverted classroom is designed to
maximize use of class by using other
technologies as the "place" for other
learning activities
Putting Learning in its Place

9. Design of Inverted Class: Overview
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

10. Design of Inverted Class: Overview
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Self Study:
Familiarity with core concepts,
notations, issues, results
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

11. Design of Inverted Class: Video Lectures

12. Design of Inverted Class: Video Lectures

13. Design of Inverted Class: Overview
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Familiarity with core concepts,
notations, issues, results
Motivate preparation for group work
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

14. Design of Inverted Class: Quizzes

15. Design of Inverted Class: Overview
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
Social Learning:
Procedural skill in tracing algorithm, analysis,
proving properties
Reasoning about conceptual issues and principles
Collaborative skills and ability to work on teams
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

16. Design of Inverted Class: Schedule
15-20 minutes:
! Solutions to prior HW, quiz
! Introduction to day's topic
45 minutes: Small group problem solving
! Problems chosen to require conceptual work
that will drive group discussion
5-10 minutes:
! Problem solutions and future plans

17. Design of Inverted Class: Webster 101

18. Design of Inverted Class: Groups
16 groups of 3-4 (2/table in Webster 101)
Random Groups for first 5 weeks
! Prevent dysfunctional relationships
(freeloaders, suckers) from sticking
! Students like chance to get to know peers
Voluntary groups thereafter: less than half opt
for this

19. Design of Inverted Class: Google Docs
Create a Google Doc for each student group
! Template makes response format clearer
and grading easier
! Multiple students can edit a Google Doc at
once, supporting collaboration
! Group solutions uploaded as PDF to
Laulima 10 minutes before end of class.

20. Design of Inverted Class:
Google Docs

21. Design of Inverted Class:
Google Docs

22. Design of Inverted
Class: Google Docs

23. Design of Inverted Class: Observations
! Writeable screen
plays important role
! Students work hard
for 0.5% of grade!
! More interaction
with students about
substantial issues
in one week than
I had in an entire semester in the lecture
model!
! … and much more than I have time for …

24. Design of Inverted Class: Homework
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
Internalizing:
Procedural skill in tracing algorithm,
analysis, proving properties
Reasoning about conceptual issues
and principles
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

25. Homework 1

26. Homework 2

27. Homework 3

28. Design of Inverted Class: Exams
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Individual proctored assessment
Review understanding and skills
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

29. Design of Inverted Class: Projects
Weekly Cycle:
! Weekend-Monday & Wednesday: video lecture,
textbook, and/or web notes
! By Tuesday & Thursday Class: take quiz before
class on this material
! In Tuesday & Thursday Class: collaborate with
peers on problems that expose essential issues,
with help of instructor and TA
! Friday-Monday: Individual homework problems
Creating and analyzing new
algorithms for applications
Semester Events:
! Exam every 5 weeks
! Three programming and analysis projects

30. Design of Inverted Class: Projects
Avoid academic hoop jumping; progress towards
skills needed in work settings
Progression of responsibility:
! “I tell you what to do”
! Find OSS for Dynamic Sets; empirically test
performance against theoretical predictions
! “You make some decisions”
! Implement the Graph ADT against an interface
specification
! “Your boss wants you to solve a problem”
! Analyze some graphs with metrics from network
analysis literature: you figure out algorithms

31. Does it work?
Anecdotally
! Students: yes!
! Instructor: yes!
By Outcomes
Fall 2012 (lecture) " Fall 2013 (inverted):
! Average % points earned: 62% " 77%
! Average % on final exam: 48% " 68%
! Ratios of students who did not pass (C or
less): 17/40 " 9/56
! 7/7 who were repeats from my fall 2011 &
2012 passed in 2013

32. Number of Women Enrolled
A curious trend
! Fall 2012: 5/40
! Fall 2013: 10/56
! Spring 2014: 12/51

33. Caveats
Faculty/TA Workload
! Lecture videos are a summer project
! Generates more material to grade
! Make sure your department understands the
workload and supports you and your TA
Student Workload
! Should be at least 90 minutes per class
! PCC approved increase to 4 credits
! The rest of the workload is "germane",
student complaints notwithstanding!

34. Infrastructure
! Tendency to get lost in their own devices
! Encourage paper: phone camera to record
! Design table/screen for groups of 2-4
! Touch screen would be nice
Need a coordinated campus-level effort to
support conversions to inverted format
! Continue building "innovative classrooms" with
faculty co-designers
! Course release and media staff support for
video production
! Increase TA allocation to early adopters

35. Mahalo!
Dan Suthers, Dept. of ICS
suthers@hawaii.edu
Lilt.ics.hawaii.edu