Science of Social Learning (Learning Solutions 2016)

Sharin’ Ain’t “Social”
The Science of Social Learning
Greg Bybee
VP, Learning Solutions, NovoEd
greg@novoed.com
March 17, 2016

Introduction
Principles of Andragogy
Quick NovoEd Introduction
Principles of Social Learning
Research on Team Learning
Other Best Practices
Closing Thoughts
Agenda

I’m an education technologist
● McKinsey & Company Consultant
● Coursera, Product Manager
● Renren, Education Business Development
● NewSchools Venture Fund, EdTech Fellow
● NovoEd, Learning Products (2013-present)
At NovoEd, I lead our learning solutions team
● Learning experience design (and ISD)
● Learning solution strategy consulting
● Course and program operations and facilitation
● Customer success and support
I have a diverse background in training
● MA-Ed & MBA @ Stanford
● Chair, Student Achievement @ Leadership High
● Instructor @ 辰熙中英文学校 (China)
● School Building @ ILAE in Ethiopia
● Education Market Consultant and Volunteer
Hi, I’m Greg
Nice to meet
you, too!

Group Activity!!
● Form groups of 4-8.
● Answer the question: “what
makes for an effective adult
learning experience”
○ Focus not on outcomes of the
learning, but the definition of
the learning process
○ You’ll have 8 minutes.
● Add your ideas as text here:
http://padlet.com/novoed/ls16
Weren’t you
supposed to
tell us that?

Source: Learning Solutions Session, March 17, 2016

Social collaborative
- Metacognition, teaching
- Multiple perspectives
- Peer-to-peer adaptivity
- Felt accountability
Construction and Inquiry
- Problem-Based
- Project-Based
- Role plays, scenarios
- Case method
Engaging
- Intrinsically Motivated
- Relevant, authentic
My list
(Okay admit I took more than 8 minutes...)
Experiential
+
Coherent
Self-directed, autonomous
Mastery, goal attainment, progression
Applied, relevant, authentic, purposeful
Engaging, fun, captivating
Inquiry-based, discovered
Constructed, created, explored
Scaffolded, workshopped
Interactive, multimodal, multisensory
“Knowing, being, doing”
Accountable, measured, transparent
Practiced, reinforced, rehearsed
Leveled, ZPD, personalized, adaptive
Chunked, cognitive load, working memory
Coherent, sequenced, schematic
Curricular Design
- Sequencing, chunking
- Scaffolded, coherent

Malcolm
Knowles’
Theory of
Andragogy
Assumptions
1. Self-concept
2. Experience reservoir
3. Readiness to learn
4. Orientation to learning
5. Motivation to learn
Therefore, learning must be:
1. Self-directed
2. Experiential
3. Relevant and authentic
4. Problem-based
Sources: Knowles, M. (1984). The Adult Learner: A Neglected Species. &
Kearsley, G. (2010). Andragogy: The theory Into practice database.

Source: Pappas, C. (2013). “The Adult Learning Theory.” eLearning Industry. Online.

But this training
is costly...
Airfare – Average of $800 per person
Hotel – 3 nights @ $200 per night $600
Catered Meals – 3 days @ $150 per day = $450
Labor – $500 per day (assuming $125k/yr) = $1,500
Transportation – $200 per person across 3 days
Group Activity – $300 per person
Room Rental of $2500 per day x 3 = $7500
Total Investment before a facilitator = $103,750
Facilitator = $15,000 per day x 3 = $45,000
---------------------------------------------------------------------------
Total Cost Per Person: $5,800
Source: Altman, Ian (2014) How Much Does Sales Training Cost?
Forbes. Online.

...and it cannot
scale across the
organization
Companies spend 78% more on Executives than
Emerging Leaders (and 360% more per Leader)
Source: Bersin by Deloitte (2014) Leadership Development Factbook
2014: Benchmarks and Trends in U.S. Leadership Development.

Online learning offered a solution, but didn’t deliver

My Background
Shameless Plug
Closing Thoughts
Agenda

Solutions for learning
experience design
and implementation
Online platform for
experiential learning
NovoEd
develops talent
online.

We power the best talent development programs

Principles of
NovoEd all
learning
Intuitive, User Friendly
Experience Orientation
Pervasive and Social Learning
Data-Driven, Evidence-Based

These drive
20x higher
engagement
CompletionRate
2%
44%
Traditional Social
Stanford Professor Chuck Eesley conducted this
research in 2014 based on data from Technology
Entrepreneurship (12/13) with 26,248 students.
50%
Figure 1. Completion Rate By Social Model in MOOCs

Learners are
more engaged
when they learn
together. 68
NPS
50-50
0
95%+
User Satisfaction
60+
Net Promoter Score
Source: Survey of NovoEd learners (January 2016)

My Background
Quick NovoEd Introduction
Closing Thoughts
Agenda

What is social
learning?
The use of peer-to-peer (social) interaction
as a pedagogical technique to drive
learning.
Examples:
● 1:1 tutoring
● Reciprocal Teaching
● Think-Pair-share
● Jigsaw
● Group work
● Collaborative brainstorming
Source: I made this up last night after talking to another speaker.

Social learning
is more than
(just) sharing.
Communities visible to
themselves with real identity
Peer-to-peer feedback, coaching,
and mentorship
Collaboration in teams on
relevant projects
Discussions with forums, and
messaging
User generated content sharing
with discovery

Why is social
learning
valuable?
Theories
● Relatedness motivation (Deci & Ryan)
● Felt accountability (Sutton & Rao)
● Social context of learning (Bandura)
● Cooperative learning (Dewey, Ross, et al)
● Social constructivism (Piaget)
● Joint inquiry (Dewey)
● Shared reflection of ELM (Kolb)
● Diversity of Learning Styles (Kolb)
● Mastery learning (Bloom)
● Community of Inquiry (Peirce, Dewey, Garrison, et al)
● Social development theory (Vygotsky)
● Truly adaptive learning (Bybee)
But you don’t have to subscribe to a single theory to
leverage the techniques.
Source: I compiled this list last night - send me any I missed!

The benefits of social learning
Communities visible to
themselves
Peer-to-peer feedback
and coaching
Collaboration in teams
on projects
Discussions with forums,
messaging
User generated content
sharing
● Social obligation, transparency, “felt accountability”
● Self-regulating and adapting
● Build networks, the new “content management system”
● Metacognition in providing feedback or mentorship
● Personalized and adaptive; “network of 1:1 tutors”
● More relevant guidance, “learning at speed of trust”
● Inquiry-driven, constructivist, PBL
● Practice, rehearsal, interactivity
● Self-directed and autonomous, right in ZPD
● Share diversity of skills, experiences, and perspectives
● Crowd sourced, rapid response time
● Relevant and authentic dialog drives connectedness
● Provides multiple representations & perspectives
● Application focused, authentic, and relevant
● Accountability and social proof

Agenda
My Background
NovoEd Introduction
Social Learning
Team Learning
Best Practices
Closing Thoughts
Impact of Teams
Felt Accountability
Team Size
Team Selection
Team Heterogeneity
Scaffolding Teamwork

Context and Methodology
Chuck Eesley
Assistant Professor,
Stanford University
● Teaching online since 2012
● Morgenthaler Faculty Fellow
● 2010 Best Dissertation Award,
Academy of Management
● Ph.D., MIT; BS, Duke University
Technology Entrepreneurship
Chuck Eesley, Stanford University
September 16, - November 17, 2013
● Eight-week course offered on
NovoEd
● Team-based, experiential
pedagogy
● 8 assignments, each a part of a
team-based project to find and
evaluate a startup idea
● Taught on NovoEd 11+ times
and at Stanford since 2004
View the course at https://novoed.
com/venture17
Multivariate regressions, descriptive
statistics, and t-tests of difference in
means
● n = 26, 248 students
● Students self-selected into
experimental groups
● Dependent Variables:
engagement and satisfaction
measures
● Independent variables: whether
the student participated in a
team, if they worked individually,
and if they had a mentor
● Control Variables: demographics,
engagement level, and others
For more information, please contact
Professor Eesley at cee@stanford.edu.
Researcher Context Methodology

16x higher
completion with
peers and
feedback
CompletionRate
21%
2%
44%
293
Individual with
Discussions
Teams Teams with
Feedback
Chuck Eesley, Stanford Professor, conducted this
50%
Figure 2. Completion Rate By Social ModelRESULTS:

5x more sign-ins
by individuals in
teams
AverageSign-InsPerCourse
39
8
44
293
Individual with
Discussions
Teams Teams with
Feedback
Chuck Eesley, Stanford Professor, conducted this
50
Figure 3. Sign-Ins Per User, By CohortRESULTS:

Agenda
My Background
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams
Felt Accountability
Team Size
Team Selection
Team Heterogeneity
Scaffolding Teamwork

Rao & Sutton (2013) suggest felt accountability
drives intrinsic motivation and engagement
Sutton and Rao discuss NovoEd
and the value of felt accountability
in Scaling Up Excellence
"[NovoEd] built in numerous clever
and easy-to-use social features to
create a peer-powered network
that would link, organize,
evaluate, and mentor students."
Huggy Rao Robert Sutton

Agenda
My Background
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams
Felt Accountability (Stanford)
Team Size (NovoEd)
Team Selection (NovoEd)
Team Heterogeneity (NovoEd)
Scaffolding Teamwork (Presidio)

8 public MOOCs with self-formed teams
on NovoEd from 2014-15
● Stanford GSB - Scaling Up
● Stanford - Tech Entrepreneurship
● Stanford - Creativity: Music
● IDEO/ACUMEN - Design Kit
● Princeton- Global History Lab
● Princeton - Making Gov't Work
● Maastricht - Project-Based
Learning
3866 teams total
Important Note: Learners self-selected
into various team sizes
Andrew Linford
Instructional Programs,
NovoEd
● Coro Fellow (2013-14)
● Instructor, Ministry of Education,
Singapore (2011-13)
● BA, Stanford University
Retrospective cohort data analysis and
descriptive statistics.
● n1
= 10, 315 students
n2
= 3,866 teams
percentage of assignments
completed
● Independent variables: team size
● Control Variables: none
● Analysis done Sept 2015
Andrew at andrewl@novoed.com

Note: Team size one is off chart (1734). Average exclude team size of 1.
Including that, average is 2.7.
Source: NovoEd Analysis (2015). 8 MOOCs, 3,866 teams, 10,315 students
Learners prefer
smaller teams,
average is 4
Figure 4. Team Size PreferencesRESULTS:

Source: NovoEd Analysis (2015). 8 MOOCs, 3,866 teams, 10,315 students
Empirically, the
optimal team
size is 7.
Figure 5. Team Size vs. Assignment Completion
PercentofAssignmentsCompleted
RESULTS:

Agenda
My Background
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Impact of Teams (Stanford)
Felt Accountability (Stanford)
Team Size (NovoEd)
Team Selection (NovoEd)
Team Heterogeneity (NovoEd)
Scaffolding Teamwork (Presidio)

3 private offerings of Presidio Institute’s
Introduction to Cross Sector
Leadership: Building Teams
420 learners total
Important Note: In two of the courses,
learners self-selected into teams (220
learners). In the other course (200
learners), learners were assigned teams
by the teaching team based on meeting
availability times.
● n1
= 420 learners
○ Learners engaging in
course-wide discussions
○ Learners chatting with
team in private work
space
○ Learners messaging other
learners
○ Learners commenting on
assignment submissions
● Control Variables: course size
● Analysis done Feb 2016
Andrew Linford
Manager, Support and Technical Ops
NovoEd
Alison Gold
Manager of Leadership Education
Course Instructor
Presidio Institute
AndrewAlison

Autonomous Team Formation Increases Engagement
RESULTS:
Source: NovoEd and Presidio Institute Analysis (2016). 3 Courses, 420 learners
PercentLearnersEngaginginSocialActivity
PercentLearnersEngaginginSocialActivity

● Focused on organic teams
formed by the students and
analyze preferences of students
when selecting a team.
● Analyzed the differences
between randomly-assigned and
self-selected teams.
● Analyzed signals to discern
which activities had the greatest
correlation with course
completion.
● n1
= 11 courses
● n2
= 24,000 active learners
Milad at m.eftekhar@gmail.com.
Milad Eftekhar
Data Science Intern, NovoEd
Co-authored with Amin Saberi and
Farnaz Ronaghi
● Former data researcher at
Microsoft and Stanford
● Ph.D., University of Toronto
● BS & MS, Computer
Engineering, Sharif University of
Technology
● Analysis conducted on 11 public
MOOCs with teams offered
summer 2014.
● Courses were four to eight weeks
long focusing on various
business topics.
● Enrollments ranged from 200 to
more than 25,000.
● Demographic data comes from
self-completed student profiles.
Data includes age, education,
gender, and location.
Working Paper: “Team Formation
Dynamics: A Study Using Online
Learning Data.”

Learners
preferred teams
with similarly
aged members.
Figure 6. Homogeneity by age in team selection
Source: Eftekhar, et al (2015) Team Formation Dynamics: A Study Using
Online Learning Data.
RESULTS:

Figure 7. Age is a homophilic preference.
Age homophily
resulted in more
successful
teams.
CumulativeDensity
Average Age Distance
RESULTS:

Figure 8. Distance is a homophilic preference, particularly
longitude due to timezones.
Individuals
prefer to join
teams in similar
time zones.
CumulativeDensity
Average Distance (km)
RESULTS:

Figure 9. Education Level is a homogenous preference
Teams preferred
and worked best
with similar
education levels
CumulativeDensity
Average Education Distance
RESULTS:

Figure 10. Skill diversity is a heterogenic preference
Successful
teams have
diverse skill
sets.
CumulativeDensity
Skill Entropy
RESULTS:

Note: Courses tended to 2/3 male, 1/3 female.
Learners prefer
gender mix,
though 2/3
identify as male
RESULTS: Figure 11. Gender mix by course

Results are consistent across all courses.
Team
preferences are
rational.
Characteristic Most Effective
SimilarAge
SimilarLocation
SimilarEducation Level
DiverseGender
DiverseSkill Set
Preference
Similar
Similar
Similar
Diverse
Diverse
RESULTS: Table 1. Team characteristic preferences and effectiveness

How did you run
your teams?
Did any group...
● Use an ice breaker?
● Assign roles (timekeeper, note
taker)?
● Divide into smaller groups?
● Google or use external sources?

● Students randomly assigned to
two cohorts / groups.
● Group 1 was given team
guidelines during week 2, with a
suggested agenda and roles for
the experience. Group 2 had no
instructions.
● The post course survey then
asked how well teams functioned
and final projects were assessed.
● n1
= 97 students
n2
= 103 students
Drew at drew@novoed.com
Drew Remiker
Instructional Programs
NovoEd
Alison Gold
Manager of Leadership Education
Course Instructor
Presidio Institute
● Course: Introduction to Cross
Sector Leadership: Building
Teams by the Presidio Institute
● Offered as private beta to
members from United Way,
Points of Light, Americorps,
Kresge Foundation, ProInspire,
WYMAN, and others
● Providing structure for team
meetings and roles will result in a
higher self-reported quality of
teamwork and completion of final
team assignment compared to
organic team development.
Researchers Context Methodology
Alison Drew

Source: Presidio Institute (2015) Introduction to Cross Sector Leadership:
Building Teams, Team Scaffolding Study. “Guidelines for First Team
Meeting”
Example of
teamwork
scaffolding
EXHIBIT:

Source: Presidio Institute (2015) Introduction to Cross Sector Leadership:
Building Teams, Team Scaffolding Study.
Collaboration
scaffolding
improved
completion and
teamwork
quality
Characteristic Group 1 Group 2
Quality of teamwork 4.03 / 5.0 3.79 / 5.0
Completed final
assignments
8 / 10 7 / 9
Figure 12. Team scaffolding results
Marginal improvement in self-reported teamwork quality
and final assignment completion
RESULTS:

Agenda
My Background
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Closing Thoughts
Notifications
Course Champions

6 public MOOCs offerings before and
after new notifications offering was
released.
● Stanford GSB - Scaling Up
Without Screwing Up
● Stanford - Tech Entrepreneurship
Part 2
● Agder - Success Unleashed
41,657 teams total
Important Note: Almost nothing
changed within the courses when
offered a second time with the
notifications feature available, although
were slightly smaller. Learners self-
selected teams in all courses.
Andrew Linford
NovoEd
● Coro Fellow (2013-14)
● Instructor, Ministry of Education,
Singapore (2011-13)
● BA, Stanford University
● n1
= 41,657 learners
● Dependent Variables: number of
learners
● Independent variables: selection
of different social indicators
● Control Variables: course size
● Analysis done March 2016

Notifications
● Personal alerts about ongoing social activity
(including responses in discussion forums),
assignment reminders, and more
● Daily Course Digests with assignment reminders
and an overview of course activity
● Easy reminders about and access to teaching team
communication

Huge Impact on
Submissions;
Positive Impact
on Other Social
Activities
RESULTS:
Source: NovoEd Analysis (2016). 6 Free Public MOOCs, 41,657 learners
Figure 13. Social Activities Pre/Post Notifications

Agenda
My Background
NovoEd Introduction
Social Learning
Research on Teams
Best Practices
Notifications
Course Champions

● Program: Philanthropy University
● September - December, 2015
● Course champions chosen from
active learners in the middle of
the program (beginning of
November)
● 7 courses, 200,291 unique
learners
Lisa Brefini
Philanthropy University TA
Course Operations Specialist
NovoEd
Andrew Linford
NovoEd
Retrospective learner and course
champion data analysis.
● 7 courses, each with 5-12 course
champions
● Comparison - course (and course
champions) social activity before
and after course champions are
selected
● Analysis performed December
2015
Researchers Context Methodology
Lisa Andrew

Course
Champions
increased
course social
activity
RESULTS:
Source: NovoEd (2015). Philanthropy University Courses from 2015.
Course
Champions
Selected
Figure 14. Weekly Discussion Activity

● That experience must be engaging, driven by
inquiry, social, and coherent.
● Adults require self-discovery and (extra)
practice to overcome preexisting schema
● Knowledge acquisition < skill development <
behavior change. Increasingly experiential.
● Online learning is no different. Scale the best
of offline learning, not the worst (lectures).
● Social techniques include discussions,
collaboration, sharing, feedback, community.
● Peer learning enables personalization,
metacognition, and motivation.
● Teams and group work is the most critical.
CLOSING THOUGHTS
Learning must
be experiential.

Learn Together.
http://novoed.com

Thank you! Please feel free to contact me!
Greg Bybee
Vice President, Learning
NovoEd
greg@novoed.com
@gregbybee
http://linkedin.com/in/gregbybee
Email me for slides or the
research briefs. I’d also love to
help you implement in your org.

Science of Social Learning (Learning Solutions 2016)

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Science of Social Learning (Learning Solutions 2016)

Similar a Science of Social Learning (Learning Solutions 2016) (20)

Último

Último (20)

Science of Social Learning (Learning Solutions 2016)