SlideShare una empresa de Scribd logo
1 de 49
Using Learning Analytics
Data to Understand
Project-Based Learning
The Case of the Carrot
Wizards in Globaloria-
West Virginia
ICLS 2012 Pre-Conference Data Workshop
***
Rebecca Reynolds, Ph.D.
Assistant Professor, Library & Information Science
Rutgers University School of Communication and
Information
Background Info
• The following slides provide background on
the Constructionist / design-based research
innovation, “Globaloria,” being implemented
by a NYC non-profit called the World Wide
Wokrshop, in several US states with grant and
state education department support (WV, TX,
CA, FL, NY)
Globaloria Game Design Program Learning Objectives: Cultivate the Six
Contemporary Learning Abilities (6CLAs)
Developing games in a social learning system offers students
Constructionist, participatory design and programming experiences
leading to dispositions, practices and expertise necessary for successful
engagement in today’s digital, knowledge-based economy
The Six Contemporary Learning Abilities (6CLAs):
Developing games in a social learning system offers participatory
experiences leading to practices necessary for engagement in today’s
knowledge-based economy
Globaloria is currently active
in 4 U.S. states: CA, TX, FL, WV
Globaloria-California
California, 2011/2012:
• Supported by $1M Knight
Foundation grant
• “Formal” implementation
at 2 middle schools (67
students)
• “Informal” implementation
at 2 Boys and Girls Club
community centers (38
students)
• First year of California
participation
• Potential to compare
implementation context
variable differences (time
on task; provision of grades
vs. none; linkage to core
curriculum vs. open choice
game topics, etc.)
Globaloria-TX
East Austin College
Prep, 2011/2012:
• 1 charter middle
school
• 3 grades (6th, 7th,
8th)
• ~250 students
• 3rd year of
school’s
participation
• 2 educators
• Unique
longitudinal
implementation
(all EACPA
students take
Globaloria daily,
for credit and a
grade for 3 years)
Globaloria-West Virginia2011/2012:
• 5th year of Globaloria
implementation in West Virginia
• Individual schools and teachers
range in N of participation years
• Majority of students, 1 year only
• 17 middle schools
• 50 high schools
• All in-school implementation,
daily, for credit, for grade
• Mix of elective game design
classes, and required classes
linked to the core curriculum
• Multi-level analysis and
implementation context variable
differences increasingly being
measured (e.g., teacher N of
participation years; extent of
teacher training; student game
subject focus;
Domains of Learning and Expertise
• Constructionist digital literacy (skills needed in knowledge
economy => 6-CLAs)
• Computational thinking through game design in Flash and
programming in Actionscript
• Core curricular subject matter:
– When game subjects are linked to core curriculum and students
deepen knowledge about topic through online research and
design
• STEM career interests: Technology & Engineering;
Computer Science
• Motivation, Affect, Attitudes, Life Choices, New Possibilities
and Horizons, esp. for disadvantaged students
Flash software, Wiki Environment, Curriculum,
Tutorials
“Hands On” Training Sessions (virtual, local)
• Globaloria Academy – In-person, intensive trainings (3)
• Online Mini Webinars - Web-based workshops (7)
Globaloria Mentors Program
Experienced educators take on a leadership role by
supporting other educators
“24/7” Virtual Support
• Expert Support via wikis, blogs, email, WebEx
• Educator Community Development – private
educators community wiki, peer-to-peer mentoring, weekly
educators newsletter, sharing teaching & learning reports
Rewards and Recognition
•Teachers: Stipends and Graduate credits are earned
•Students: Nationally-Recognized Game Design
Competitions
Learning Supports for Students and Educators:
Syllabus integrates learning about the
game subject topic and
game programming and mechanics
Syllabus integrates learning about the
game subject topic and
game programming and mechanics
Syllabus
Topics
Addressing
Game Subject
Syllabus Topics
Addressing Game
Programming and
Mechanics
www.myglife.org/usa/wv
Syllabus integrates learning about the
game subject topic and
game programming and mechanics
Data Collection for
Research Occurs via 6
Main Technology and
Curriculum Platforms
Developed by the
World Wide Workshop
Data Collection for
Research Occurs via 6
Main Technology and
Curriculum Platforms
Developed by the
World Wide Workshop
Data Collection for
Research Occurs via 6
Main Technology and
Curriculum Platforms
Developed by the
World Wide Workshop
Data Collection for
Research Occurs via 6
Main Technology and
Curriculum Platforms
Developed by the
World Wide Workshop
The Globaloria
Learning
Formula:
Discovery-based
project-based,
student-
centered, social
learning
Some Globaloria project research
goals at a high level
• Understanding the influences and forces at play in this social learning
system; mapping the terrain from a social science perspective
• Design-based research: Iterative improvement of the social learning
system; generalization towards learning theory, and the instructional
design of effective learning innovations
Connecting multiple levels of analysis in our research will be necessary to accomplish
the above . . . And ideally, the below:
• Evaluative Diagnostics:
– Use predictive modeling to identify trajectories of student success / failure
across time (what factors at multiple levels of analysis influence student
learning?)
– Apply predictive findings towards development of diagnostic tools for early
intervention that teachers can use for formative assessment
• Summative Evaluation: Developing tools for program evaluation and
student assessment
2 Main Strands of Research
Quantitative multi-level analysis modeling (w. Ming Ming Chiu)
• Purpose: Model the complex relationships among variables of importance. Currently,
DV is evaluation scores for games based on content analysis of their quality
• Present data sources:
– Teacher and student pre / post surveys
– Teacher progress reports
– Content analysis evaluation of team games
– Wiki log files and behavioral site metrics data (edits, uploads)
– Other data TBA (e.g., game subjects; teacher grades; student role-taking; resource use)
Qualitative analysis, comparison of cases (w. Cindy Hmelo-Silver)
• Purpose: investigate and explain dynamics / inter-relationships among variables;
identify key phenomena driving the system; add to quantitative model as relevant
• Present data sources:
– Semi-structured teacher and student team interviews at two timepoints this past semester
(18 middle school teams, TX & WV, and their educators)
• Main themes: Teacher/peer/e-learning/non-profit scaffolding; student problem self-structuring;
student/teacher resource use; areas of challenge/enjoyment; teamwork processes
– Future: Participant observation in situ in the classroom
Challenging Assumptions
• To what extent do complex multi-level phenomena and
dynamics in this system break down into discrete
variables that can be investigated for their inter-
relationships in quantitative modeling? Is the
phenomena in such a social learning system better
described quantitatively? Qualitatively? Mixed
methods? How can varying analyses inform each
other?
– Perhaps something we can discuss today
Analytical difficulty Statistical Discourse Analysis strategy
Outcome variable
Binary
Infrequent
Teams within classrooms
(Nested data)
Multiple outcomes
Independent variables
Indirect, mediation effects
False positives
Logit (Kennedy, 2008)
Logit bias estimator
(King & Zeng, 2001)
Multilevel analysis
(Goldstein, 1995)
Multi-outcome analysis
(Goldstein, 1995)
Multilevel M-Tests
(Krull & MacKinnon, 2001)
Two-stage linear step-up procedure
(Benjamini et al., 2006)
Mean pre-intrinsic
motivation
+0.176 *
Mean increase in
intrinsic motivation
+0.163 *
Mean SWF flash
uploads per month
+0.166 *
Mean age –0.276 ***
Team Structure
Design mixed
genre game
+0.833 **
Team Processes
Design
Game play
Concept
Mean self-
learning hours
Programming
–0.452 ***
Teacher Outcome
Analysis highlights
3 levels
Classroom (teacher)
Team (mean of group members)
Team process (mean of team processes)
Test hypotheses about outcomes
w/ explanatory variables at lower levels
--characteristics of samples
Mean, SD, max, min, …
Model multiple outcomes simultaneously
Qualitative analysis
1. What are the theoretical instructional design assumptions implicit in the
design and development of Globaloria?
– Guided discovery-based learning, building on Constructionism
– Scaffolding provided via a range of structures (explored in research)
2. How are students at different schools interacting with the range of
scaffolds provided? Autonomous/controlled/task-driven
– A focus on student self-structuring of problems and resource uses to support their
learning, at various stages of the process
3. Linking Scaffolds, Problem-Solving, and Resource Use:
– To what extent does it appear that the structuring elements / scaffolds above help
students meet learning goals: conceptualize the idea for a game, develop a prototype,
implement a game in Flash, learn programming in Actionscript, complete the final game,
do this in a team context (combining files), and upload the final game to the wiki?
4. Does this process appear to bring about other outcomes besides discrete
CLAs/Constructionist digital literacy, and/or other domains of expertise
(computational thinking; STEM career interest)
– Resourcefulness?
Approach to the ICLS 2012
Data Workshop
What can we learn about multi-level phenomena from
an in-depth focus on 1 class of 2010/2011 game
designers, using an eclectic array of data sources?
• Investigate the experience of a single case study team of
game designers from TVHS, consisting of 3 HS students who
won the annual game design competition in the 2010/2011
school year with their Civics content game, “Henry the
Hedgehog”
– Understanding team processes can expose some of the
values/norms built into the system (what makes a good game?)
• Investigate relationships among variables that appear
influential to learning.
• Consider this case in context, given that similar data are
available for over 2000 students across 3+ years
• Apply findings to ongoing research being conducted
Data Sources Provided to Participants, and Corresponding Levels of Analysis
Level of
Analysis
Local context Data source Filenames
Innovation Curriculum Description of Globaloria syllabus,program
overview
2.Curriculum.pdf
ICLS2012_dataworkshop.pptx
E-learning
platform
Description of features of e-learning platform;
screenshots; wiki log files (trace data of student
activity on the wiki)
2.Curriculum.pdf
ICLS2012_dataworkshop.pptx
13. Aggregate wiki log _ contentanalysis.xlsx
System WV
Department
of Education
Game competition press release; information on
judges and judging criteria.
14.game_comps_press_2011-1.pdf
SampleWikiCurricLesson_Drag and Drop.docx
WikiScreenshots folder
Organizational School Description of the school 3.Case Study Data File, Carrot Wizards.docx
ICLS2012_dataworkshop.pptx
World Wide
Workshop
non-profit
Description of non-profit, goals 3.Case Study Data File, Carrot Wizards.docx
ICLS2012_dataworkshop.pptx
Class Teacher Progress report from teacher offering reflection on
PD experiences, learning and teaching; and
observations of students -- individuals and teams
4.teacherprogress report.doc
Team Game design
teams
Team key (who is in what team); Carrot Wizards
case study report; evaluation of team games; wiki
log files
5.teammembers_fullclass_tvhs_y4.csv
7.1.CarrotWizardsFINALGAME.docx
8.yr4_globaloria_eval_researchfile_TVHS.xlsx
13. Aggregate wiki log _ contentanalysis.xlsx
Individual Student
participants
Pre- and Post-Surveys; student blog URLs; case
study report; wiki log files
9.yr4_globaloria_postsurvey_qualitative_TVHS.xls
10.yr4_globaloria_presurvey_qualitative_TVHS.xls
11.blogURLsforCarrotWizards.docx
13. Aggregate wiki log _ contentanalysis.xlsx
Artifact Student
games;
student wiki
work
Actual team games; wiki screenshots; game
evaluation
6.File_Cwfinal2atyler.fla
7.1.CarrotWizardsFINALGAME.docx
7.File_Cwfinal2atyler.swf
3.Case Study Data File, Carrot Wizards.docx
WikiScreenshots folder
8.yr4_globaloria_eval_researchfile_TVHS.xlsx
Further context about TVHS
• WV high school class at TVHS took Globaloria as a game design
elective, meeting daily for 90 minutes per day, for credit and a
grade across an entire school year (2010/2011)
• This was TVHS and educator A. Leef’s first year in the program
implementing Globaloria; she participated in the standard
Globaloria Academy PD training and ongoing offerings throughout
the year.
• Students in 8 teams of 1, 2 or 3, created a total of 8 games about a
range of topics relating to either Civics or STEM
• 2 games focused on Civics: Carrot Wizards’ Henry the Hedgehog
game, and Skull Crusher’s game Blast.
• Henry the Hedgehog game won the Globaloria Annual Civics Game
Design competition in the 2010/2011 school year
Analysis: Narrowing in on Team Process step of
quantitative multi-level analysis, using Wiki log files
• Behavioral trace log file data available, measuring
individual-level edits to wiki pages, and uploads of
game artifacts, graphics, .FLAs and .SWFs.
• Research Questions:
– How do edits / uploads of teams at TVHS vary across time?
• Totals, team means, indivs-within-teams
– General observations about wiki use patterns
– What assumptions/limitations associated with using
frequency data in multi-level analysis?
– What stories do the data tell, that the aggregate frequency
numbers miss?
Preliminary Researcher Analysis of
Data Sources: Wiki log files
• Edits to team pages (main locus on the wiki for team-level activity):
• Game project file uploads
Team Rankings,
Total Sum Evaluation Score
Team Name:TOTAL
SUM
Carrot_Wizards 57
Shock-a-holics 53
LEET_HAX 51
Bull_Loany_Productions 50
Gummy worms.(: 49
Thomas the Train's
Pancakes of Pain 49
TEAM_SKULL_CRUSHER 46
TOMORROW 19
Aggregate Sum Results, Edits
Team Name
N of Team
Members Total Edits Mean Edits
Carrot_Wizards 3 161.00 53.67
Gummy_worms.(: 2 144.00 72.00
LEET_HAX 1 66 66
Shock-a-holics 3 91 30.33
TEAM_SKULL_CRU
SHER 2 60 30
Thomas_the_Train
's_Pancakes_of_Pa
in 3 60 20
TOMORROW 1 12 12
Descriptive Results
Total Edits to Wiki Team Page
Week of 9/20/10 – week of 5/9/11
0.00
20.00
40.00
60.00
80.00
100.00
120.00
140.00
160.00
180.00
Total edits
Mean edits
Total N of Edits to Team Wiki Pages,
Week of 9/20/10 – week of 5/9/11
How does team activity vary across time?
• Carrot Wizards appears to have invested greater effort in editing their team wiki page at
the beginning than other teams. Evidence of possible impact of early planning.
• Carrot Wizards and Gummy Worms appear most consistent in their wiki editing across
time
• Carrot Wizards appears to have exerted more effort towards the end of the school year
than other teams
Note: 1 outlier data point from a single day by a single student in 2010w12, Leethax team omitted
Mean Edits to Team Wiki Pages,
Week of 9/20/10 – week of 5/9/11
(Mean = Total N of page edits / N of team members)
How does standardizing for N of team members influence patterns?
• Gummy worms had fewer team members, thus logged more average activity with consistency as a
team
Note: 1 outlier data point froma single day by a single student in 2010w12, Leethax team omitted
Explaining Gummy Worms greater N of
edits (individuals within teams)
• Consistency in editing among both team members
Explaining Gummy Worms greater N of
edits (individuals within teams)
• Tyler was primary wiki team page editor. The other 2 team members
only did so in S2 at the end.
• What is the difference in team dynamic, between having 2 vs. 3 team
members?
Total N of File Uploads,
Week of 9/20/10 – week of 5/9/11
How does team activity vary across time?
• All teams have greater frequency/consistency of uploads in S1; uploads become more
concentrated during S2
• File types vary to a greater extent in S1 (includes more static graphics files)
• Carrot Wizards and Gummy Worms files are mainly SWFs and FLAs in S2
Explaining Gummy Worms greater N of
uploads (individuals within teams)
• Brianna uploads most frequently and consistently
• Nicole outlier in week 30
Carrot Wizards individuals-within-team
Uploads
• Relative consistency among team members
• What was occurring at week 29-34 in the program curriculum to spark high uploads
among these 2 teams?
Connecting team and individual levels
of analysis
• How is wiki work distributed across individual
team members?
– Procedure:
• For Carrot Wizards only, reviewed log files across
time, by team member
• Review actual wiki content edits made by team
members across time
• Note categorical types of wiki content edits
• Make observations regarding individual activity within
the team
– Link inferences to team-level findings above
Carrot Wizards distribution of individual
contributions to wiki
Carrot Wizards distribution of individual
contributions to wiki
Carrot Wizards distribution of individual
contributions to wiki
Carrot Wizards distribution of individual
contributions to wiki
Carrot Wizards distribution of individual
contributions to wiki
Connecting Wiki activity
to Evaluation Scores
• To what extent do patterns of wiki activity
relate to evaluation outcomes?
– This is a next step in our analysis.
– We welcome the input of other researchers
Next Steps
• Summarize findings on individual
contributions within the team context
• Consider results in relation to game evaluation
findings
Opportunities for Workshop
Participants
• Conduct social network analysis / sequential
interaction analysis on individual and team level
wiki activity results for TVHS teams
– Raw aggregated wiki log file and evaluation outcome
data are available in the Excel file, “13. Aggregate wiki
log _ contentanalysis.xls”
• Relate wiki raw frequency and social network
analysis findings to final game evaluation
outcomes
• Explore relationships between qualitative findings
in Dropbox, and results reported herein

Más contenido relacionado

La actualidad más candente

Analytics Goes to College: Better Schooling Through Information Technology wi...
Analytics Goes to College: Better Schooling Through Information Technology wi...Analytics Goes to College: Better Schooling Through Information Technology wi...
Analytics Goes to College: Better Schooling Through Information Technology wi...bisg
 
Learning connections 2019_UCC_McGuinness & Fulton
Learning connections 2019_UCC_McGuinness & FultonLearning connections 2019_UCC_McGuinness & Fulton
Learning connections 2019_UCC_McGuinness & FultonUniversity College Dublin
 
Nssa las vegas_4-2-12__final
Nssa las vegas_4-2-12__finalNssa las vegas_4-2-12__final
Nssa las vegas_4-2-12__finalLamar University
 
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society CITE
 
Amy lukic presentation_software_assessment_e_tool
Amy lukic presentation_software_assessment_e_toolAmy lukic presentation_software_assessment_e_tool
Amy lukic presentation_software_assessment_e_toolMrsLukic
 
Dr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based LearningDr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based LearningCITE
 
The Future Looks Digital - TCCTA
The Future Looks Digital - TCCTAThe Future Looks Digital - TCCTA
The Future Looks Digital - TCCTAdrpmcgee
 
Citing web sources_lesson_plan_intermediate
Citing web sources_lesson_plan_intermediateCiting web sources_lesson_plan_intermediate
Citing web sources_lesson_plan_intermediatethinkict
 
Munassir etec647 e presentation
Munassir etec647 e presentationMunassir etec647 e presentation
Munassir etec647 e presentationMunassir Alhamami
 
Role of data analytics in educational industry
Role of data analytics in educational industryRole of data analytics in educational industry
Role of data analytics in educational industryRuthVanlalremruati
 
Education Data Sciences and the Need for Interpretive Skills
Education Data Sciences and the Need for Interpretive Skills Education Data Sciences and the Need for Interpretive Skills
Education Data Sciences and the Need for Interpretive Skills Philip Piety
 
Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...
Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...
Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...eMadrid network
 
Educational Technologies: What should you be thinking about next?
Educational Technologies: What should you be thinking about next?Educational Technologies: What should you be thinking about next?
Educational Technologies: What should you be thinking about next?Jason Zagami
 
Data-Driven Learning Strategy
Data-Driven Learning StrategyData-Driven Learning Strategy
Data-Driven Learning StrategyJessie Chuang
 
Next Generation of Learners
Next Generation of Learners Next Generation of Learners
Next Generation of Learners Julie Evans
 
Reducing the Cost of College: New Directions for Faculty eAuthoring
Reducing the Cost of College: New Directions for Faculty eAuthoringReducing the Cost of College: New Directions for Faculty eAuthoring
Reducing the Cost of College: New Directions for Faculty eAuthoringErin Gratz
 

La actualidad más candente (20)

Analytics Goes to College: Better Schooling Through Information Technology wi...
Analytics Goes to College: Better Schooling Through Information Technology wi...Analytics Goes to College: Better Schooling Through Information Technology wi...
Analytics Goes to College: Better Schooling Through Information Technology wi...
 
Learning connections 2019_UCC_McGuinness & Fulton
Learning connections 2019_UCC_McGuinness & FultonLearning connections 2019_UCC_McGuinness & Fulton
Learning connections 2019_UCC_McGuinness & Fulton
 
Nssa las vegas_4-2-12__final
Nssa las vegas_4-2-12__finalNssa las vegas_4-2-12__final
Nssa las vegas_4-2-12__final
 
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
Prof. Gerald KNEZEK: Implications of Digital Generations for a Learning Society
 
Amy lukic presentation_software_assessment_e_tool
Amy lukic presentation_software_assessment_e_toolAmy lukic presentation_software_assessment_e_tool
Amy lukic presentation_software_assessment_e_tool
 
Dr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based LearningDr. David Gibson: Challenge-Based Learning
Dr. David Gibson: Challenge-Based Learning
 
The Future Looks Digital - TCCTA
The Future Looks Digital - TCCTAThe Future Looks Digital - TCCTA
The Future Looks Digital - TCCTA
 
Citing web sources_lesson_plan_intermediate
Citing web sources_lesson_plan_intermediateCiting web sources_lesson_plan_intermediate
Citing web sources_lesson_plan_intermediate
 
Munassir etec647 e presentation
Munassir etec647 e presentationMunassir etec647 e presentation
Munassir etec647 e presentation
 
HB633OW
HB633OWHB633OW
HB633OW
 
Role of data analytics in educational industry
Role of data analytics in educational industryRole of data analytics in educational industry
Role of data analytics in educational industry
 
Education Data Sciences and the Need for Interpretive Skills
Education Data Sciences and the Need for Interpretive Skills Education Data Sciences and the Need for Interpretive Skills
Education Data Sciences and the Need for Interpretive Skills
 
Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...
Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...
Seminario eMadrid "Reinventar la educación". Seiji Isotani, University of Sao...
 
PhD Presentation: "Supporting collaborative learning among Cuban university s...
PhD Presentation: "Supporting collaborative learning among Cuban university s...PhD Presentation: "Supporting collaborative learning among Cuban university s...
PhD Presentation: "Supporting collaborative learning among Cuban university s...
 
Web quest clip 1
Web quest clip 1Web quest clip 1
Web quest clip 1
 
Educational Technologies: What should you be thinking about next?
Educational Technologies: What should you be thinking about next?Educational Technologies: What should you be thinking about next?
Educational Technologies: What should you be thinking about next?
 
Data-Driven Learning Strategy
Data-Driven Learning StrategyData-Driven Learning Strategy
Data-Driven Learning Strategy
 
Next Generation of Learners
Next Generation of Learners Next Generation of Learners
Next Generation of Learners
 
Berlin 2017 publish
Berlin 2017 publishBerlin 2017 publish
Berlin 2017 publish
 
Reducing the Cost of College: New Directions for Faculty eAuthoring
Reducing the Cost of College: New Directions for Faculty eAuthoringReducing the Cost of College: New Directions for Faculty eAuthoring
Reducing the Cost of College: New Directions for Faculty eAuthoring
 

Destacado

Ppt elluminate 030613
Ppt elluminate 030613Ppt elluminate 030613
Ppt elluminate 030613alfedu8
 
NetPlus Alliance 2013 Presentation
NetPlus Alliance 2013 PresentationNetPlus Alliance 2013 Presentation
NetPlus Alliance 2013 PresentationJennifer Murphy
 
How Russian and Romanian press presented the Cypriot No
How Russian and Romanian press presented the Cypriot No  How Russian and Romanian press presented the Cypriot No
How Russian and Romanian press presented the Cypriot No Dorina.Izbisciuc
 
Los loros.
Los loros.Los loros.
Los loros.sgprim
 
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolers in a G...
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolersin a G...Inquiry and Resource Use Strategies that Emerge Among Middle Schoolersin a G...
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolers in a G...Rebecca Reynolds
 
Asist 2013 panelpresentation_social media_final
Asist 2013 panelpresentation_social media_finalAsist 2013 panelpresentation_social media_final
Asist 2013 panelpresentation_social media_finalRebecca Reynolds
 
Rebecca Reynolds Research Agenda
Rebecca Reynolds Research AgendaRebecca Reynolds Research Agenda
Rebecca Reynolds Research AgendaRebecca Reynolds
 
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...Rebecca Reynolds
 
UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...
UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...
UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...Rebecca Reynolds
 
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...Rebecca Reynolds
 
Innovación en la Administración Pública
Innovación en la Administración PúblicaInnovación en la Administración Pública
Innovación en la Administración PúblicaRaúl GONZÁLEZROMERO
 
Top 8 head security resume samples
Top 8 head security resume samplesTop 8 head security resume samples
Top 8 head security resume samplesKrisAllen345
 
Aria on the Bay miami
Aria on the Bay   miamiAria on the Bay   miami
Aria on the Bay miamiSergio Pripas
 

Destacado (20)

Ppt elluminate 030613
Ppt elluminate 030613Ppt elluminate 030613
Ppt elluminate 030613
 
Water
WaterWater
Water
 
NetPlus Alliance 2013 Presentation
NetPlus Alliance 2013 PresentationNetPlus Alliance 2013 Presentation
NetPlus Alliance 2013 Presentation
 
Lake Huron Shoreline | 2013 Grey Bruce Escape magazine
Lake Huron Shoreline | 2013 Grey Bruce Escape magazineLake Huron Shoreline | 2013 Grey Bruce Escape magazine
Lake Huron Shoreline | 2013 Grey Bruce Escape magazine
 
Cortafuegos verde
Cortafuegos verdeCortafuegos verde
Cortafuegos verde
 
How Russian and Romanian press presented the Cypriot No
How Russian and Romanian press presented the Cypriot No  How Russian and Romanian press presented the Cypriot No
How Russian and Romanian press presented the Cypriot No
 
Los loros.
Los loros.Los loros.
Los loros.
 
Scan6
Scan6Scan6
Scan6
 
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolers in a G...
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolersin a G...Inquiry and Resource Use Strategies that Emerge Among Middle Schoolersin a G...
Inquiry and Resource Use Strategies that Emerge Among Middle Schoolers in a G...
 
Asist 2013 panelpresentation_social media_final
Asist 2013 panelpresentation_social media_finalAsist 2013 panelpresentation_social media_final
Asist 2013 panelpresentation_social media_final
 
Rebecca Reynolds Research Agenda
Rebecca Reynolds Research AgendaRebecca Reynolds Research Agenda
Rebecca Reynolds Research Agenda
 
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
Challenges in Defining, Designing, and Measuring “Digital Literacy” Developm...
 
UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...
UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...
UT-Austin Guest Lecture, ""Patterns and Outcomes of Youth Engagement in Colla...
 
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
Collaborative Information Seeking in the Wild: Student-Initiated Solutions a...
 
Innovación en la Administración Pública
Innovación en la Administración PúblicaInnovación en la Administración Pública
Innovación en la Administración Pública
 
Top 8 head security resume samples
Top 8 head security resume samplesTop 8 head security resume samples
Top 8 head security resume samples
 
Портфоліо Мартинюк І. В.
Портфоліо Мартинюк І. В.Портфоліо Мартинюк І. В.
Портфоліо Мартинюк І. В.
 
3000 Word Essay
3000 Word Essay3000 Word Essay
3000 Word Essay
 
Aria on the Bay miami
Aria on the Bay   miamiAria on the Bay   miami
Aria on the Bay miami
 
byeinkapp keynote
byeinkapp keynotebyeinkapp keynote
byeinkapp keynote
 

Similar a Using Learning Analytics Data to Understand Project-Based Learning

Empowering Students and Educators with New Media Literacies Necessary to Part...
Empowering Students and Educators with New Media Literacies Necessary to Part...Empowering Students and Educators with New Media Literacies Necessary to Part...
Empowering Students and Educators with New Media Literacies Necessary to Part...Rebecca Reynolds
 
Monaghan DET-CHE 2013
Monaghan DET-CHE 2013Monaghan DET-CHE 2013
Monaghan DET-CHE 2013Jim Monaghan
 
Aera2013knowledgebuilding final
Aera2013knowledgebuilding finalAera2013knowledgebuilding final
Aera2013knowledgebuilding finalRebecca Reynolds
 
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_finaltransform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_finalSharon Stoerger
 
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_finaltransform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_finalSharon Stoerger
 
Big data in education
Big data in educationBig data in education
Big data in educationMart Laanpere
 
Rae t4 d-knowledge-economy-sa-urs-dec2017
Rae t4 d-knowledge-economy-sa-urs-dec2017Rae t4 d-knowledge-economy-sa-urs-dec2017
Rae t4 d-knowledge-economy-sa-urs-dec2017MYRA School of Business
 
Collaborative learning presentation
Collaborative learning presentationCollaborative learning presentation
Collaborative learning presentationShiKia Carter
 
Empowering the Instructor with Learning Analytics
Empowering the Instructor with Learning AnalyticsEmpowering the Instructor with Learning Analytics
Empowering the Instructor with Learning AnalyticsMichael Wilder
 
Conole learning design
Conole learning designConole learning design
Conole learning designGrainne Conole
 
Personal learning environments brenton dass 201225820
Personal learning environments brenton dass 201225820Personal learning environments brenton dass 201225820
Personal learning environments brenton dass 201225820Brenton Dass
 
Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...
Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...
Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...Karel Van Isacker
 
Currents in Information Literacy: Standards, Lessons, and Learners
Currents in Information Literacy: Standards, Lessons, and LearnersCurrents in Information Literacy: Standards, Lessons, and Learners
Currents in Information Literacy: Standards, Lessons, and LearnersWestern Illinois University
 
The Pedagogy of Virtual, Cross-culture, Probel Based Learning
The Pedagogy of Virtual, Cross-culture, Probel Based LearningThe Pedagogy of Virtual, Cross-culture, Probel Based Learning
The Pedagogy of Virtual, Cross-culture, Probel Based LearningDaniela Munca-Aftenev
 
Requirements for Learning Analytics
Requirements for Learning AnalyticsRequirements for Learning Analytics
Requirements for Learning AnalyticsTore Hoel
 
Gbl technology models
Gbl technology modelsGbl technology models
Gbl technology modelscarena
 

Similar a Using Learning Analytics Data to Understand Project-Based Learning (20)

Empowering Students and Educators with New Media Literacies Necessary to Part...
Empowering Students and Educators with New Media Literacies Necessary to Part...Empowering Students and Educators with New Media Literacies Necessary to Part...
Empowering Students and Educators with New Media Literacies Necessary to Part...
 
Monaghan DET-CHE 2013
Monaghan DET-CHE 2013Monaghan DET-CHE 2013
Monaghan DET-CHE 2013
 
Aera2013knowledgebuilding final
Aera2013knowledgebuilding finalAera2013knowledgebuilding final
Aera2013knowledgebuilding final
 
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_finaltransform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
 
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_finaltransform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
transform_large_lecture_course_COHLIT_DKreiger_SStoerger_final
 
Big data in education
Big data in educationBig data in education
Big data in education
 
Social Learning Analytics
Social Learning AnalyticsSocial Learning Analytics
Social Learning Analytics
 
Building Blocks for College Readiness
Building Blocks for College ReadinessBuilding Blocks for College Readiness
Building Blocks for College Readiness
 
EdTechProgramNationalUniversity
EdTechProgramNationalUniversityEdTechProgramNationalUniversity
EdTechProgramNationalUniversity
 
Rae t4 d-knowledge-economy-sa-urs-dec2017
Rae t4 d-knowledge-economy-sa-urs-dec2017Rae t4 d-knowledge-economy-sa-urs-dec2017
Rae t4 d-knowledge-economy-sa-urs-dec2017
 
Collaborative learning presentation
Collaborative learning presentationCollaborative learning presentation
Collaborative learning presentation
 
Empowering the Instructor with Learning Analytics
Empowering the Instructor with Learning AnalyticsEmpowering the Instructor with Learning Analytics
Empowering the Instructor with Learning Analytics
 
Conole learning design
Conole learning designConole learning design
Conole learning design
 
Personal learning environments brenton dass 201225820
Personal learning environments brenton dass 201225820Personal learning environments brenton dass 201225820
Personal learning environments brenton dass 201225820
 
Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...
Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...
Event “Enhancing the skills of youth with learning disabilities” on 22 June 2...
 
Currents in Information Literacy: Standards, Lessons, and Learners
Currents in Information Literacy: Standards, Lessons, and LearnersCurrents in Information Literacy: Standards, Lessons, and Learners
Currents in Information Literacy: Standards, Lessons, and Learners
 
The Pedagogy of Virtual, Cross-culture, Probel Based Learning
The Pedagogy of Virtual, Cross-culture, Probel Based LearningThe Pedagogy of Virtual, Cross-culture, Probel Based Learning
The Pedagogy of Virtual, Cross-culture, Probel Based Learning
 
Berlin_presentation _30.09.2022.pptx
Berlin_presentation _30.09.2022.pptxBerlin_presentation _30.09.2022.pptx
Berlin_presentation _30.09.2022.pptx
 
Requirements for Learning Analytics
Requirements for Learning AnalyticsRequirements for Learning Analytics
Requirements for Learning Analytics
 
Gbl technology models
Gbl technology modelsGbl technology models
Gbl technology models
 

Último

How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptxmary850239
 
Optical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxOptical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxPurva Nikam
 
A gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceA gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceApostolos Syropoulos
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptxSlides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptxCapitolTechU
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17Celine George
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17Celine George
 
Department of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdfDepartment of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdfMohonDas
 
Over the counter (OTC)- Sale, rational use.pptx
Over the counter (OTC)- Sale, rational use.pptxOver the counter (OTC)- Sale, rational use.pptx
Over the counter (OTC)- Sale, rational use.pptxraviapr7
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 

Último (20)

How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptx
 
Optical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxOptical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptx
 
A gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceA gentle introduction to Artificial Intelligence
A gentle introduction to Artificial Intelligence
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptxSlides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
Slides CapTechTalks Webinar March 2024 Joshua Sinai.pptx
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17
 
Department of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdfDepartment of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdf
 
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdfPersonal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
 
Over the counter (OTC)- Sale, rational use.pptx
Over the counter (OTC)- Sale, rational use.pptxOver the counter (OTC)- Sale, rational use.pptx
Over the counter (OTC)- Sale, rational use.pptx
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 

Using Learning Analytics Data to Understand Project-Based Learning

  • 1. Using Learning Analytics Data to Understand Project-Based Learning The Case of the Carrot Wizards in Globaloria- West Virginia ICLS 2012 Pre-Conference Data Workshop *** Rebecca Reynolds, Ph.D. Assistant Professor, Library & Information Science Rutgers University School of Communication and Information
  • 2. Background Info • The following slides provide background on the Constructionist / design-based research innovation, “Globaloria,” being implemented by a NYC non-profit called the World Wide Wokrshop, in several US states with grant and state education department support (WV, TX, CA, FL, NY)
  • 3. Globaloria Game Design Program Learning Objectives: Cultivate the Six Contemporary Learning Abilities (6CLAs) Developing games in a social learning system offers students Constructionist, participatory design and programming experiences leading to dispositions, practices and expertise necessary for successful engagement in today’s digital, knowledge-based economy
  • 4. The Six Contemporary Learning Abilities (6CLAs): Developing games in a social learning system offers participatory experiences leading to practices necessary for engagement in today’s knowledge-based economy
  • 5. Globaloria is currently active in 4 U.S. states: CA, TX, FL, WV
  • 6. Globaloria-California California, 2011/2012: • Supported by $1M Knight Foundation grant • “Formal” implementation at 2 middle schools (67 students) • “Informal” implementation at 2 Boys and Girls Club community centers (38 students) • First year of California participation • Potential to compare implementation context variable differences (time on task; provision of grades vs. none; linkage to core curriculum vs. open choice game topics, etc.)
  • 7. Globaloria-TX East Austin College Prep, 2011/2012: • 1 charter middle school • 3 grades (6th, 7th, 8th) • ~250 students • 3rd year of school’s participation • 2 educators • Unique longitudinal implementation (all EACPA students take Globaloria daily, for credit and a grade for 3 years)
  • 8. Globaloria-West Virginia2011/2012: • 5th year of Globaloria implementation in West Virginia • Individual schools and teachers range in N of participation years • Majority of students, 1 year only • 17 middle schools • 50 high schools • All in-school implementation, daily, for credit, for grade • Mix of elective game design classes, and required classes linked to the core curriculum • Multi-level analysis and implementation context variable differences increasingly being measured (e.g., teacher N of participation years; extent of teacher training; student game subject focus;
  • 9. Domains of Learning and Expertise • Constructionist digital literacy (skills needed in knowledge economy => 6-CLAs) • Computational thinking through game design in Flash and programming in Actionscript • Core curricular subject matter: – When game subjects are linked to core curriculum and students deepen knowledge about topic through online research and design • STEM career interests: Technology & Engineering; Computer Science • Motivation, Affect, Attitudes, Life Choices, New Possibilities and Horizons, esp. for disadvantaged students
  • 10. Flash software, Wiki Environment, Curriculum, Tutorials “Hands On” Training Sessions (virtual, local) • Globaloria Academy – In-person, intensive trainings (3) • Online Mini Webinars - Web-based workshops (7) Globaloria Mentors Program Experienced educators take on a leadership role by supporting other educators “24/7” Virtual Support • Expert Support via wikis, blogs, email, WebEx • Educator Community Development – private educators community wiki, peer-to-peer mentoring, weekly educators newsletter, sharing teaching & learning reports Rewards and Recognition •Teachers: Stipends and Graduate credits are earned •Students: Nationally-Recognized Game Design Competitions Learning Supports for Students and Educators:
  • 11. Syllabus integrates learning about the game subject topic and game programming and mechanics
  • 12. Syllabus integrates learning about the game subject topic and game programming and mechanics Syllabus Topics Addressing Game Subject Syllabus Topics Addressing Game Programming and Mechanics www.myglife.org/usa/wv
  • 13. Syllabus integrates learning about the game subject topic and game programming and mechanics
  • 14. Data Collection for Research Occurs via 6 Main Technology and Curriculum Platforms Developed by the World Wide Workshop
  • 15. Data Collection for Research Occurs via 6 Main Technology and Curriculum Platforms Developed by the World Wide Workshop
  • 16. Data Collection for Research Occurs via 6 Main Technology and Curriculum Platforms Developed by the World Wide Workshop
  • 17. Data Collection for Research Occurs via 6 Main Technology and Curriculum Platforms Developed by the World Wide Workshop
  • 19. Some Globaloria project research goals at a high level • Understanding the influences and forces at play in this social learning system; mapping the terrain from a social science perspective • Design-based research: Iterative improvement of the social learning system; generalization towards learning theory, and the instructional design of effective learning innovations Connecting multiple levels of analysis in our research will be necessary to accomplish the above . . . And ideally, the below: • Evaluative Diagnostics: – Use predictive modeling to identify trajectories of student success / failure across time (what factors at multiple levels of analysis influence student learning?) – Apply predictive findings towards development of diagnostic tools for early intervention that teachers can use for formative assessment • Summative Evaluation: Developing tools for program evaluation and student assessment
  • 20. 2 Main Strands of Research Quantitative multi-level analysis modeling (w. Ming Ming Chiu) • Purpose: Model the complex relationships among variables of importance. Currently, DV is evaluation scores for games based on content analysis of their quality • Present data sources: – Teacher and student pre / post surveys – Teacher progress reports – Content analysis evaluation of team games – Wiki log files and behavioral site metrics data (edits, uploads) – Other data TBA (e.g., game subjects; teacher grades; student role-taking; resource use) Qualitative analysis, comparison of cases (w. Cindy Hmelo-Silver) • Purpose: investigate and explain dynamics / inter-relationships among variables; identify key phenomena driving the system; add to quantitative model as relevant • Present data sources: – Semi-structured teacher and student team interviews at two timepoints this past semester (18 middle school teams, TX & WV, and their educators) • Main themes: Teacher/peer/e-learning/non-profit scaffolding; student problem self-structuring; student/teacher resource use; areas of challenge/enjoyment; teamwork processes – Future: Participant observation in situ in the classroom
  • 21. Challenging Assumptions • To what extent do complex multi-level phenomena and dynamics in this system break down into discrete variables that can be investigated for their inter- relationships in quantitative modeling? Is the phenomena in such a social learning system better described quantitatively? Qualitatively? Mixed methods? How can varying analyses inform each other? – Perhaps something we can discuss today
  • 22. Analytical difficulty Statistical Discourse Analysis strategy Outcome variable Binary Infrequent Teams within classrooms (Nested data) Multiple outcomes Independent variables Indirect, mediation effects False positives Logit (Kennedy, 2008) Logit bias estimator (King & Zeng, 2001) Multilevel analysis (Goldstein, 1995) Multi-outcome analysis (Goldstein, 1995) Multilevel M-Tests (Krull & MacKinnon, 2001) Two-stage linear step-up procedure (Benjamini et al., 2006)
  • 23. Mean pre-intrinsic motivation +0.176 * Mean increase in intrinsic motivation +0.163 * Mean SWF flash uploads per month +0.166 * Mean age –0.276 *** Team Structure Design mixed genre game +0.833 ** Team Processes Design Game play Concept Mean self- learning hours Programming –0.452 *** Teacher Outcome
  • 24. Analysis highlights 3 levels Classroom (teacher) Team (mean of group members) Team process (mean of team processes) Test hypotheses about outcomes w/ explanatory variables at lower levels --characteristics of samples Mean, SD, max, min, … Model multiple outcomes simultaneously
  • 25. Qualitative analysis 1. What are the theoretical instructional design assumptions implicit in the design and development of Globaloria? – Guided discovery-based learning, building on Constructionism – Scaffolding provided via a range of structures (explored in research) 2. How are students at different schools interacting with the range of scaffolds provided? Autonomous/controlled/task-driven – A focus on student self-structuring of problems and resource uses to support their learning, at various stages of the process 3. Linking Scaffolds, Problem-Solving, and Resource Use: – To what extent does it appear that the structuring elements / scaffolds above help students meet learning goals: conceptualize the idea for a game, develop a prototype, implement a game in Flash, learn programming in Actionscript, complete the final game, do this in a team context (combining files), and upload the final game to the wiki? 4. Does this process appear to bring about other outcomes besides discrete CLAs/Constructionist digital literacy, and/or other domains of expertise (computational thinking; STEM career interest) – Resourcefulness?
  • 26. Approach to the ICLS 2012 Data Workshop What can we learn about multi-level phenomena from an in-depth focus on 1 class of 2010/2011 game designers, using an eclectic array of data sources? • Investigate the experience of a single case study team of game designers from TVHS, consisting of 3 HS students who won the annual game design competition in the 2010/2011 school year with their Civics content game, “Henry the Hedgehog” – Understanding team processes can expose some of the values/norms built into the system (what makes a good game?) • Investigate relationships among variables that appear influential to learning. • Consider this case in context, given that similar data are available for over 2000 students across 3+ years • Apply findings to ongoing research being conducted
  • 27. Data Sources Provided to Participants, and Corresponding Levels of Analysis Level of Analysis Local context Data source Filenames Innovation Curriculum Description of Globaloria syllabus,program overview 2.Curriculum.pdf ICLS2012_dataworkshop.pptx E-learning platform Description of features of e-learning platform; screenshots; wiki log files (trace data of student activity on the wiki) 2.Curriculum.pdf ICLS2012_dataworkshop.pptx 13. Aggregate wiki log _ contentanalysis.xlsx System WV Department of Education Game competition press release; information on judges and judging criteria. 14.game_comps_press_2011-1.pdf SampleWikiCurricLesson_Drag and Drop.docx WikiScreenshots folder Organizational School Description of the school 3.Case Study Data File, Carrot Wizards.docx ICLS2012_dataworkshop.pptx World Wide Workshop non-profit Description of non-profit, goals 3.Case Study Data File, Carrot Wizards.docx ICLS2012_dataworkshop.pptx Class Teacher Progress report from teacher offering reflection on PD experiences, learning and teaching; and observations of students -- individuals and teams 4.teacherprogress report.doc Team Game design teams Team key (who is in what team); Carrot Wizards case study report; evaluation of team games; wiki log files 5.teammembers_fullclass_tvhs_y4.csv 7.1.CarrotWizardsFINALGAME.docx 8.yr4_globaloria_eval_researchfile_TVHS.xlsx 13. Aggregate wiki log _ contentanalysis.xlsx Individual Student participants Pre- and Post-Surveys; student blog URLs; case study report; wiki log files 9.yr4_globaloria_postsurvey_qualitative_TVHS.xls 10.yr4_globaloria_presurvey_qualitative_TVHS.xls 11.blogURLsforCarrotWizards.docx 13. Aggregate wiki log _ contentanalysis.xlsx Artifact Student games; student wiki work Actual team games; wiki screenshots; game evaluation 6.File_Cwfinal2atyler.fla 7.1.CarrotWizardsFINALGAME.docx 7.File_Cwfinal2atyler.swf 3.Case Study Data File, Carrot Wizards.docx WikiScreenshots folder 8.yr4_globaloria_eval_researchfile_TVHS.xlsx
  • 28. Further context about TVHS • WV high school class at TVHS took Globaloria as a game design elective, meeting daily for 90 minutes per day, for credit and a grade across an entire school year (2010/2011) • This was TVHS and educator A. Leef’s first year in the program implementing Globaloria; she participated in the standard Globaloria Academy PD training and ongoing offerings throughout the year. • Students in 8 teams of 1, 2 or 3, created a total of 8 games about a range of topics relating to either Civics or STEM • 2 games focused on Civics: Carrot Wizards’ Henry the Hedgehog game, and Skull Crusher’s game Blast. • Henry the Hedgehog game won the Globaloria Annual Civics Game Design competition in the 2010/2011 school year
  • 29. Analysis: Narrowing in on Team Process step of quantitative multi-level analysis, using Wiki log files • Behavioral trace log file data available, measuring individual-level edits to wiki pages, and uploads of game artifacts, graphics, .FLAs and .SWFs. • Research Questions: – How do edits / uploads of teams at TVHS vary across time? • Totals, team means, indivs-within-teams – General observations about wiki use patterns – What assumptions/limitations associated with using frequency data in multi-level analysis? – What stories do the data tell, that the aggregate frequency numbers miss?
  • 30. Preliminary Researcher Analysis of Data Sources: Wiki log files • Edits to team pages (main locus on the wiki for team-level activity): • Game project file uploads
  • 31. Team Rankings, Total Sum Evaluation Score Team Name:TOTAL SUM Carrot_Wizards 57 Shock-a-holics 53 LEET_HAX 51 Bull_Loany_Productions 50 Gummy worms.(: 49 Thomas the Train's Pancakes of Pain 49 TEAM_SKULL_CRUSHER 46 TOMORROW 19
  • 32. Aggregate Sum Results, Edits Team Name N of Team Members Total Edits Mean Edits Carrot_Wizards 3 161.00 53.67 Gummy_worms.(: 2 144.00 72.00 LEET_HAX 1 66 66 Shock-a-holics 3 91 30.33 TEAM_SKULL_CRU SHER 2 60 30 Thomas_the_Train 's_Pancakes_of_Pa in 3 60 20 TOMORROW 1 12 12
  • 33. Descriptive Results Total Edits to Wiki Team Page Week of 9/20/10 – week of 5/9/11 0.00 20.00 40.00 60.00 80.00 100.00 120.00 140.00 160.00 180.00 Total edits Mean edits
  • 34. Total N of Edits to Team Wiki Pages, Week of 9/20/10 – week of 5/9/11 How does team activity vary across time? • Carrot Wizards appears to have invested greater effort in editing their team wiki page at the beginning than other teams. Evidence of possible impact of early planning. • Carrot Wizards and Gummy Worms appear most consistent in their wiki editing across time • Carrot Wizards appears to have exerted more effort towards the end of the school year than other teams Note: 1 outlier data point from a single day by a single student in 2010w12, Leethax team omitted
  • 35. Mean Edits to Team Wiki Pages, Week of 9/20/10 – week of 5/9/11 (Mean = Total N of page edits / N of team members) How does standardizing for N of team members influence patterns? • Gummy worms had fewer team members, thus logged more average activity with consistency as a team Note: 1 outlier data point froma single day by a single student in 2010w12, Leethax team omitted
  • 36. Explaining Gummy Worms greater N of edits (individuals within teams) • Consistency in editing among both team members
  • 37. Explaining Gummy Worms greater N of edits (individuals within teams) • Tyler was primary wiki team page editor. The other 2 team members only did so in S2 at the end. • What is the difference in team dynamic, between having 2 vs. 3 team members?
  • 38. Total N of File Uploads, Week of 9/20/10 – week of 5/9/11 How does team activity vary across time? • All teams have greater frequency/consistency of uploads in S1; uploads become more concentrated during S2 • File types vary to a greater extent in S1 (includes more static graphics files) • Carrot Wizards and Gummy Worms files are mainly SWFs and FLAs in S2
  • 39. Explaining Gummy Worms greater N of uploads (individuals within teams) • Brianna uploads most frequently and consistently • Nicole outlier in week 30
  • 40. Carrot Wizards individuals-within-team Uploads • Relative consistency among team members • What was occurring at week 29-34 in the program curriculum to spark high uploads among these 2 teams?
  • 41. Connecting team and individual levels of analysis • How is wiki work distributed across individual team members? – Procedure: • For Carrot Wizards only, reviewed log files across time, by team member • Review actual wiki content edits made by team members across time • Note categorical types of wiki content edits • Make observations regarding individual activity within the team – Link inferences to team-level findings above
  • 42. Carrot Wizards distribution of individual contributions to wiki
  • 43. Carrot Wizards distribution of individual contributions to wiki
  • 44. Carrot Wizards distribution of individual contributions to wiki
  • 45. Carrot Wizards distribution of individual contributions to wiki
  • 46. Carrot Wizards distribution of individual contributions to wiki
  • 47. Connecting Wiki activity to Evaluation Scores • To what extent do patterns of wiki activity relate to evaluation outcomes? – This is a next step in our analysis. – We welcome the input of other researchers
  • 48. Next Steps • Summarize findings on individual contributions within the team context • Consider results in relation to game evaluation findings
  • 49. Opportunities for Workshop Participants • Conduct social network analysis / sequential interaction analysis on individual and team level wiki activity results for TVHS teams – Raw aggregated wiki log file and evaluation outcome data are available in the Excel file, “13. Aggregate wiki log _ contentanalysis.xls” • Relate wiki raw frequency and social network analysis findings to final game evaluation outcomes • Explore relationships between qualitative findings in Dropbox, and results reported herein

Notas del editor

  1. How do we plan to grow exponentially in a way that is sustainable? Strong and innovative Professional Development programs for educators, principals and students ensure the community can self-manage, grow and develop.Hands-on training: Mentor program – cascading and taken to scaleVirtual support – sustainable, scalable – walking our talkRewards & Recognition – we pay
  2. In addition to the amazing impact we have seen over the past few years researching and evaluating the Globaloria program we…We are in unique position of having everything we do be rooted in years of academic research and real-world practice. In the 1980s and 1990s, while at Harvard and the MIT Media Lab, Idit really founded the concept that children learn best by designing – when they are programming computers instead of computers programming them. Ground breaking research with children in Bronx, showing how software designing could change their relationship to education and their engage them in learning in a way that had not been seen before
  3. In addition to the amazing impact we have seen over the past few years researching and evaluating the Globaloria program we…We are in unique position of having everything we do be rooted in years of academic research and real-world practice. In the 1980s and 1990s, while at Harvard and the MIT Media Lab, Idit really founded the concept that children learn best by designing – when they are programming computers instead of computers programming them. Ground breaking research with children in Bronx, showing how software designing could change their relationship to education and their engage them in learning in a way that had not been seen before
  4. In addition to the amazing impact we have seen over the past few years researching and evaluating the Globaloria program we…We are in unique position of having everything we do be rooted in years of academic research and real-world practice. In the 1980s and 1990s, while at Harvard and the MIT Media Lab, Idit really founded the concept that children learn best by designing – when they are programming computers instead of computers programming them. Ground breaking research with children in Bronx, showing how software designing could change their relationship to education and their engage them in learning in a way that had not been seen before
  5. We have taken all this learning and experience and established the cornerstone of the Globaloria Program – It’s Learning Formula. I want to take a minute on this slide: Self-Led Learning- students determine the needs of their learning as they progress through the process of designing their own games. It is purposeful, self-directed and hands-on learingPeer-to-peer learning – it is collaborative learning. Students take on the role of both educators and students. They have a critical role in their own learning and in the advancement of their learning community. They are invested. Co-Learning – we are breaking down the traditional hierarchy in the classroom and having students and educators learn together. Students don’t learn for the grade, but as part of a community of creators. Expert-guided learning – we do not divorce learning from the real-world…it is the real world. Students engage with real professional and see the relevance of their learning in today’s world.