This presentation discusses scholarly definitions for the research construct “digital literacy,” identifies limitations in conceptualizations to-date, fand presents a proposed framework of Six Contemporary Learning Abilities (or 6-CLAs: Create, Manage, Publish, Socialize, Research, Surf). This explicated framework offers a more structured definition based on student-centered social constructivist learning theory. The article then presents an empirical investigation of digital literacy development, drawing on the framework, and its proposed approach for operationalizing technology activities (whether as research constructs or instructional activities). The empirical analysis is situated in the context of an innovative educational program implementation of game design based learning for middle and high school students offered in a U.S. state, in the 2011/2012 school year. The study explores how student engagement in activities representing the 6-CLA dimensions factor, inter-correlate, change from pre- to post-program, and bring about student transfer of that engagement, from school to home environments. Findings reveal that the dimensions proposed hang together well, students change in their engagement as a result of the intervention across multiple dimensions in both school and home contexts, and at-school engagement in the dimensions contributes to at-home engagement in them (in various ways as reported). The study offers support for the proposed framework, provides some evidence of digital divide effects for the intervention, presents questions for further inquiry, and offers a conceptual and research design stake in the ground for other researchers interested in the digital literacy construct.

1. Challenges in Defining,
Designing, and
Measuring
“Digital Literacy”
Development in
Learners:
A Proposed Framework
AERA, PHL, April 2014
Rebecca Reynolds, Assistant Professor
School of Communication & Information
Dept. of Library and Information Science
Rutgers University
John Wolf, Ph.D., Instructor, NJIT

2. “Digital Literacy” in Naturalistic vs. Prescriptive Contexts
Naturalistic Phenomena in General Population
• Digital Divide, Levels 1 and 2 (Hargittai & DiMaggio, 2002; Barzilai-
Nahon, 2006)
• Important cultural, social and political activities occur in online
environments, and participation in such activities requires
digital skills and knowledge (Mossberger, Tolbert, and McNeal, 2007;
Jenkins, 2009, Hobbs, 2010; Horrigan, 2011)
• More sophisticated forms of content creation, participatory
engagement, digital knowledge and cultural benefits have
been associated with privilege: higher socio-economic status
and level of education (Pew, 2007; Hargittai and Walejko, 2008)
• Connects digital literacy to capacity for social mobility (Hargittai,
2010)

3. “Digital Literacy” in Naturalistic vs. Prescriptive Contexts
Prescriptive Instructional Contexts
• Strand 2 of the U.S. FCC’s NBP of 2010 focuses on
awareness raising and digital literacy efforts, esp. in schools
• NETP of 2010: Optimizing learning and instructional design in
US schools with technology-integrated project / problem-
based learning interventions.
• Prepare learners for 21st century work cultures
• Role-taking, epistemic learning
• Learning (engage & empower), Assessment (measure what
matters), Teaching (prepare & connect), Infrastructure (access &
enable), Productivity (redesign & transform)
• AASL’s Standards for 21st Century Learner and ISTE National
Education Technology Standards for students and teachers:
Discuss student-level dispositions and latest versions bring in
creativity and productivity

4. Definitions and Operationalizations Vary
• Example 1, scholarship: Web-use skill
(Hargittai,2005): “a user’s ability to locate content on
the web effectively and efficiently” with no definition
offered in her updated work in 2009; operationalizes
digital literacy as a user’s familiarity with terminology
reflecting certain technology concepts
(JPEG, frames, preference settings, newsgroups, PDF, refresh/ reload,
advanced search, weblog, bookmark, bookmarklet, spyware, bcc (on
e-mail), blog, tagging, tabbed browsing, RSS, wiki, malware, social
bookmarking, pod-casting, phishing, web feeds, firewall, cache,
widget, favorites, torrent)

5. Definitions and Operationalizations Vary
• Example 2, scholarship: Barron (2004) defines
technological fluency as learners’ ability “to express
themselves creatively, to reformulate knowledge, and to
synthesize new information. [The concept] entails a process
of lifelong learning in which individuals continually apply what
they know to adapt to change and acquire more knowledge to
be more effective at applying technology to their work and
personal lives.” Operationalization: students’ self -
reported frequency of participation in the following
twelve constructive activities

6. Definitions and Operationalizations Vary
Example 3, scholarship: Eshet & Aviram
• Skeptical hypothesis 2 centers on technology
situatedness of the definition one uses of digital literacy.
• The definer must prioritize particular technology contexts of
the moment, and any definition reflects the norms and
values of the definer in assigning important to given
technologies
• “Clash of civilizations”
”Old, modern, rationalist, linear, conceptual, book-based culture of Western
societies is CLASHING with new emerging post modern, branching, multi-
media based reproduction oriented culture that has been developed in the
last 20 years largely due to the spread of technology”

7. Definitions and Operationalizations Vary
Example 1, policy: AASL
Standards: “The definition of
information literacy has become
more complex as resources and
technologies have changed.
Information literacy has
progressed from the simple
definition of using reference
resources to find information.
Multiple literacies, including
digital, visual, textual, and
technological, have now joined
information literacy as crucial
skills for this century.”

8. Definitions and Operationalizations Vary
Example 2, policy: NETS Standards

9. Definitions and Operationalizations Vary
Example 3, scholarship: Eshet & Aviram
• Conservative hypothesis in most studies rests on
assumption that digital literacy is a set of distinct skills that we
can measure in the population.
• Theoretical questions (are skills distinct or are they really something
else, like intelligence or motivation?)
• Operation-oriented questions (how to measure)
• Didactic questions (can they be taught?)
• Development and design questions (how are they best taught?)
• Skeptical hypothesis 1: No such thing as “digital literacy;” it is
orthogonal with individual level capacities:
• Motivation / Curiosity, Learning styles, Creativity, Intelligences,
Capacities, Personality types
• This hypothesis places the individual and his/her expertise at
forefront

10. Critical Issues
• Reynolds (2008): Research and policy definitions hold significant
limitations for advancement of research because of the
definitional variation, technology determinedness of
operationalizations resulting in a need for constant update, and in
policy, limited structural, material means for realizing ends
specified (e.g., time, curricula, budgets).
• Yet, given Digital Divide, and learning opportunities
technology poses, the need for intervention clearly remains.
• To design, implement and study processes and effects of
interventions that inculcate “digital literacy” in learners requires
clearly defining learning objectives and outcomes for the
intervention, and developing the appropriate pedagogy.
• Eshet & Aviram’s skeptical hypotheses advise (a) a focus on
individual expertise, and (b) problematization of technology
situatedness

11. Critical Issues
• To take it 1 step further, a learning sciences approach to
designing digital literacy intervention requires theoretical
grounding of the instructional design principles

12. This Study
• The definition and framework for digital literacy that follows
emerged out of an existing learning innovation called
Globaloria that applies social cognitive theory (e.g.,
Vygotsky) and Constructionism (e.g., Papert & Harel, 1991)
in its design.
• Reynolds & Harel Caperton 2009 outline the theoretical linkages
between G’s instructional design principles and Constructionism
• Reynolds & Hmelo-Silver (2013) identify conceptual linkages
among Globaloria’s design, Constructionism and the knowledge
building literature of Scardamalia & Bereiter.
• The definition of “Contemporary Learning Abilities” we
propose is student centered, and clearly distinguishes
between the abilities and the technology contexts out of
which they may emerge.

13. 1
2
3
4
Contemporary Learning Ability
(CLA)
Practices Representing Each CLA & How They Are Articulated/Integrated in Globaloria
Invention, creation and completion
of a digital project stemming from
an original idea
Brainstorming and developing game and simulation ideas and storylines using Web2.0 tools
Writing an original game narrative and a proposal to explain it
Generating creative ideas for designs to express the subject of the game and the user experience
Planning/programming/completing a game demo that illustrates the original game design and
functionality
Developing knowledge of the game's domain or topic through game invention/creation/research
Project planning, project
management, teamwork (e.g., role-
taking,
task delegation),
problem-solving
Coordinating and managing the design/creation/programming of game elements
Managing the project’s execution by creating/organizing a wiki and by sharing project assets and
progress updates
Managing team work by defining and assigning team roles/coordinating tasks/executing roles
Project troubleshooting for self and others
Gaining leadership experience through the project management of all game production elements
Publishing and distribution of
self-created digital media artifacts
to an audience and/or community
of peers
Creating a wiki profile page and project pages
Integrating and publishing text/video/photos/audio/programming code/animations/digital designs
on wiki pages
Posting completed assignments/game design iteration and assets/notes and reflections about
projects to wiki
Developing a blog
Giving and getting feedback
about project through
social interaction,
participation, exchange
Collaborating by using Web2.0 tools such as posting to wikis/blogs/open source help
forums/instant messaging
Exchanging/sharing feedback and resources by posting information/links/source code questions/
answers
Reading and commenting on others’ blogs and wiki pages
Presenting final digital projects for others both virtually in game galleries and in person in live
game demonstrations

14. 5
6 Figure 1. Globaloria learning objectives: Promote development of six contemporary learning abilities (CLAs)
Inquiry, information-seeking,
agentive
use of resources (human and
text/digital content), to support
the artifact’s topic/message,
and design/execution
Searching the Web for answers and help on specific issues related to programming games
Searching and finding resources on MyGLife.org network, website, and wiki
Searching the Web for new Flash design, animation and programming resources
Searching for information in support of the game’s educational subject and storyline
Surfing, experimentation and
play with existing Web
applications and tools
Surfing to MyGLife.org starter kit site and other game sites and playing games online
Keeping track of and bookmarking surfing results that are relevant to projects
Browsing Web2.0 content sites such as YouTube, Flickr, blogs, Google tools

15. Constructionist, knowledge-building conditions supporting development
of 6-CLAs (Reynolds & Harel Caperton 2009; Reynolds & Hmelo-Silver,
2013)

16. This Study
Building evidence base for the six dimensions of the CLAs
Research Questions and Hypotheses
RQ1: What is the empirical relationship among the six
proposed dimensions of contemporary learning?
RQ2: How does the introduction of Globaloria intervention in
school affect frequency of engagement in CLA-based activities
and practices in non-school settings?
H1: The intervention will have a larger effect on frequency of
engagement for the more constructionist dimensions of
contemporary learning.

17. INTERVENTION: Guided discovery-based game design program
and curriculum offered by the WorldWide Workshop. MS, HS
teachers and students gain experience and expertise in a range of
agentive digital practices.

18. Learning Management System
as Information System

19. Learning Management System
as Information System

20. Methods
Non-experimental pre/post survey design, 2011/2012
school year of Globaloria
• Out of 1,356 middle and high school student participants, a
total of 1,063 completed the pre-survey (78.4%), and
approximately 670 completed the post-survey (49.4%).
• Sample includes pre/post only students: 282 middle school
and 781 high school students participated.
• The pre-intervention sample was 29.0% female and 71.0%
male.

21. Methods
Operationalization of Digital Literacy:
• Self-reported frequency of engagement in twenty different
Globaloria activities, mapped to the CLAs
• Improvements: Definition and operationalization of digital
literacy are theoretically grounded in the social cognitivist,
Constructionist and knowledge-building inspired instructional
design features
• That is . . . The chosen technology activities included in
our operationalization of digital literacy reflect the
theory that inspired the instructional technology
context’s design
About this, we are explicit.

22. Results
• Factor analysis, reliability analysis indicate that the given survey
items for each factor hang together as 6 factors (note, all CLAs
had 3 or more items, except the Publish factor (CLA 3) which had
only 1 item).
• Bivariate correlational analyses indicate that at school, pre-
program results indicated the strongest apparent correlations
among the more constructionist dimensions of contemporary
learning (CLAs 1-3)
• Repeated measures ANOVAs indicate that students increased in
their frequency of engagement in all CLA dimensions @School
from pre- to post-program except Surfing (CLA 6)
• Repeated measures ANOVAs indicate that students increased in
their frequency of engagement in CLA dimensions 1,2, 3 and 5
(Create, Manage, Publish, Research) @Home from pre- to post-
program

23. Regression Model Results
• We ran multiple regression models investigating how
change in program engagement in all 6 CLA dimensions
@School contributed to change in the 6 dimensions
@Home. For instance:
• Does change in Creation @School contribute to
change in Creation @Home?
• Does change in Research @School contribute to
change in Creation @Home?

24. Regression Model Results

25. Regression Model Results
• Neither Gender nor proxy for SES (parent education)
predicted change in engagement @Home (very interesting
in light of digital divide research; Globaloria factors may
mitigate digital divide factors; Reynolds & Chiu, 2013 in fact
bear this out)
• Engagement @School led to an increase in engagement
@Home, even though no homework.

26. Results
• Factors predicting change in Create @ Home:
• Change in Create @ School
• Change in Socialize @ School
• Change in Surf @ School
• Factors predicting change in Manage @ Home:
• Change in Manage @ School
• Change in Socialize @ School
• Change in Surf @ School
• Factors predicting change in Publish @ Home:
• Change in Socialize @ School
• Change in Surf @ School
• Factors predicting change in Socialize @ Home:
• Change in Socialize @ School
• Change in Surf @ School
• Factors predicting change in Research @ Home:
• Change in Research @ School
• Change in Surf @ School
• Factors predicting change in Surf @ Home:
• Change in Research @ School
• Change in Surf @ School

27. Discussion and Implications
• Claim that the practices are contributing to each other in this
social cognitivist learning experience are supported by the
findings
• Surfing, experimentation, play, along with social
engagement in the context drive transfer of
engagement to home (measure of motivation)
• For the more effortful, creative work (Creation,
Publish), experiencing those practices @School is
influencing students to engage at home, too.
• Contribution:
• Study shows how when Digital Literacy expertise
domains, technology contexts and instructional design
features are more clearly articulated and identified, it
creates opportunities to explicate factors of influence

28. Discussion and Implications
• This general learning objectives approach of digital literacy
conceptualization can support improvements in more
pragmatic and executable educational policy
recommendations, instructional design, measurement and
assessment

29. Rebecca.reynolds@gmail.com
Rutgers University website
http://comminfo.rutgers.edu/directory/rbreynol/index.html
Thanks to IMLS!
Thanks to my partners!
Globaloria.org
Worldwideworkshop.org
Thank you!