Assessing digital literacy levels: the design of an instrument
1. Assessing digital literacy levels:
the design of an instrument
AILA 2014
Brisbane – Australia -
August
Rosinda de Castro Guerra
Ramos
F. Katherine Asega
2. This research aims to investigate how to
measure the level of Digital Literacy (DL) of
in-service English teachers in Brazil in order to
promote their digital inclusion.
RESEARCH OBJECTIVE
3. MOTIVATION
SOCIAL:
Changes in society.
Transition between "old
and new, real and virtual,
the printed and digital"
(Buzato, 2006) requires a
set of competences
related to Technologies.
Educating efficient digital citizens.
4. MOTIVATION
Three reasons to include ICT
in education:
1. Potential benefits of ICT for
teaching and learning.
2. The need to acquire Digital
Competence to be functional
in our knowledge society
(Eshet-Alkalai, 2004).
3. Digital inclusion depends more
on knowledge and skills than
on access and use (Erstad,
2010b).
EDUCATIONAL:
Integration of ICT in practice.
Educating efficient digital citizens.
5. MOTIVATION
PROFESSIONAL:
The need for teacher development.
Educating efficient digital citizens.
• To become digitally literate.
• To help students to become
collaborative, problem
solving, creative learners
through using ICT.
BUT...
What level of digital literacy
should teachers have to
make a more appropriate
use of ICTs in practice?
6. Transition between "old and new, real and virtual,
the printed and digital" (Buzato, 2006) requires a set of
competences related to Technologies.
Three reasons to include ICT in education:
1. Potential benefits of ICT for teaching and learning.
2. The need to acquire Digital Competence to be functional
in our knowledge society (Eshet-Alkalai, 2004).
3. Digital inclusion depends more on knowledge and skills
than on access and use (Erstad, 2010b).
SOCIAL REASONS
7. Our objective is to present
the process of designing
an assessment instrument
to evaluate
the level of DL
of Brazilian English Teachers in-service.
PRESENTATION OBJECTIVE
8. Digital Literacy and its concepts.
Ala-Mutka, K. (2011); Bawden, D. (2001); Bawden, D. (2008);
Erstad, O. (2010b); Eshet-Alkalai, Y. (2004); Eshet-Alkalai, Y., &
Chajut, E. (2010); European Commission (2013); European
Commission (2012); Martin, A. (2006); Martin, A., & Grudziecki,
J. (2006); UNESCO (2011).
THEORETICAL FOUNDATION
Digital Competence.
Ala-Mutka, K. (2011); Bawden, D. (2001); (Eshet-
Alkalai, 2004). European Commission (2013);
European Commission (2012).
Assessment.
Bachman & Palmer (1996).
9. “Digital Literacy is the awareness, attitude and ability
of individuals to appropriately use digital tools and
facilities to identify, access, manage, integrate,
evaluate, analyze and synthesize digital resources,
construct new knowledge, create media expressions,
and communicate with others, in the context of specific
life situations, in order to enable constructive social
action; and to reflect upon this process.”
(Martin & Grudziecki, 2006).
THEORETICAL FOUNDATION
Digital Literacy Concept
10. "Digital Competence involves the confident and critical
use of Information Society Technology (IST) for work,
leisure and communication. It is underpinned by basic
skills in ICT: the use of computers to retrieve, assess,
store, produce, present and exchange information, and
to communicate and participate in collaborative
networks via the Internet."
(European Parliament and the Council, 2006).
THEORETICAL FOUNDATION
Digital Competence Concept
11. • The DIGCOMP study was launched by European
Commission's Joint Research Centre (JRC) - Prospective
Technological Studies (IPTS).
• Carried out between January 2011 and December 2012.
• AIM: to identify descriptors and develop conceptual
framework of Digital Competence.
• RESULTS: two different interrelated outputs - a self-
assessment grid and a framework.
THEORETICAL FOUNDATION
Digital Competence Project (DIGCOMP)
12. Self-Assessment Grid
• Proposes the areas of Digital
Competence and descriptors
for three proficiency levels.
• It can be used as a tool to:
– identify the level of digital
competence of each
citizen;
– to understand how to
improve digital
competence.
THEORETICAL FOUNDATION
Framework
• Identifies (for each area) the
related competences.
• Provides descriptors on three
levels and examples
knowledge, attitudes and
skills.
• It can be used by curricula and
developers to develop the
digital competence of a
specific target group.
Digital Competence Project (DIGCOMP)
15. DIGITAL LITERACY TEST DESIGN
Test
Design
Putting into
Practice - Stage 3
Conceptual
Mapping
Browsing
Assessment
Conceptual
Mapping
Browsing
Digital Literacy
Available
Tests
Checking and
comparing
Conceptual
Mapping
Looking for
Descriptors
Methodology
16. Online Proficiency Test that will
measure the Digital Literacy (DL)
considering three different levels of
knowledge, attitudes and skills: basic,
intermediate, advanced.
Digital Literacy Test description
Aims to identify the level of DL of in-service English teachers
in Brazil and inform the participants how to improve digital
competence according to the stated needs.
DIGITAL LITERACY TEST DESIGN
17. PHASE 1
Computer Literacy
TOOL FOCUS TEST
• Aims to evaluate:
- the skills to operate a variety of
computer applications
packages (word processing,
databases, spreadsheets);
- general IT skills, such as
copying disks and generating
hard-copy printout.
first step towards advanced
knowledge, skills and attitudes.
PHASE 2
Digital Competence
KNOWLEDGE, ATTITUDE AND SKILL
FOCUS TEST
• Aims to evaluate digital
competence into three levels:
basic, intermediate, advanced.
Taking into account 5 different
areas: information,
communication, content-creation,
safety and problem-solving.
DIGITAL LITERACY TEST DESIGN
General Description
18. Question Description
PHASE 1 – Computer Literacy
• Total of questions: 15
- 5 questions of Computer Basics
(computer devices and its
functions).
- 10 questions of Computer
Concept (software and
programs usage).
- Total amount of time: 7 min.
Close ended questions, choose
from a list of choice to identify the
computer skills.
Question Sample
1. Which computer devices(components)
do you know how to use?
Choose the skills that best describe your experience:
( ) Monitor
( ) Touch Screen
( ) Keyboard
( ) Mouse
( ) Touchpad
( ) CD-Rom drive
( ) USB drive
( ) External drive
( ) Printer
( ) Scanner
DIGITAL LITERACY TEST DESIGN
19. Question Description
PHASE 2 – Digital Competence
• Total of questions: 22
- 3 questions of Information: browsing, evaluating and storing.
- 6 questions of Communication: interaction, sharing, engaging,
collaborating, online norms and digital identity.
- 5 questions of Content-creation: developing content, integrating
resources, copyright, program modification.
- 4 questions of Safety: protecting devices, personal data, health,
environment.
- 4 questions of Problem solving: solving technical problems,
Identifying needs and technological responses, innovating and
creatively using technology.
• Total amount of time: 22 min.
Close ended questions, multiple
choice with questions that
represent the competence level.
DIGITAL LITERACY TEST DESIGN
20. Question Sample
PHASE 2 – Digital Competence
Safety area
DIGITAL LITERACY TEST DESIGN
Basic level
Intermediate level
Advanced level Close
Need to start from
Computer Literacy
Each and every area
will have its own
result, thought they
are related.
21. Preliminary Results - Stage 3: pilot phase
PHASE 1 – Computer Literacy
• Submitted the questions to analysis and interviewed the
participants:
- 2 different Information Technology specialists.
- 2 different English teachers (basic level of experience).
• The results:
- one specialist considered the questions obsolete due to used
terminology use and technology advances. The other suggested
some terms adjustment.
- both English Teachers know how to use the computer but they are
not familiar with the used terms.
• Conclusion: we are rewriting the questions based on the observation,
adjusting the terms and focusing more on the use of software and
programs based on daily task and use experience.
DIGITAL LITERACY TEST DESIGN
22. PHASE 2 – Digital Competence
• Will Submit the questions to:
- 2 different Information Technology specialists.
- 10 different English teachers (from basic to advanced level).
• The expected results:
- to map the level proficiency.
- to adjust questions and answers.
- to map exam time consuming.
- to experience a different online interface based on programming
and automatic results and reports.
DIGITAL LITERACY TEST DESIGN
Preliminary Results - Stage 3: pilot phase
23. FRAMEWORK
• The framework is based on the recognition that both cognitive and
technical skills are necessary for people to be functional in a digital
society.
• Target group: adults.
• It supports a vision of digital
• literacy as the ability to create, communicate and collaborate to
organise and produce
• information; to understand and apply knowledge of the functions of
ICT; to use ICT for
• thinking and learning; and to develop a critical appreciation of the
role of ICT in society.
• Target group: primary and lower secondary schools.
24. Ala-Mutka, K. (2011). Mapping Digital Competence: Towards a Conceptual
Understanding. Seville: JRC-IPTS. Retrieved from
http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=4699
Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and
developing useful language tests. Oxford: Oxford University Press.
Bawden, D. (2001). Information and digital literacies: a review of concepts. Journal
of Documentation, 57(2), 218-259.
Bawden, D. (2008). Origins and Concepts Of Digital Literacy. In C. Lankshear & M.
Knobel (Eds.), Digital Literacies: Concepts, Policies & Practices (pp. 17-32).
Erstad, O. (2010a). Conceptions of Technology Literacy and Fluency. In P. Penelope,
B. Eva & M. Barry (Eds.), International Encyclopedia of Education (pp. 34-41).
Oxford: Elsevier.
Eshet-Alkalai, Y. (2004). Digital Literacy. A Conceptual Framework for Survival Skills
in the Digital Era. Journal of Educational Multimedia & Hypermedia, 13(1), 93-106.
Eshet-Alkalai, Y., & Chajut, E. (2010). You can teach old dogs new tricks: The factors
that affect changes over time in digital literacy. Journal of Information Technology
Education, 9, 173-181.
REFERENCES
25. European Commission (2011). Mapping Digital Competence: Towards a Conceptual
Understanding. Luxembourg: Publications Office of the European Union.
European Commission (2011). Mapping Digital Competence: Towards a Conceptual
Understanding. Luxembourg: Publications Office of the European Union.
European Commission (2012). Digital Competence in Practice: An Analysis of
Frameworks. Luxembourg: Publications Office of the European Union.
European Commission (2013). DIGCOMP: A Framework for Developing and
Understanding Digital Competence in Europe.
Martin, A. (2006). Literacies for the Digital Age. In A. Martin & D. Madigan (Eds.),
Digital Literacies for Learning (pp. 3-25). London: Facet.
Martin, A., & Grudziecki, J. (2006). DigEuLit: Concepts and Tools for Digital Literacy
Development. ITALICS: Innovations in Teaching & Learning in Information &
Computer Sciences, 5(4), 246-264.
UNESCO (2008) . ICT Competency Standards for Teachers. United Nations
Educational, Scientific and Cultural Organization.
REFERENCES
Notas del editor
in order to help students to become collaborative, problem solving, creative learners through using ICT.
ICT usage is becoming more extensive across society: more people are using technologies
around the word, for more time and for different purposes. The extensiveness of use is
moreover derived by the digitalisation of society in general, as many of the activities we
undertake have a digital component. Exposure to digital tools embraces the domains of
work, learning, leisure, participation, socialisation, and consumerism. As society is becoming
digitalized, the knowledge, attitudes and skills (KAS) that are needed to be digitally
competent are becoming manifold: being digitally literate today is not restrained to the
understanding of hardware and software devices. For this reason, Digital Competence is
currently being defined by several authors as closely related to other types of literacy
(Bawden, 2001; Eshet-Alkalai, 2004; Sefton-Green, et al., 2009). The concepts of Digital
Competence and related literacies have been reviewed in the first word package of this
study (Ala-Mutka, 2011) and are synthesised in the following diagram:
in order to help students to become collaborative, problem solving, creative learners through using ICT.
in order to help students to become collaborative, problem solving, creative learners through using ICT.
The DIGCOMP proposal consists of two different interrelated outputs:
a self-assessment grid that proposes the areas of Digital Competence and descriptors for three proficiency levels;
a framework identifying, for each area, all the related competences, and providing for each competence a general description, descriptors on three levels, examples of the knowledge, attitudes and skills, and examples of applicability for different purposes.
Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful
language tests. Oxford: Oxford University Press.
Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful
language tests. Oxford: Oxford University Press.