This document discusses a study analyzing students' self-evaluations of their digital competencies and how these relate to their test results and online behavior. The study collected data from students using questionnaires, knowledge tests, Google Analytics, and Smartlook. It found a moderate negative correlation between self-evaluation and time spent on course pages, indicating students overestimate their abilities. There was a positive correlation between time spent and test scores, showing more time studying improves outcomes. Information and data literacy was deemed most important, reflecting in measures. Critical thinking and problem-solving were emphasized as key competencies beyond the framework. Overall, the study provided insights into relationships between self-perception, online behaviors, and performance.

1. DIGITAL COMPETENCE: FROM
SELF-EVALUATION TO ANALYSIS
OF STUDENTS LEARNING
BEHAVIOUR
Dr. Michal Cerny
Faculty of Education & Faculty of Arts
Masaryk University, Czech
@eduinteres
https://cernyedtech.wordpress.com/

2. Digital competence in Scopus

3. DigComp 2.1: The Digital Competence
Framework for Citizens
• 1) Information and data literacy: To articulate information needs, to locate and retrieve digital data,
information and content. To judge the relevance of the source and its content. To store, manage, and
organise digital data, information and content.
• 2) Communication and collaboration: To interact, communicate and collaborate through digital technologies
while being aware of cultural and generational diversity. To participate in society through public and private
digital services and participatory citizenship. To manage one’s digital identity and reputation.
• 3) Digital content creation: To create and edit digital content To improve and integrate information and
content into an existing body of knowledge while understanding how copyright and licences are to be
applied. To know how to give understandable instructions for a computer system.
• 4) Safety: To protect devices, content, personal data and privacy in digital environments. To protect physical
and psychological health, and to be aware of digital technologies for social well-being and social inclusion. To
be aware of the environmental impact of digital technologies and their use.
• 5) Problem solving: To identify needs and problems, and to resolve conceptual problems and problem
situations in digital environments. To use digital tools to innovate processes and products. To keep up-to-
date with the digital evolution.

4. Open online course

5. Open online course

6. Research questions
• 1) How do the students themselves assess the digital competence?
• 2) Is there a correlation between their evaluation and test results?
• 3) Is there a relationship between how they are evaluated and what is
their movement on the course pages?
• 4) What topics are interested in on the site? Reflected somehow in
the other measurable parameters?
• 5) How do students introduce a "digitally competent person"?

7. Research tool
• Google Analytics
• Smartlook
• Final questionnaires
• The self-evaluation questionnaire
• Knowledge test (final exam)

8. Data from Google Analytics illustrate
nonlinear learning

9. Google Analytics data shows the geographic
distribution of students

10. Smartlook: Heatmap shows clicks on one
selected page

11. Participants
• The first were students of bachelor's degree in Information and
Library Studies at the Faculty of Arts, Masaryk University, who studied
compulsory course VIKBA14 Digital Competence, which is included in
the third semester of the recommended study plan. In total, there
were 52 of them, 41 of whom filled the test. The course was
organized in the form of blended learning.
• The second sample was the students of the new KPI33 course The
new Digital Competence Development Course was purely e-learning
and open to all university students as an optional subject. A total of
94 students participated in the course, 60 of which were from the
Faculty of Arts,…

12. Data source
• So I have the following data sources in total:
• 111 final questionnaires with information on a digitally competent citizen.
• 110 self-evaluation assessments completed.
• 151 completed tests (KPI33 students had up to three test passes)
• Data from Google Analytics
• Data from Smartlook

13. How do students evaluate themselves as part
of their digital competencies?
TABLE 1 OF COMPETENCE STUDENTS KPI33 BY 8 (AND 0) LEVELS AND AVERAGE ON EACH COMPETENCE
COMPETENCE / LEVEL 0 1 2 3 4 5 6 7 8 AVERG.
BROWSING, SEARCHING AND FILTERING DATA, INFORMATION AND
DIGITAL CONTENT
0 1 6 9 17 8 19 4 0 4,5
EVALUATING DATA, INFORMATION AND DIGITAL CONTENT 0 1 3 6 21 10 17 5 1 4,8
MANAGING DATA, INFORMATION AND DIGITAL CONTENT 0 1 8 1 21 12 17 3 1 4,6
INTERACTING THROUGH DIGITAL TECHNOLOGIES 0 1 5 8 17 6 21 4 2 4,7
SHARING THROUGH DIGITAL TECHNOLOGIES 0 1 5 11 12 7 23 5 0 4,7
ENGAGING IN CITIZENSHIP THROUGH DIGITAL TECHNOLOGIES 3 2 12 25 6 2 11 3 0 3,5
COLLABORATING THROUGH DIGITAL TECHNOLOGIES 0 1 2 9 13 7 23 7 2 5,0
NETIQUETTE 1 0 7 16 15 8 13 4 0 4,2
MANAGING DIGITAL IDENTITY 1 1 8 11 26 3 10 4 0 4,0
DEVELOPING DIGITAL CONTENT 1 3 5 10 12 7 19 6 1 4,5
INTEGRATING AND RE-ELABORATING DIGITAL CONTENT 3 4 10 7 21 3 14 1 1 3,8
COPYRIGHT AND LICENCES 0 5 13 15 20 2 6 3 0 3,5
PROGRAMMING 14 19 9 4 6 3 4 3 2 2,3
PROTECTING DEVICES 0 7 10 17 15 7 7 1 0 3,5
PROTECTING PERSONAL DATA AND PRIVACY 1 8 7 18 12 2 12 3 1 3,7
PROTECTING HEALTH AND WELL-BEING 1 2 16 10 19 7 7 2 0 3,6
PROTECTING THE ENVIRONMENT 3 4 7 14 19 5 10 2 0 3,7
SOLVING TECHNICAL PROBLEMS 2 9 8 18 9 5 9 4 0 3,5
IDENTIFYING NEEDS AND TECHNOLOGICAL RESPONSES 1 4 16 14 14 3 9 3 0 3,5
CREATIVELY USING DIGITAL TECHNOLOGIES 0 6 18 16 7 7 6 4 0 3,4
IDENTIFYING DIGITAL COMPETENCE GAPS 1 9 11 16 17 5 5 0 0 3,2

14. How do students evaluate themselves as part
of their digital competencies?
TABLE 2 OF COMPETENCE STUDENTS VIKBA14 BY 8 (AND 0) LEVELS AND AVERAGE ON EACH COMPETENCE
COMPETENCE / LEVEL 0 1 2 3 4 5 6 7 8 AVERG
.
BROWSING, SEARCHING AND FILTERING DATA,
INFORMATION AND DIGITAL CONTENT
0 1 1 8 16 5 10 0 0 4.3
EVALUATING DATA, INFORMATION AND DIGITAL CONTENT
0 1 2 8 13 8 6 3 0 4.3
MANAGING DATA, INFORMATION AND DIGITAL CONTENT
0 0 4 8 13 4 9 3 0 4.4
INTERACTING THROUGH DIGITAL TECHNOLOGIES
0 1 3 8 12 8 7 2 0 4.3
SHARING THROUGH DIGITAL TECHNOLOGIES 1 0 6 11 6 10 3 4 0 4.0
ENGAGING IN CITIZENSHIP THROUGH DIGITAL
TECHNOLOGIES
1 3 9 9 11 3 3 1 1 3.4
COLLABORATING THROUGH DIGITAL TECHNOLOGIES
0 1 4 4 16 6 7 2 1 4.4
NETIQUETTE 0 0 4 9 13 8 4 3 0 4.2
MANAGING DIGITAL IDENTITY 0 3 3 8 13 6 5 3 0 4.0
DEVELOPING DIGITAL CONTENT 0 1 4 11 10 5 5 5 0 4.2
INTEGRATING AND RE-ELABORATING DIGITAL CONTENT
0 0 12 10 11 3 3 2 0 3.5
COPYRIGHT AND LICENCES 0 2 7 9 16 2 4 1 0 3.6
PROGRAMMING 4 12 10 5 1 0 5 3 1 2.7
PROTECTING DEVICES 0 3 10 8 8 3 7 2 0 3.7
PROTECTING PERSONAL DATA AND PRIVACY 1 1 7 12 7 3 7 3 0 3.8
PROTECTING HEALTH AND WELL-BEING 2 3 6 8 15 3 5 0 0 3.5
PROTECTING THE ENVIRONMENT 0 1 9 7 16 6 2 0 0 3.6
SOLVING TECHNICAL PROBLEMS 0 7 11 9 5 3 2 3 1 3.2
IDENTIFYING NEEDS AND TECHNOLOGICAL RESPONSES
0 4 7 13 5 2 9 1 0 3.6
CREATIVELY USING DIGITAL TECHNOLOGIES 1 2 9 10 9 4 3 3 0 3.5
IDENTIFYING DIGITAL COMPETENCE GAPS 0 4 9 5 11 5 5 2 0 3.7

15. How does this assessment relate to the test
results?
TABLE 3. SELF-ASSESSMENT DATA
COMPETENCE AREAS CORRECTNESS [%] SELF-
EVALUATIONVIKBA14
SELF-EVALUATION
KPI33
SELF-EVALUATION
AVRG.
INFORMATION AND DATA
LITERACY
65.9 4.3 4.6 4.5
COMMUNICATION AND
COLLABORATION
64.5 4.1 4.4 4.2
DIGITAL CONTENT CREATION 72.6 3.5 3.5 3.5
SAFETY 72.8 3.6 3.6 3.6
PROBLEM SOLVING 66.6 3.5 3.4 3.4

16. Is there a relationship between how they are
evaluated and what is their movement on the
course pages
• There is a -0.6 correlation between
the correct answers in the test
questions and the self-assessment,
which can be interpreted as not being
able to evaluate students adequately.
• In terms of self-learning behaviour
and web analytics, the -0.7
correlation is satisfying, which is
between self-assessment and total
time spent on the page.
• The positive correlation between
time spent on the page and the
correct answer is didactically valuable
and is relatively strong (0.6).
TABLE 4. WEB ANALYTICS, SELF-ASSESSMENT AND STUDY RESULTS
COMPETENCE AREAS CORRECTNESS
[%]
SELF-EVALUATION
AVRG.
TIME
SPENT
WITH
DOMAIN
[S]
DOMAIN
INTERACTION
TIME
SPENT ON
THE PAGE
[S]
INTERACTION
VISIT
INFORMATION AND
DATA LITERACY
65.9 4.5 931 2 747 186.2 3.1
COMMUNICATION
AND COLLABORATION
64.5 4.2 1 466 1 782 183.3 3.5
DIGITAL CONTENT
CREATION
72.6 3.5 1 171 1 975 195.2 4.0
SAFETY 72.8 3.6 1 280 1 313 213.3 2.8
PROBLEM SOLVING 66.6 3.4 1 248 638.0 208 1.4

17. What topics are interested in on the site? Are they
reflected in other measurable parameters?
TABLE 5. SORTING DOMAINS OF IMPORTANCE BY IMPORTANCE. STUDENTS VIKBA14
VIKBA14 MAX. 2 3 4 MIN. SCORE
INFORMATION AND DATA
LITERACY
19 9 9 0 1
159
COMMUNICATION AND
COLLABORATION
3 2 8 19 7
92
DIGITAL CONTENT
CREATION
0 6 2 8 23
69
SAFETY 16 15 5 3 0 161
PROBLEM SOLVING 1 7 15 9 7 103
TABLE 6. SORTING DOMAINS OF IMPORTANCE BY IMPORTANCE. STUDENTS KPI33
VIKBA14 MAX. 2 3 4 MIN. SCORE
INFORMATION AND DATA
LITERACY
22 12 5 1 0
175
COMMUNICATION AND
COLLABORATION
3 4 7 17 9
95
DIGITAL CONTENT
CREATION
0 3 4 11 22
68
SAFETY 14 12 8 4 2 152
PROBLEM SOLVING 1 9 16 7 7 110

18. What topics are interested in on the site? Are they
reflected in other measurable parameters?
TABLE 7. SORTING DOMAINS OF IMPORTANCE BY IMPORTANCE.
STANDARDIZED SCORE.
COMPLEX VIKBA14 KPI33
IMPORTANT
SCORE
INFORMATION AND DATA
LITERACY 4.0 4.3 4.1
COMMUNICATION AND
COLLABORATION 2.3 2.3 2.3
DIGITAL CONTENT
CREATION 1.7 1.7 1.7
SAFETY 4.0 3.7 3.9
PROBLEM SOLVING 2.6 2.7 2.6
• Correlations offer insight into the
possible links between the variables
being monitored. Values ​​of 0.43
reached the relationship between
self-assessment and perceived
domain importance, while a negative
correlation of mean significance can
be seen between perceived
usefulness and time spent studying
the domain (-0.48). There is only a
weak correlation of 0, 25 between
site interaction and perceived
importance. Interestingly, there is no
correlation relationship at all between
the correctness of the response and
the perceived usefulness (-0.1).

19. How do students introduce a "digitally
competent person"?
• What students perceive very
strongly is the emphasis on critical
thinking in the broadest sense.
Critical thinking, which is not part
of DigComp, has been frequently
mentioned and emphasized by
students as something necessary
for life, both in the context of
information literacy and, for
example, in assessing the impact of
technology on human life.
• It is also often bridging to the next
"meta-competence", which is the
ability to solve problems. The
students do not perceive this as a
dimension (as DigComp does), but
as a specific competence that is
general but without internal
division.
• At the same time, however,
students reflect on the framework
not as a flat, composed of partial
competencies, but as a hierarchical
one.

20. Discussion
• In terms of self-learning behaviour and web analytics, the -0.7 correlation is
exciting, which is between self-assessment and total time spent on the page.
• In other words - the stronger the students in the area, the less time they spend
on the page, their content can be expected to be easier for them to adopt.
Conversely, the positive correlation between time spent on the page and the
correct answer is didactically valuable and is relatively strong (0.6).
• It may be interpreted to mean that the time spent studying a given dimension has
a positive effect on the test result, which mostly corresponds to its construction.
• Values ​​of 0.43 achieve a relationship between self-esteem and perceived domain
importance, while a negative correlation of mean significance can be seen
between perceived usefulness and time spent studying the domain (-0.48).
• There is only a weak correlation of 0.25 between site interaction and perceived
importance. It can, therefore, be expected that there is a specific link between
perceived usefulness and interactions, albeit not strong.

21. Thanks for your attention.