Phd thesis slides

1. 1/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
On the Development of Computational Thinking
Skills in Schools through Computer Programming
with Scratch
Author: Jes´us Moreno-Le´on
Director: Gregorio Robles
Doctoral Program on Information Technology and Communications
International Doctoral School
Madrid, March 15th, 2018
Jes´us Moreno-Le´on CT in schools through programming with Scratch

2. 2/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
(cc) 2018 Jes´us Moreno Le´on
Some rights reserved. This work licensed under Creative Commons
Attribution-ShareAlike License. To view a copy of full license, see
http://creativecommons.org/licenses/by-sa/4.0/ or write to
Creative Commons, 559 Nathan Abbott Way, Stanford,
California 94305, USA.
Some of the figures have been taken from the Internet
Source, and author and licence if known, is specified.
For those images, fair use applies.
Jes´us Moreno-Le´on CT in schools through programming with Scratch

3. 3/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
Motivation
Jes´us Moreno-Le´on CT in schools through programming with Scratch

4. 4/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
Coding in schools
One of the main trends in the educational landscape worldwide
Code to learn vs learn to code
Huge disparity of criteria in terms of target ages, approaches,
methodologies...
Lack of standarization due to the lack of research in this field
Urgent attention from academia is required: assessment,
transference and affecting factors
Jes´us Moreno-Le´on CT in schools through programming with Scratch

5. 5/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
Five broad categories of technologies
Unplugged
Arrow-based visual environments
Block-based visual environments
Textual programming languages
Connected with the physical world
Jes´us Moreno-Le´on CT in schools through programming with Scratch

6. 6/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
Which technologies are using educators at schools?
Figure: Programming languages taught at primary and middle schools
(blue) and secondary/high schools (gray) [Sentance, 2015].
Jes´us Moreno-Le´on CT in schools through programming with Scratch

7. 7/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
Scratch statistics
Jes´us Moreno-Le´on CT in schools through programming with Scratch

8. 8/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
The pioneers
Logo programming language
Developed in the 1960s
Millions of students learnt to
program at school during
the 1970s and 80s
“Disappeared” from the
educational landscape since
mid-90s
Seymour Papert’s picture: jgora.net
Jes´us Moreno-Le´on CT in schools through programming with Scratch

9. 9/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Computer programming and CT in schools
Tools to introduce CT in education
It’s all just a little bit of history repeating?
Worrying signs from England
“Only a small fraction of pupils choose, or have the option to
choose, to take qualifications at GCSE (5.5%) or A level
(1.7%)”
“teachers just do not have the knowledge to teach this
subject”
“computer science could become a niche subject, taught in
only a few schools”
Jes´us Moreno-Le´on CT in schools through programming with Scratch

10. 10/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Goals of this thesis
Jes´us Moreno-Le´on CT in schools through programming with Scratch

11. 11/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
The three pillars of the thesis
Jes´us Moreno-Le´on CT in schools through programming with Scratch

12. 12/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Research questions
1 Is it possible to automatically assess the development of
CT skills by inspecting learners’ Scratch creations?
2 Does the development of CT skills through programming
activities with Scratch enhance the learning of other
subjects?
3 Are there social and non-cognitive factors affecting the
development of CT skills when programming with
Scratch?
Jes´us Moreno-Le´on CT in schools through programming with Scratch

13. 13/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Background
Jes´us Moreno-Le´on CT in schools through programming with Scratch

14. 14/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Seymour Papert’s Mindstorms
“[...] Their visions of how to integrate computational thinking
into everyday life was insufficiently developed. But there will be
more tries, and more and more. And eventually, somewhere, all the
pieces will come together and it will catch.”
Jes´us Moreno-Le´on CT in schools through programming with Scratch

15. 15/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Wing’s seminal paper
“CT involves solving problems, designing systems, and
understanding human behavior, by drawing on the concepts
fundamental to computer science.”
Jes´us Moreno-Le´on CT in schools through programming with Scratch

16. 16/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Text network analysis of CT definitions: main topics and
influential keywords
Jes´us Moreno-Le´on CT in schools through programming with Scratch

17. 17/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Text network analysis of CT definitions: influential
communities of words
Cluster Words in the context Color
1 problem, process, solve, formulate SpringGreen
2 computer, concept, approach, tool Blue
3 abstraction, simulation, information, data Lime
4 thinking, decomposition, logic, debugging Fuchsia
5 algorithm, perform, systematically, efficiently Yellow
Jes´us Moreno-Le´on CT in schools through programming with Scratch

18. 18/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Categories of CT assessment tools
Summative
Formative Skill-transfer Attitudes Vocabulary
Aptitudinal Content
CT-test Fairy A. REACT Bebras CTS Language
CA test Concepts Quizly CTP-Quiz
MBPA Israel NCV CTSiM
Practices
Jes´us Moreno-Le´on CT in schools through programming with Scratch

19. 19/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Summative-aptitudinal CT assessment tools
Figure: Sample of a CT-test item [Rom´an-Gonz´alez, 2015].
Jes´us Moreno-Le´on CT in schools through programming with Scratch

20. 20/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Formative, iterative CT assessment tools
Figure: CTP-Graph of a Centipede game [Koh et al., 2010].
Jes´us Moreno-Le´on CT in schools through programming with Scratch

21. 21/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Formative, iterative CT assessment tools
Tool Language Audience State
REACT Agent Sheets Teachers In production
Quizly App Inventor Teachers and students In production
NCV Scratch Students Under evaluation
Practices Blockly Students In development
Jes´us Moreno-Le´on CT in schools through programming with Scratch

22. 22/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Transference to other domains
In Mindstorms, Papert posited that programming activities
with Logo would help students develop abilities that they
would be able to transfer to other non-coding scenarios
The impact of programming on learning other subjects was
intensively investigated during the 1980s and early 1990s
with promising results
Jes´us Moreno-Le´on CT in schools through programming with Scratch

23. 23/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Different tools, different results?
New visual languages aim to overcome the limitations of
textual languages:
Syntax difficulties
Lack of variety of products
Solitude of the learning context
A new wave of research should be carried out to check if
previous results are replicated and to provide evidence on
emerging questions
Jes´us Moreno-Le´on CT in schools through programming with Scratch

24. 24/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Lonely and isolated programmers?
Programmers do not like social interactions
Relationships between programming abilities and
introversion (1980s and 90s)
Social software development sites like GitHub
and GitLab
New research proves that extroversion and social
participation correlate with programming skills
Background picture: pogdesign.co.uk
Jes´us Moreno-Le´on CT in schools through programming with Scratch

25. 25/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Collaboration in Scratch
“They give feedback through comments on projects, they work
together on joint projects, they remix one another’s projects,
they crowd-source artwork for their projects, they create tutorials
to share their knowledge with one another”[Resnick, 2012]
Jes´us Moreno-Le´on CT in schools through programming with Scratch

26. 26/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
Effects of participating in the community
Positive consequences that partnership produces on the
elaboration of the projects created by young learners (3 cases)
Collaboration promoted an enhancement of the quality of the
projects (5 cases)
New research making use of much larger samples is required
to state stronger conclusions in this regard
Jes´us Moreno-Le´on CT in schools through programming with Scratch

27. 27/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
What is CT?
On the quest for assessing CT
Code to learn?
Social factors of computational thinking
The non-cognitive side of CT
Some CT definitions introduce non-cognitive aspects as an
important part of these skills
The nomological network of CT has not been totally
established
New investigations must study whether and which
non-cognitive factors underlie CT aiming to optimize its
development in education
Jes´us Moreno-Le´on CT in schools through programming with Scratch

28. 28/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Methodology and results
Jes´us Moreno-Le´on CT in schools through programming with Scratch

29. 29/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Methodology
Jes´us Moreno-Le´on CT in schools through programming with Scratch

30. 30/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
A formative assessment tool for Scratch projects
Could we create a lint-like tool to support learners and
educators?
Dr. Scratch, inspired by Scrape and is based on Hairball
Jes´us Moreno-Le´on CT in schools through programming with Scratch

31. 31/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Free software - standing on the shoulders of giants
Total Physical Source Lines of Code 1,809
Dominant language Python
Schedule Estimate, Months 4.42
Estimated Average Number of Developers 1.01
Total Estimated Cost to Develop $50,345
Table: Estimated effort for the development of the Hairball fork (using
the basic COCOMO estimation model [Boehm et al., 2000]).
Jes´us Moreno-Le´on CT in schools through programming with Scratch

32. 32/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
First step: extending the features of Hairball
Plug-ins to detect bad programming habits
1 Characters that use the default, non-meaningful
name that Scratch assigns to new objects
2 Repetition of code
Jes´us Moreno-Le´on CT in schools through programming with Scratch

33. 33/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Bad/default naming of sprites
Jes´us Moreno-Le´on CT in schools through programming with Scratch

34. 34/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Repetition of code
Example of repeated code Solution to avoid repeated code
Blocks should be created to
avoid repetition of code
Jes´us Moreno-Le´on CT in schools through programming with Scratch

35. 35/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Scratch projects repository analysis
Default names Duplicated scripts Defined blocks
Projects 79 62 17
Mean 5.94 7.23 1.11
Median 3 2 0
Maximum 67 71 25
Table: Analysis of 100 ramdonly downloaded Scratch projects
Jes´us Moreno-Le´on CT in schools through programming with Scratch

36. 36/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Second step: development of the CT assessment feature
Remixing other researchers’ ideas
New frameworks for studying and assessing the development
of CT [Brennan and Resnick, 2012].
Progression of Early CT Model [Seiter and Foreman, 2013].
Evaluation of games to gauge understanding of programming
concepts [Wilson et al., 2012].
Towards the Automatic Recognition of CT for Adaptive Visual
Language Learning [Koh et al., 2010].
Jes´us Moreno-Le´on CT in schools through programming with Scratch

37. 37/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Assessment of CT
CT dimension Basic Developing Proficient
Data representation modifiers of sprites
properties
operations on vars operations on lists
Logical Thinking if if else logic operations
User interactivity green flag key pressed, sprite
clicked, ask and wait,
mouse blocks
when %s is >%s,
video, audio
Algotithmic notions
of flow control
sequence of blocks repeat, forever repeat until
Abstraction and
problem decomposi-
tion
more than one script
and more than one
sprite
def block when I start as clone
Parallelism Two scripts on green
flag
Two scripts on key
pressed, two scripts
on sprite clicked on
the same sprite
Two scripts on when
I receive message,
two scripts when %s
is >%s, two scripts
on when backdrop
change to
Synchronization wait Broadcast, when I re-
ceive message, stop
all, stop program,
stop programs sprite
wait until, when
backdrop change to,
broadcast and wait
Table: Level of development for each CT dimension
Jes´us Moreno-Le´on CT in schools through programming with Scratch

38. 38/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Assessment of CT: Logical Thinking
Different levels of development of logical thinking: basic (top),
developing (center) and proficient (bottom).
Jes´us Moreno-Le´on CT in schools through programming with Scratch

39. 39/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Assessment of CT: an example
Evaluation of Catch me if you can (available at
https://scratch.mit.edu/projects/138397021/)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

40. 40/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Assessment of CT: an example
Evaluation of Catch me if you can (available at
https://scratch.mit.edu/projects/138397021/)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

41. 41/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Web-based service: Dr. Scratch
Alpha version of the tool, presented during the Scratch Conference
2014 at the MIT (Cambridge, MA, USA)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

42. 42/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Web-based service: Dr. Scratch
Present version of the tool, developed after the Google RISE
Award 2015
Jes´us Moreno-Le´on CT in schools through programming with Scratch

43. 43/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Web-based service: Dr. Scratch
Feedback report
Jes´us Moreno-Le´on CT in schools through programming with Scratch

44. 44/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Web-based service: Dr. Scratch
Ideas to improve the score
Jes´us Moreno-Le´on CT in schools through programming with Scratch

45. 45/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process
Ecological validity
Convergent validity
Discriminant validity
Jes´us Moreno-Le´on CT in schools through programming with Scratch

46. 46/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Ecological Validity (I)
Ecological validity
Are young learners able to analyze their projects
and independently learn from the tips that the
tool provides?
Workshops with over 100 students (10 to 14
years) in 8 schools
Jes´us Moreno-Le´on CT in schools through programming with Scratch

47. 47/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Ecological Validity (II)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

48. 48/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Ecological Validity (and III)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

49. 49/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Convergent Validity (I)
Convergent validity
Comparison of the evaluations provided by Dr.
Scratch with other measurements of similar
constructs
(Human) expert evaluators
Software engineering complexity metrics
Background picture: Joanna Bourne
Jes´us Moreno-Le´on CT in schools through programming with Scratch

50. 50/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Convergent Validity (II)
Comparison with human experts
Programming contest for students (Google,
FECyT)
Jury: 16 experts in computer science education
4 groups of evaluators, average years of
experience with Scratch: 3.5 to 4 years
53 projects, 450 evaluations
Jes´us Moreno-Le´on CT in schools through programming with Scratch

51. 51/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Convergent Validity (III)
Scatter plot for experts evaluation (x-axis) and Dr. Scratch
assessment (y-axis).
Jes´us Moreno-Le´on CT in schools through programming with Scratch

52. 52/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Convergent Validity (IV)
Metric Value
Cyclomatic complexity 3
Vocabulary 14
Length 14
Volume 66.42
Difficulty 53.30
Effort 293.86
Implementation of CC and Halstead’s analyzer for Scratch
projects
Jes´us Moreno-Le´on CT in schools through programming with Scratch

53. 53/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Convergent Validity (V)
Scatter plot for Dr. Scratch assessment (x-axis) and Cyclomatic
Complexity (y-axis).
Jes´us Moreno-Le´on CT in schools through programming with Scratch

54. 54/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Convergent Validity (and VI)
Scatter plot for Dr. Scratch assessment (x-axis) and Halstead’s
vocabulary (y-axis).
Jes´us Moreno-Le´on CT in schools through programming with Scratch

55. 55/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Discriminant Validity (I)
Discriminant validity
Projects shared in the Scratch repository are
categorized under one or more project types:
games, animations, music, art and stories.
Is Dr. Scratch able to detect differences in the
CT dimensions developed when programming
different types of projects?
Background picture: Bodie Pyndus
Jes´us Moreno-Le´on CT in schools through programming with Scratch

56. 56/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Discriminant Validity (II)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

57. 57/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Validation Process: Discriminant Validity (and III)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

58. 58/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Methodology
Jes´us Moreno-Le´on CT in schools through programming with Scratch

59. 59/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the English classroom (I)
The study
Quasi-Experimental Design
65 students, 4th and 5th graders
Control groups and experimental groups
Pre and Post tests
Surveys
Jes´us Moreno-Le´on CT in schools through programming with Scratch

60. 60/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the English classroom (II)
Scratch helped me to learn
English
Scratch made me want to learn
more English
Jes´us Moreno-Le´on CT in schools through programming with Scratch

61. 61/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the English classroom (III)
Experimental group Control group
Initial test 5.05 5.13
Final test 7.7 7.55
Improvement 2.65 2.42
Table: Pre-test and Post-test results
Jes´us Moreno-Le´on CT in schools through programming with Scratch

62. 62/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the English classroom (and IV)
4th grade
Teacher with coding experience
5th grade
First-time programmer teacher
Jes´us Moreno-Le´on CT in schools through programming with Scratch

63. 63/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the Maths classroom (I)
The study
Quasi-Experimental Design
42 students, 6th grade
Control groups and experimental groups
Pre and Post tests
3 months
Jes´us Moreno-Le´on CT in schools through programming with Scratch

64. 64/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the Maths classroom (II)
Mathematical processes investigated
Modeling of processes and reality phenomena
Reasoning (making predictions and justifying
arguments)
Problem formulation and problem solving
Exercising (comparison and execution of
procedures and algorithms)
Following guidelines set by the Ministry of
National Education of Colombia
Jes´us Moreno-Le´on CT in schools through programming with Scratch

65. 65/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the Maths classroom (III)
Pre-test. Mean and deviation for control (left) and experimental
(right) groups.
Jes´us Moreno-Le´on CT in schools through programming with Scratch

66. 66/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the Maths classroom (IV)
Post-test. Mean and deviation for control (left) and experimental
(right) groups.
Jes´us Moreno-Le´on CT in schools through programming with Scratch

67. 67/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Coding in the Maths classroom (and V)
Post-test comparison. Means obtained for the control and
experimental groups by mathematical process.
Jes´us Moreno-Le´on CT in schools through programming with Scratch

68. 68/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Comparing three quasi-experimental research designs (I)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

69. 69/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Comparing three quasi-experimental research designs (II)
School Grade n (control) n (exp.) Subject
Coraz´on de Mar´ıa 6th 26 25 Math
La Jota 6th 25 24 Social studies
La Inmaculada 2nd 15 14 Language arts
Jes´us Moreno-Le´on CT in schools through programming with Scratch

70. 70/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Comparing three quasi-experimental research designs (III)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

71. 71/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Comparing three quasi-experimental research designs (IV)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

72. 72/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Comparing three quasi-experimental research designs (V)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

73. 73/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Comparing three quasi-experimental research designs (and
VI)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

74. 74/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Methodology
Jes´us Moreno-Le´on CT in schools through programming with Scratch

75. 75/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (I)
Data set of public activity in the Scratch online community
during its first five years of activity (2007–2012)
1,056,951 users
1,928,699 projects
120,097 galleries
1,313,200 friends
7,788,414 comments in projects
Background picture: xkl
Jes´us Moreno-Le´on CT in schools through programming with Scratch

76. 76/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (II)
Measuring the sophistication of the projects from three points
of view:
Breadth: range of different features that programmers use
Depth: amount with which programmers use those features
Finesse: user’s ability to solve programming problems
effectively and creatively
Jes´us Moreno-Le´on CT in schools through programming with Scratch

77. 77/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (III)
Sociability is measured as the sum of the social actions:
number of favorites, galleries, friends and comments
Jes´us Moreno-Le´on CT in schools through programming with Scratch

78. 78/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (IV)
4 groups of users using the quartiles of the the number of days
that they spend in the community
Jes´us Moreno-Le´on CT in schools through programming with Scratch

79. 79/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (V)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

80. 80/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (VI)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

81. 81/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
How socialization affects the learning of coding skills (VII)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

82. 82/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Non-cognitive factors of CT (I)
The study
Correlations between CT, self-efficacy and the dimensions of
the Big Five model of personality: Openness to Experience,
Conscientiousness, Extroversion, Agreeableness, and
Neuroticism.
1,251 students from 24 schools, enrolled in CS from 5th to
10th grade
CTt [Rom´an-Gonz´alez, 2015] and some additional self-efficacy
items
99 students took the Big Five Questionnaire-Children
version [Barbaranelli et al., 2003]
Jes´us Moreno-Le´on CT in schools through programming with Scratch

83. 83/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Non-cognitive factors of CT (II)
As expected, positive correlations with Openness and
Conscientiousness
Unexpectedly, positive correlation with Extraversion
Positive correlation with CT-Self efficacy (medium size
difference in favor of males)
Jes´us Moreno-Le´on CT in schools through programming with Scratch

84. 84/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Non-cognitive factors of CT (and III)
Nomological network of CT including cognitive and non-cognitive factors
Jes´us Moreno-Le´on CT in schools through programming with Scratch

85. 85/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
Development and validation of Dr. Scratch
CT across the curriculum
Social and non-cognitive factors of CT
Summary of results
Jes´us Moreno-Le´on CT in schools through programming with Scratch

86. 86/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
An assessment tool for the community
Evidence for educators and policy makers
Future research
Conclusions
Jes´us Moreno-Le´on CT in schools through programming with Scratch

87. 87/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
An assessment tool for the community
Evidence for educators and policy makers
Future research
Who is using Dr. Scratch?
Learners and educators from all over the world: over 214,000
analyzed projects since August 2015
Scholars:
To analyze students’ performance
To assess teachers’ CT and programming skills
To find code smells and quality issues
To validate other tools
As part of an evaluation framework
Jes´us Moreno-Le´on CT in schools through programming with Scratch

88. 88/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
An assessment tool for the community
Evidence for educators and policy makers
Future research
Transference of CT across the K-12 curriculum
School level: differences based on the age of the students
School subject: simple vs complex subjects
Teacher training: great impact
Jes´us Moreno-Le´on CT in schools through programming with Scratch

89. 89/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
An assessment tool for the community
Evidence for educators and policy makers
Future research
Social and non-cognitive factors affecting CT
Educators could choose between platforms for learning to
program based on their social and software evolution features
The one-size-fits-all approach is leaving some students,
especially adolescent girls, behind
A diversity of computing contexts must be offered
Jes´us Moreno-Le´on CT in schools through programming with Scratch

90. 90/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
An assessment tool for the community
Evidence for educators and policy makers
Future research
Future Work (I)
Validation of Dr. Scratch:
Factorial validity. Simplification of the feedback report?
Face validity. Survey for over 500 educators?
Integration of CT across the curriculum: more research is
needed to state stronger conclusions
Social factors affecting programming and CT: a controlled
experiment is needed to consider causality
Background picture: Simon Cunningham
Jes´us Moreno-Le´on CT in schools through programming with Scratch

91. 91/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
An assessment tool for the community
Evidence for educators and policy makers
Future research
Future Work (and II)
Background picture: mountainmadness.com
Jes´us Moreno-Le´on CT in schools through programming with Scratch

92. 92/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
On the Development of Computational Thinking
Skills in Schools through Computer Programming
with Scratch
Jes´us Moreno-Le´on
Doctoral Program on Information Technology and Communications
International Doctoral School
Madrid, March 15th, 2018
Jes´us Moreno-Le´on CT in schools through programming with Scratch

93. 93/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
References I
Barbaranelli, C., Caprara, G. V., Rabasca, A., and Pastorelli, C. (2003).
A questionnaire for measuring the big five in late childhood.
Personality and Individual Differences, 34(4):645–664.
Boehm, B. W., Madachy, R., Steece, B., et al. (2000).
Software cost estimation with Cocomo II with Cdrom.
Prentice Hall PTR.
Brennan, K. and Resnick, M. (2012).
New frameworks for studying and assessing the development of computational thinking.
In Proceedings of the 2012 annual meeting of the American Educational Research Association, Vancouver,
Canada, pages 1–25.
Koh, K. H., Basawapatna, A., Bennett, V., and Repenning, A. (2010).
Towards the automatic recognition of computational thinking for adaptive visual language learning.
In 2010 IEEE Symposium on Visual Languages and Human-Centric Computing, pages 59–66.
Resnick, M. (2012).
Mother’s day, warrior cats, and digital fluency: Stories from the Scratch online community.
In Proceedings of the Constructionism 2012 Conference: Theory, Practice and Impact, pages 52–58.
Rom´an-Gonz´alez, M. (2015).
Computational thinking test: Design guidelines and content validation.
In Proceedings of the 7th Annual International Conference on Education and New Learning Technologies
(EDULEARN 2015), pages 2436–2444.
Jes´us Moreno-Le´on CT in schools through programming with Scratch

94. 94/94
Motivation
Goals of this thesis
Background
Methodology and results
Conclusions
References II
Seiter, L. and Foreman, B. (2013).
Modeling the learning progressions of computational thinking of primary grade students.
In Proceedings of the Ninth Annual International ACM Conference on International Computing Education
Research, ICER ’13, pages 59–66, New York, NY, USA. ACM.
Sentance, S. (2015).
Annual national survey 2015: Results.
Technical report, Computing At School.
Wilson, A., Hainey, T., and Connolly, T. (2012).
Evaluation of computer games developed by primary school children to gauge understanding of
programming concepts.
In European Conference on Games Based Learning, page 549. Academic Conferences International Limited.
Jes´us Moreno-Le´on CT in schools through programming with Scratch