EDUCON16 "Comparing Computational Thinking Development Assessment Scores with Software Complexity Metrics" by Gregorio Robles. URJC. 11/04/16.

1. EDUCON 2016, Abu Dhabi
Comparing Computational Thinking Development
Assessment Scores with Software Complexity
Metrics
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez
jesus.moreno@programamos.es, grex@gsyc.urjc.es, mroman@edu.uned.es
KGB-L3, Universidad Rey Juan Carlos
IEEE EDUCON 2016, Abu Dhabi, April 11th 2016
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

2. EDUCON 2016, Abu Dhabi
(cc) 2016 Jes´us Moreno-Le´on, Gregorio Robles and Marcos Rom´an-Gonz´alez
Some rights reserved. This work licensed under Creative Commons
Attribution-ShareAlike License. To view a copy of full license, see
http://creativecommons.org/licenses/by-sa/3.0/ or write to
Creative Commons, 559 Nathan Abbott Way, Stanford,
California 94305, USA.
Some of the figures have been taken from the Internet
Source, and author and licence if known, is specified.
For those images, fair use applies.
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

3. EDUCON 2016, Abu Dhabi
Code to learn (I)
Logo programming language
Developed in the 1960s
Its educational impact was
intensively investigated in
the 70s and 80s
Students’ improvements in
maths (and other
disciplines) were proved
“Disappeared” from the
educational landscape since
mid-90s
Seymour Papert’s picture: jgora.net
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

4. EDUCON 2016, Abu Dhabi
Code to learn (and II)
Computational Thinking is back in town
Alice, Greenfoot, Kodu, Scratch...
Code.org, EU Code Week, Africa Code Week, ArabCode.org...
If there is no evidence showing educational impact of
programming, this resurgence of programming in schools
could disappear in a few years.
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

5. EDUCON 2016, Abu Dhabi
Dr. Scratch: Analysis of CT skills
Measuring logic development skills
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

6. EDUCON 2016, Abu Dhabi
Dr. Scratch: Analysis of CT skills
Dr. Scratch allows learners to evaluate their projects to
receive a computational thinking score
Gamified feedback with tips and tricks
The computational thinking score ranges from 0 to 21 points
It is based on the degree of development of different
dimensions of the computational thinking competence:
1 abstraction and problem decomposition
2 logical thinking
3 synchronization
4 parallelism
5 algorithmic notions of flow control
6 user interactivity
7 and data representation
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

7. EDUCON 2016, Abu Dhabi
Dr. Scratch: Analysis of CT skills
Example Dr. Scratch feedback: http://www.drscratch.org
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

8. EDUCON 2016, Abu Dhabi
Classic Software Complexity Metrics
Cyclomatic Complexity (CC)
is a graph-theoretic complexity measure that can be used to
manage and control program complexity.
based on the number of linear independent paths in a program,
and can be used to establish the number of test cases in the
basis path testing methodology.
Halstead’s metrics
identifiy certain properties of a program that can be measured
and the relationships between them to assess software
complexity.
widely used in software engineering to estimate maintenance
efforts and guide software testing by identifying complex, hard
to maintain modules.
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

9. EDUCON 2016, Abu Dhabi
Example Scratch program
Figure: Example Scratch program
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

10. EDUCON 2016, Abu Dhabi
Software Metrics
Figure: Software metrics for the previous program
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

11. EDUCON 2016, Abu Dhabi
Research question
Is the Computational Thinking
score given by Dr. Scratch a
complexity value that could be
compared with the classic
software engineering metrics?
Background picture: rebel-performance.com
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

12. EDUCON 2016, Abu Dhabi
Methodology
Random selection of 25 projects of each of the following
categories:
1 stories
2 animations
3 games
4 and art creations.
Although 100 projects were downloaded, 5 of them produced
an error while being analyzed, which limits the sample size to
95 projects.
We developed a plug-in to obtain CC and Halstead from
Scratch projects
The mean score was 13.75, while both median and mode were
15.
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

13. EDUCON 2016, Abu Dhabi
Correlation between Dr. Scratch and CC and Halstead
Figure: Correlation between Dr. Scratch and CC and Halstead
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

14. EDUCON 2016, Abu Dhabi
Correlation by dimension
Figure: Correlation by dimension
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

15. EDUCON 2016, Abu Dhabi
In short...
There is correlation between Dr. Scratch CT score and
McCabe’s Cyclomatic Complexity and Halstead’s metrics
Provides validation of the complexity assessment process of
Dr. Scratch
The range of Dr. Scratch CT scores, from 0 to 21 points,
could not be flexible enough to represent the differences of
complexity for complex projects
An increment in the range of the evaluation of dimensions
could enhance correlation
But beware... not everything is (computational) complexity!
Background picture: flamingcow.co.uk
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco

16. EDUCON 2016, Abu Dhabi
Comparing Computational Thinking Development
Assessment Scores with Software Complexity
Metrics
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez
jesus.moreno@programamos.es, grex@gsyc.urjc.es, mroman@edu.uned.es
KGB-L3, Universidad Rey Juan Carlos
IEEE EDUCON 2016, Abu Dhabi, April 11th 2016
Jes´us Moreno-Le´on, Gregorio Robles, Marcos Rom´an-Gonz´alez Comparing Computational Thinking Development Assessment Sco