This presentation reports the initial findings of a multi-year study that is surveying major and non-major students’ understanding of the different computing disciplines. This study is based on work originally conducted by Courte and Bishop-Clark from 2009, but which uses a broadened study instrument that provided additional forms of analysis. Data was collected from 199 students from a single institution who were computer science, information systems/information technology and non-major students taking a variety of introductory computing courses. Results show that undergraduate computing students are more likely to rate tasks as being better fits to computer disciplines than are their non-major (NM) peers. Uncertainty among respondents did play a large role in the results and is discussed alongside implications for teaching and further research.

1. Computing
StudentUnderstanding
oftheComputingDisciplines
! is Not a Rock Band

2. This presentation reports the initial
findings of a multi-year study that is surveying
major and non-major students’ understanding of
the different computing disciplines.
about !

3. the team
Faith-Michael Uzoka
Computer Science & Information Systems
!
Randy Connolly
Computer Science & Information Systems
Marc Schroeder
Computer Science & Information Systems
Namrata Khemka-Dolan
Computer Science & Information Systems
Janet Miller
Counselling

4. 01Background to this presentation
02The context of our study
03How we did our study
04What our study found.
05What it all means
06Some problems and conclusions
RelatedWork
Methodology
Introduction Results
Discussion
Limitations andConclusions
outline !

5. Introduction
tothestudy

6. One of the most important
achievements in computing education has been the
recognition and elaboration of the five different computing
disciplines.
introduction !
1Computer
Engineering
2Computer
Science
3Information
Systems
4Information
Technology
5Software
Engineering

7. The title of this paper refers to the
the fact that the computing disciplines should be
understood to be quite unlike the distinct roles in a
typical rock band.
introduction !

8. The computing disciplines have
considerable overlap between them.
!
Despite this overlap, universities have to
offer distinct computing degrees that
typically do not blend curricula between the
different disciplines.

9. For students, their initial understanding
of the different computing disciplines may
play a large role in how they decide which
(if any) computing program to register in.
!

10. work
related

11. This study is an extension of work by Courte
and Bishop-Clark (C&BC) and then validated in
a subsequent study by Battig and Shariq.
Courte, J. and Bishop-Clark, C. 2009. Do students differentiate between computing disciplines?
In Proceedings of the 40th ACM technical symposium on Computer science education (SIGCSE '09).
Battig, M. and Shariq, M. 2011. A Validation Study of Student Differentiation Between Computing Disciplines.
In Information Systems Education Journal. 9, 5 (October 2011).
related work !

12. In their (C&BC) study, computing and
non-computing students were asked
to associate job task descriptions with the
best disciplinary fit.
!

13. C&BC’s results suggest that students
do not always have a clear understanding of
disciplinary scopes (especially SE and IT).
!

15. Like the C&BC study, this study examines
student knowledge of the five different
computing disciplines.
methodology !

16. Unlike the C&BC study, this study tried to
capture the overlap between the computing
disciplines in the design of its survey.
!

17. In the C&BC study, students were
given 15 task descriptions and for each
task they had to indicate which of the five
disciplines was the best fit for that task.
!
Designs hardware to implement
communication systems
CE CS IS IT SE
⃝ ⃝ ⃝ ⃝ ⃝

18. The main drawback to the prior studies
was that the students had to choose a
single discipline for a task …
!
which does not capture the possibility of
overlap between the disciplines.

19. To address that drawback, our study
allowed the participants to choose how much
each task fit with each of the five disciplines.
!
X
X
X
X
X

20. Our questionnaire had demographic-
related questions and then 31
discipline/task questions.
!

21. !
1 Designs hardware to implement communication systems
2 Uses new theories to create cutting edge software
3 Builds hardware devices such as iPods
4 Is business oriented
5 Focuses on large-scale systems development
6 Integrates computer hardware and software
7 Troubleshoots and designs practical technical applications
8 Focuses on the theoretical aspects of technology
9 Combines knowledge of business and technology
10 Applies technology to solve practical problems
11 Designs testing procedures for large-scale systems
12 Selects computer systems to improve business processes
13 Applies technical knowledge for product support
14 Utilizes theory to research and design software solutions
15 Manages large scale technological projects
Our first 15 questions
were the same as the earlier C&BC study:

22. !plus 15 new tasks added by the authors
16 Develops software systems that are maintainable, reliable, efficient, and satisfy customer requirements
17 Focuses on information, and views technology as a tool for generating, processing and distributing it
18 Utilizes sound engineering practices to create computer applications
19 Provides a support role, within an organization, to help others make the best use of its technical and information
resources
20 Uses a wide range of foundational knowledge to adapt to new technologies and ideas
21 Uses technology to give a business a competitive advantage
22 Develops devices that have hardware and software in them
23 Applies mathematical and theoretical knowledge in order to compare and produce computational solutions and
choose the best one
25 Understands both technology and business, but with a focus more on the technical side
26 Uses programming skills to create or modify business solutions
27 Develops or maintains web sites
28 Manages a team of software developers
29 Manages a company’s computing department
30 Evaluates and improves the usability (user experience) of computing systems
31 Works with an organization’s data assets

23. !… and an additional task that is not typically
associated with the computing field
24
Focuses exclusively on hardware design, including digital electronics, with
little or no involvement in software design

24. The intent of the study was to find out if
relatively-inexperienced students
understood the tasks associated with
different computing disciplines, prior to
enrolment in computing courses/program.
!

25. timeline
Questionnaires were
provided across ten
sections of six
introductory computing
classes at Mount Royal
University in the Fall 2012
semester.
Note that there are two
computing programs at
MRU: a computer
science program and a
blended IS/IT program.
SEPT
Of 250 questionnaires
that were distributed, 199
questionnaires were
properly filled and coded
for analysis.
JAN
Rank ordering analysis
along with standard
statistical analysis.
!
2012 20132013
MAY

26. re tus sl

27. results !
79%
responserate

28. 25%
IS/IT
!
23%
CS
52%
Non-Major
results programofstudy

29. 59%
17%
12%
year 2
year 1
year 3
9%
4%
other
year 4
results !
yearofstudy

30. 72%
Male
results !
gender
28%
Female

31. 81%
11%
5%
< 2 Years
None
2-5 Years
3%
More than 5
results !
computingexperience

32. Rank order analysis was utilized
to determine the students’ ranking of the
disciplinary tasks relative to the five computing
disciplines.
!
A further analysis was carried out to determine the
levels of match between students’ task rankings
and the disciplinary best fit.
results rankorderanalysis

33. !
examplerankordering
Don’t Know % Level of Fit CE CS IS IT SE
15.6%
0 (No Answer) 19.6% 26.1% 25.1% 25.1% 24.1%
1 (No Fit) 2.0% 7.0% 9.5% 9.5% 20.6%
2 1.0% 10.6% 18.6% 15.1% 11.1%
3 7.5% 23.6% 21.1% 16.1% 11.1%
4 20.1% 23.1% 14.1% 20.1% 13.6%
5 (Best Fit) 49.7% 9.5% 11.6% 14.1% 19.6%
Mean 3.56 2.39 2.24 2.39 2.28
Median 4.00 3.00 2.00 3.00 2.00
Mode 5 0 0 0 0
Rank 1 3 5 2 4
Question #1 Designs hardware to implement communication systems

34. !
disciplinematchdistributions
Match Level CE CS IS IT SE
Very Accurate (5) 4 (100%) 3 (60%) 7 (78%) 5 (50%) 3 (38%)
Accurate (4) 0 (0%) 2 (40%) 1 (11%) 2 (20%) 2 (25%)
Ok (3) 0 (0%) 0 (0%) 0 (0%) 2 (20%) 2 (25%)
Fair (2) 0 (0%) 0 (0%) 1(11%) 1(10%) 1 (12%)
Poor (1) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 0 (0%)
Total Tasks 4 5 9 10 8
Limitation: not every discipline
had the same number of matching
questions

35. Analyses were conducted to determine
if there were any clear differences between the
three groups of respondents, that is, between IS/IT,
CS, and NM (non-major) students.
!
Responses to individual tasks were analyzed using
one-way ANOVAs.
results differencesbetweengroups

36. Statistically significant differences (p < 0.05)
were found between program groups on 19 of the 31
questions …
!
Most of these differences occurred between the
IS/IT and NM groups.

37. In general, the IS/IT students were more
likely to rank the real-world tasks as better fits to the
IS and IT disciplines …
!

38. A significant percentage of respondents
either answered “Don’t Know” for a task or didn’t
provide a response for a discipline on a task.
!
results studentuncertainty

39. !
results studentuncertainty
Don’t Know % Level of Fit CE CS IS IT SE
15.6%
0 (No Answer) 19.6% 26.1% 25.1% 25.1% 24.1%
1 (No Fit) 2.0% 7.0% 9.5% 9.5% 20.6%
2 1.0% 10.6% 18.6% 15.1% 11.1%
3 7.5% 23.6% 21.1% 16.1% 11.1%
4 20.1% 23.1% 14.1% 20.1% 13.6%
5 (Best Fit) 49.7% 9.5% 11.6% 14.1% 19.6%
Question #1 Designs hardware to implement communication systems
X
X
X

40. A significant percentage of respondents
either answered “Don’t Know” for a task or didn’t
provide a response for a discipline on a task.
!
results studentuncertainty
Discipline Non-Responses Percentage
Don’t Know % CE CS IS IT SE
20.0% 31.4% 30.9% 31.2% 30.9% 31.6%

41. Non-major students answered DK more
frequently, but based on our ANOVA cutoff this
difference between groups was not significant.
!
Nonetheless, the finding that all our respondents
were uncertain with one out over every five tasks is
a significant finding.

42. results !
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
CE CS IS SE IT
IS/IT CS NM
disciplineclusterscores
Since each disciple
was associated
with specific tasks,
these scores were
combined and averaged
to create Discipline Cluster
Scores.

43. !
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
CE CS IS SE IT
IS/IT CS NM
Due to the high
uncertainty
percentages
discussed earlier,
we also compared the
cluster scores with the non-
responses removed.

44. The data was subjected to a one-way
analysis of variance that revealed only three
significant differences between groups.
!
The IS/IT students made significantly better
matches on three of the cluster scores compared
to their non-major peers – for CS, IS and IT tasks.

45. Since Task 24 was not associated
with any of the computer disciplines under study,
all Task 24 answers were combined and averaged
for each program group.
!
resultsnon-computingtaskperformance
IS/IT CS Non-Majors
Task 24 Average Scores 2.08 2.10 1.89
When analyzed using a one-way analysis of
variance, no significant differences between groups
were discovered (in task 24 performance).
Results showed that Task 24 responses were
significantly lower than those for the other tasks.

46. Discussion

47. By focusing on students who were
taking an introductory computing course,
we tried to provide insight into whether
students who enroll in computing programs
have a clear understanding of the
disciplinary outcomes of the respective
programs.
!
discussion

48. Based on our rank-order analysis
the best matching occurred for tasks that
related to CE,
followed by IS (78%),
CS (60%),
IT (50%)
and SE (38%).
!
These results are reasonably close to those
encountered by the C&BC study.

49. Results showed statistically significant
differences between the three program of
study groups (IS/IT, CS, and NM) on 19 of
the 31 items.
!

50. Non-major students tended to rank tasks
as having a lower fit with each discipline.
!
IS/IT students tended to rank tasks
as having a higher fit with each discipline.

51. At best this means the IS/IT students
were more likely to be correct in their
matchings.
!
At worse it might suggest a response bias
where IS/IT students were more likely to
assume that there is a higher fit for all tasks.
However, if there had been a response bias
in place, we would have seen similar scores
associated with Task 24, and this was
actually not the case.

52. Like the earlier C&BC study, our results
show that students are not always clear
about the disciplinary “fit” of different
computing tasks.
!
However, by allowing students to specify a
degree of disciplinary fit, our study showed
that by and large students are able to get
their discipline matches close despite being
inexperienced with computing.

53. This could be construed
as a more encouraging
result than that reported in
C&BC.
!

54. Another important result was the lower
likelihood that students would correctly
identify the IS, SE, and IT tasks.
!
also
This highlights how important it is for faculty
in these fields to better articulate what these
fields encompass, and to better
communicate this information to prospective
and current students alike.

55. Our data showed that the
most confusion about what
discipline a task belonged to
were those tasks connected to
real-world tasks.
!
also
Uses programming skills to
create or modify business
solutions
Manages large scale
technological projects
Develops or maintains web
sites
Evaluates and improves the
usability (user experience)
of computing systems
Given that these larger
projects often involve a variety
of different skills and abilities,
this uncertainty could even be
construed as a positive sign.

56. There is a rain cloud in this sunny picture
The very significant percentages of task and
discipline uncertainty across all five sub-
disciplines does indicate that all three student
groups (IS/IT,CS,NM) have large gaps in their
knowledge about the disciplines.
!
however

57. !
thus
This ignorance was likely masked in the
C&BC approach since it did not provide an
option for specifying uncertainty.
Designs hardware to implement
communication systems
CE CS IS IT SE
⃝ ⃝ ⃝ ⃝ ⃝
versus

58. limitations

59. The main limitation of our study is
similar to that of the C&BC study that
inspired it: namely, if the task descriptions
were too clear or too vague, then this would
compromise the statistics and any
conclusions drawn from them.
!
limitations
As well, the five disciplines did not have the
same number of tasks for which the
discipline was the best fit.

60. The other key limitation of our study is
that we did not have any CE or SE students
in our study due to our university not having
a CE or SE program.
!
limitations
This limitation could conceivably be
addressed in the future if data was obtained
from universities that have a CE and SE
programs.

61. conclusion

62. Over the last two decades,
computing has undergone a reasonable
level of differentiation into five sub-
disciplines.
!
conclusion
This has generated some ambiguity
about the computing subdisciplines which
resides in the minds of students, faculty,
and even employers.
Other disciplines (such as engineering) also
grapple with task/skill understanding by
students and educators.

63. Our study adds to the literature on
disciplinary task/skill identification via the
enhancement of the C&BC instrument …
!
conclusion
by identifying an additional 15 skills from the
ACM sub-discipline descriptions and by
allowing participants to specify a degree of
disciplinary fit.

64. Our results show that students
are not always clear about the disciplinary
“fit” of different computing tasks …
!
But, when students provided an opinion
about fit, major and non-major students
alike were actually often close to correctly
identifying the correct discipline.
This result is a new finding and a by-product
of our revised survey design.

65. Our study also showed that IS/IT students
had a better task understanding that those
enrolled in the computer science program.
!
finally