At the core of TEL research are artefacts of digital technology, their design, implementation, application, and evaluation. Usually, these artefacts aim to fulfil a specific educational purpose and need to satisfy a number of requirements with respect to functionality, usability, scalability, or interoperability. Software engineering is the discipline that structures, organises, and documents all aspects of the software development process in manageable steps. It explains all relevant stakeholder roles involved in the process and defines process models to handle the complexity of the software development process. In research oriented projects, software engineering goals and research goals often collide: Software engineering strives to provide a fully fledged system with a complete set of functionality and a broad coverage of use cases. Research aims for evaluating testable hypotheses based on specific aspects of a system. This leads to the problem that the complexity of the design steps, complexity of the derived/developed solution contradicts easy to measure results. Furthermore, project contexts and research contexts often collide, leading to the question how to develop technology that fulfills development needs and research needs. The lecture looks at typical situations, which occur in technology-oriented research projects and tries to show approaches to handle the inherent complexity within these. References Tchounikine, P.: Computer Science and Educational Software Design. Springer Berlin Heidelberg, Berlin, Heidelberg (2011). Goodyear, P., Retalis, S.: Technology-enhanced learning Design Patterns and Pattern Languages. Sense Publishers (2010). Mor, Y., Winters, N.: Design approaches in technology-enhanced learning. Interact. Learn. Environ. 15, 61–75 (2007). Bjork, S., Holopainen, J.: Patterns in Game Design (Game Development Series). Charles River Media (2004). Calvo, R.A., Turani, A.: E - learning Frameworks = ( Design Patterns + Software Components ). In (Goodyear & Retalis, 2010). Wang, F., Hannafin, M.J.: Design-Based Research and Technology-Enhanced Learning Environments. Source Educ. Technol. Res. Dev. 53, 5–23 (2005). Kirkwood, A., Price, L.: Technology-enhanced learning and teaching in higher education: what is “enhanced” and how do we know? A critical literature review. Learn. Media Technol. 39, 6–36 (2014). Ross, S.M., Morrison, G.R., Lowther, D.L.: Educational Technology Research Past and Present: Balancing Rigor and Relevance to Impact School Learning. Contemp. Educ. Technol. 1, 17–35 (2010).

1. The 'T' in TEL
Software Development for TEL Research
Problems, Pitfalls, and Solutions
Roland Klemke

2. Technology Enhanced Learning

3. Technology
Computer Science
Engineering
...
Learning
Learning Science
Pedagogy
...
Enhanced
???
OVERLAP?

4. Problem Area 1:
The TEL Field Itself

5. Interdisciplinarity in TEL
http://blog.richardmillwood.net/2013/05/10/learning-theory/
https://voxy.com/blog/2010/12/four-ways-in-which-technology-enhances-language-learning/

6. Feasibility
Reliability
Interoperability
Scalability
Security
Learning Effects
Knowledge Gain
Learning Time
Mastery
Usability
Acceptance
Motivation
Satisfaction
GOALS

7. Architectures
Proof of Concept
Standards
Performance
Stress Test
Pre-/Post Tests
Assessments
Observation
Interviews
Usability Study
Questionnaires
User Tracking
Observation
METHODS
Software
Engineering
Learning
Design
???

8. What to do?
Know your enemy!
● What are the targets of the
research group / project you
are involved with?
● Which methods do apply?
● Do they match the
expectations of your research
aim?
● Are you able to perform these
methods (e.g. required
technical competencies
available)?

9. Problem Area 2:
Research Approaches
in TEL

10. 3.
The software
engineering process
follows the
requirements given
by the learning
theory and
intervention design
1.
A selected
Learning Theory
requires specific
procedures
2.
We design
interventions based
on expected effects
according to the
theory
THEORY

11. Example for a theory-driven approach:
Gamification of Learning in a MOOC Context
1.
Problem & Goal identification
E.g.: Addressing high number of drop-out
2.
Theory Selection
E.g. Implementation Intention Theory
3.
Intervention Design
E.g. Stimulated Planning Game
Element
4.
Application Case Identification
E.g. MOOC on IT Security
5.
Implementation
E.g. Widget development
6.
Application and Data Collection
E.g. Stimulated Planning Game
Element
L
E
T

12. Gamification of Learning in a MOOC Context
Resulting Stimulated Planning Intervention
Typical Challenges:
Are you able to design the tools
you need?
Do you manage the technical
requirements involved?

13. 2.
Experimental
developments lead
to testable
prototypes
3.
Theory selection
and experimental
design follow
technical
constraints and
possibilities
1.
Emerging
technologies offer
new affordances.
Learning related
outcomes are fuzzy
INNOVATION

14. 1.
Analysis of technological affordances
E.g.: Sensor-based Augmented Reality
2.
Hypotheses about learning aspects
E.g. Record and re-enact expert
performance
3.
Prototypical Development
E.g. AR prototype with wearables
4.
Pilot application and data collection
E.g. Acceptance & usability data
5.
Refinement towards learning
framework
E.g. WEKIT Framework
6.
Evaluation
E.g. Measurement of learning aspects
E
T
L
Example for an innovation-driven approach:
Expertise development in the context of the WEKIT project

15. Expertise development in the
context of the WEKIT project
Technical architecture and
prototype setup
Typical Challenges:
How to apply your research to
this complex architecture?
How to balance partner
interests?

16. Expertise development in the context of the WEKIT project
Resulting theoretical Framework
Typical Challenges:
How to derive a theoretical
framework?
How do framework and software
prototype match?

17. 1.
An educational
service needs to be
delivered at a pre-
defined level of
quality
2.
The educational
services is
productive and
needs to be
evaluated
3.
Research
interventions can
(potentially) be
applied to the
existing service
SERVICE

18. 1.
Identification of requirements for
educational service
E.g.: Need for a MOOC offer
2.
Technical implementation of service
E.g. productive setup of MOOC
platform
3.
Development of educational
offer
E.g. Course content development
4.
Evaluation of service success
E.g. usage data, feedback data
5.
Identification of problems and
improvement need
E.g. Lack of student engagement
6.
Research design
E.g. Interventions to support sense of
community to increase engagement
T
L
E
Example for a service-driven approach:
Implementation of a MOOC offer

19. Gamification of Learning in a MOOC Context
Resulting Software Architecture
Typical Challenges:
Is the existing service
accessible for development?
Are you allowed to develop for
it?
What support is available?

20. Gamification of Learning in a MOOC Context
Resulting Software Architecture
Typical Challenges:
Can you talk "architecture"?
Can you deliver components?

21. 1.
An educational
service is (partially)
in place
1.
Improvements for
Learners are
expected based on
some sound theory
1.
You already have a
new technology to
apply in mind (or at
least your
supervisor has)
REALITY
Typical Challenges:
Where to start?
How to progress?
How to ensure results?
Is this even sound science?

22. What to do?
Plan! Try! Re-plan!
● Don't expect your research to
follow a straight path - expect
deviations, obstacles, setbacks
● Think in iterations!
● Identify your personal gaps and
find solutions for these
● If your gaps are on the "T"-
side: we'll get back to that later

23. Problem Area 3:
Project Constellations

24. Isolated PhD project
Advantages
● You lead the whole process
● The only requirements to meet are those
of your PhD research
● You have the maximum amount of
freedom
Disadvantages
● You have to do everything yourself
● It may be hard to find participants for your
experiments
● It may be hard to get support for aspects
you are not the expert in
Questions:
● Do you have the technical skills to run the project alone?
● Can you develop the required interventions yourself?
● Do you have access to participants for your experiments?
● What about pre-studies, beta-tests, exchange with colleagues?

25. PhD project embedded in research project
Advantages
● The project provides a research context
● The project team can support you
● Ideally, you can focus on your specific
aspects, while other aspects are covered
by other participants
Disadvantages
● Project requirements and PhD
requirements may diverge over time
● Project deadlines and PhD schedule may
collide
● Project collaboration may fall behind
expectations
Questions:
● Who helps you to align research goals and project goals?
● Do you know the research aims of the other participants? Are
there conflicts?
● Are you allowed to publish about the results relevant to your
research?

26. PhD project embedded in service project
Advantages
● A service project aims at providing a
stable, reliable, scalable service
● A service project has real users
● A service project can be expected to be
professionally organised and performed
Disadvantages
● Decisions in service projects follow project
requirements and might ignore research
● Software used in a service project might
not be tailorable (e.g. closed source)
● Schedules, deadlines, requirements may
collide
Questions:
● Is the service project open for research?
● Can you get the data?
● Do you get the technical support you need?
● Can you plan/perform/apply your interventions in the productive
service environment?

27. What to do?
Talk, talk, talk!
● Identify the possible risks in
each of the constellations as
early as possible
● Talk to all parties involved
about these risks

28. Problem Area 4:
Software Engineering
in TEL

29. What is Software Engineering?
Software engineering
provides methods and
process models to tackle
the design and
development process for
the creation of software
Components
● Problem statement
● Requirements
● Design and Specification
● Implementation
● Test
● Application

30. Why Software Engineering in TEL?
Artefacts of digital technology at the core of TEL research
Educational Purpose: key functional requirements
Software Aspects: quality, scalability
Software Engineering: discipline that structures these aspects

31. Why can’t we just apply it?
Software development goal: provide a fully fledged system with a complete set
of functionality and a broad coverage of use cases
Research goal: evaluate testable hypotheses based on specific aspects of a
system
Problem: complexity of the design step, complexity of the derived/developed
solution contradicts easy to measure results.
Project context and stakeholder context often collide, the question is how to tweak
technology also how to make it researchable

32. What to do?
Act!
● How complex are the
technological requirements of
your research?
○ Learn programming
(seriously!)
○ Team up with
programmers
○ "Wizard of Oz" approach?
○ Use of ready-made tools?

33. Discussion

34. Discussion
TEL Research is complex, but doable!
Software
Engineering
T E L
Research
Approach
Project
Constellations

35. What?
This is a lecture, why do we
have to do something?
Let's discuss!
Reflect on your own research
situation in TEL along the lines of
this presentation
● What are your biggest gaps to
close?
● Where are your highest risks?
Can you become an "E" (if you are
"L" or "T")?