A workshop presented during the European Summer School if Technology Enhanced Learning held in Greece #JTELSS2022

1. DesigningforTEL:Design-basedResearchandResearch-
basedDesign
Maka Eradze, Maha Al-Freih, Anna Dipace
24.05.2022
JTELSS 2022
Greece
2020-1-IT02-KA226-HE-095042

2. About
workshopleads
 Real person:
 Dr Maka Eradze, assistant professor
 @The University of Foggia
 Italy
 Hybrid version:
 Dr Maha Al-freih, assistant professor
 @Princess Nourah Bint Abdulrahman Uni
 Saudi Arabia
 Not present today
 Dr Anna Dipace, full professor
 @The University of Foggia
 Italy

3. Workshop
Objectives
 Discuss the concepts of design and design research
 Discuss the differences and similarities between DBR
and RDB
 Understand how to balance between research and
development
 Understand how to tackle TEL innovation design and
uptake challenges
 Create a design research idea and create a research
design

4. Outline
 Introduction
 Design based research and research-based design
 Cases
 Task presentation
 Work in groups
 Pitch your idea!

6. Design
 We face designs in everyday contexts
 Design can be tangible or implicit
 It can be digital or analogical
 Design can be a verb or a noun
“Consider the door. There is not much you can do to a door: you can open or
shut it. Suppose you are in an office building, walking down a corridor. You came
to a door. In which direction does it open? Should you pull or push, on the left or
the right? May be the door slides. If so, in which direction? I have seen doors
that slide up into the ceiling. A door poses only two essential questions: in which
direction does it move? On which side should one work it? The answers should
be given by the design, without any need for words or symbols, certainly
without any need for trial and error.” (Design of Everyday Things, D. Norman).

7. Design of
Technology in
Education

8. Ok.Butwhystill
agap?
 Think of Zoom or Teams
 The most used educational technology tool during the
pandemic
 Was not developed for education
 While there are other tools that were developed for
specifically for educational use but not adopted as
much
 Why does it happen? (almost a rhetorical question☺)
 Who designs? For whom?

10. Educational
research
“educational research is often divorced from the
problems and issues of everyday practice – a split
that resulted in a credibility gap and creates a need
for new research approaches that speak directly to
problems of practice and that lead to the development
of ‘usable knowledge’.”
Design-Based Research Collective (2003, p.5)
‘traditional’ research approaches such as
experiments, surveys, correlational analyses, with
their emphasis on description hardly provide
prescriptions that are useful for design and
development problems in education”
van den Akker (1999, p.2)

11. Educational
Design
Research
 To design and develop an intervention (e.g. programs,
teaching strategies and materials, tools and systems) as a
solution to a complex educational problem as well as to
advance our knowledge about the interventions and the
processes to design and develop them (Developmental
Studies), or alternatively
 To design and develop educational interventions (e.g.
learning processes, learning environments) with the
purpose to develop or validate theories (Validation Studies).
Design research is used as a common label for a ‘family’ of
related research designs which may vary somewhat in goals
and characteristics – such as design experiments (e.g.
Brown, 1992), design studies (Walker, 2006), design-based
research (Design-Based Research Collective, 2003),
developmental research (van den Akker, 1999) and
engineering research (Burkhardt, 2006).

12. “Wicked”
Problems
Kelly (2013) asserts that EDR is more appropriate
for wicked problems. So what are wicked problem?
- They lack of adequate theoretical basis and
resources, unclear solutions, unique and
complex contexts, and interconnected factors
that hinders progress.
- Initial state(s) are unknown or are unclear.
- Goal state(s) are unknown or are unclear.

13. Designin
education
Two views - Simon and Schön:
 Both see teachers (and other practitioners) as designers who devise new
methods and artefacts to solve problems.
 Simon - positivist, “technical rationality” stance:
 Design as a rational process of problem-solving by iterative optimisation.
 Even ill-structured problems can be decomposed into structured ones which can
be solved systematically by applying scientific principles.
 Schön places a greater emphasis on the process of problem-setting and adopts a
pragmatist-construtivist view.
 Value in the tacit knowledge of practitioners.
 Real-world domains are uncertain, ambiguous, ill-defined and complex.
 Often the problem to be solved is only fully understood when the solution is
presented. In other words, articulating the problem is half the solution – but
not necessarily the first half.
 Balance is needed (Holmberg) - in which practitioners are seen as “on-the-spot
researchers”, informed and informing scientists through a continuous dialogue.
 Educational design research seems to be the answer?
Maina et ak, 2015, Art and Science of Learning Design
Holmberg, J. (2014). Studying the process of educational design–revisiting Schön and making a case for reflective design-based research on teachers’‘conversations with
situations’. Technology, Pedagogy and Education, 23(3), 293-310.

14. Design-BasedResearch
(DBR)andResearch-Based
Design(RBD)

15. Design-Based
Research(DBR)
DBR is a “systematic but flexible
methodology aimed to improve
educational practices through
iterative analysis, design,
development, and implementation”
(Wang & Hannafin, 2005)

16. Characteristicsof
DBR
 The central goals of designing learning environments and
developing theories of learning are intertwined.
 Development and research take place through iterative
and continous cycles of design, enactment, analysis, and
redesign in collaboration with stakeholders.
 Research on designs must lead to sharable theories that
help communicate relevant implications.
 Research must account for how designs function in
authentic settings.
 The development of these accounts relies on methods that
can document and connect processes of enactment to
outcomes of interest.
 Design-Based Research Collective (2003, p.5)

17. DBR cycles according to Amiel and Reeves (2008)

18. DBR cycles according to McKenney and Reeves (2012).

19. The Integrative Learning Design Framework (ILD) (Bannan-Ritland, 2003)

20. Balanceneeded
eveninDBR
 Also for theory and artefact development
 Osmotic model as a solution to conceptualise and
view the balance
Ejersbo, L. R., Engelhardt, R., Frølunde, L., Hanghøj, T., Magnussen, R., & Misfeldt, M. (2008). Balancing product
design and theoretical insights. The handbook of design research methods in education: Innovations in science,
technology, engineering and mathematics learning and teaching, 149-164.

21. Research-Based
Design
 RBD stems from the fields of design and Human-
Computer Interaction (HCI).
 Its a design research approach aiming to design tools
in an open dialogue between designers and
participants and provide them with software
prototypes.
 In complex social activity systems, new tools bring
changes to existing activity systems. A new tool should
communicate the changes needed in the system.
 In the context of educational technology this means
that the artifacts in part communicate what learning
could be.

22. RBDprocess
andphases
 Is strongly oriented towards the exploration of new design
concepts and ideas to inform innovative solutions (Leinonen
et al., 2008) to educational problems.
 While RDB does not exclude the development of the
theory through design activities, rather the product
communicates the theoretical findings
 However, its core research focus remains on the iterative
building and redesign of effective prototypes and artefacts.
 RDB is a hypothesis generating framework where the
software (learning tool) is produced as a hypothesis

23. Aha!Sotheseare
methodologies?!
 It is a series of techniques
 It is a way of thinking
 It is a process

24. Sowhereisthe
problem?
Similarities and
differences
Food for thought:
 While these two approaches to educational design research
and development share many similarities, they utilize
educational design concepts and processes differently.
 The main difference between RBD and DBR is whether the
focus is on artefact/product or theory generation as the final
output.
 In DBR design is the means, while in RBD it’s the end result.
Problems:
▪ While DBR claims to address problems and cover the gap
of the theory and practice in education, often, the
products/artefacts developed within DBR process are not
used/adopted.
▪ Some argue that RBD lacks the research rigour of DBR
(McKinney, 2014) and is more oriented towards the design
and development of artefacts rather than theory building.

25. Case:design-
basedresearch
The “Wicked” Problem
The high dropout rate in MOOCs
DBR Study conducted from 2013 to 2017
Why “Wicked”?
 The unprecedented scalability and open access in MOOCs
 Unique and complex context
 The diversity of teacher preferences and beliefs, disciplinary
influences
 The wide variation in learners’ demographics, expectation and
patterns of engagement, behavior, and self-organization skills
 The need for new metrics that go above and beyond the traditional
benchmarks of course certification, grades, and completion
 Quantity and quality of data
DBR Framework
The Integrative Learning Design Framework (ILD) (Bannan-Ritland, 2003)
Informed Exploration → Enactment → Evaluation (local) → Evaluation (broad)

26. Case:design-
basedresearch
Humanizing Online Instructions MOOC was offered in the Canvas
open network and for four weeks from March 9th of 2015 - April 5th.
This MOOC was designed to be a professional development
experience for those who wish to improve their online teaching
practices by introducing them to the Community of Inquiry (CoI)
framework.
The HumanMOOC
Badges
- Week 0 - orientation
- Week 1 - Instructor Presence (Instructor Presence Badge)
- Week 2 - Social Presence (Social Presence Badge)
- Week 3 - Cognitive Presence (Cognitive Presence Badge)
- Earning all badges leads to the CoI Badge

27. Case:design-
basedresearch
Purpose
 Formulate a clear problem definition and theoretical conjectures.
 Develop conceptual framework and tentative design propositions.
Informed Exploration Phase
Data Sources
 Literature Review.
 Learning interactions and artifacts.
 SRL microanalytics.
 Social media postings (#HumanMOOC).
 Pre and post surveys.

28. Case:design-
basedresearch
 18.4% of all pre course survey participants who indicated their intention
for earning a badge(s) went on to achieve their goal in the MOOC.
This is compared to 2.8 if it was calculated as the fraction of individuals
who enrolled in the #HumanMOOC and successfully finished the
MOOC by earning the CoI badge.
 Identification of significant SRL processes that can account for
differences in completion as goals for design intervention in the
following stage.
 Redefinition of what persistence and completion in MOOCs mean.
Pilot Study - Interesting Findings
This served as input in my Dissertation study
Literature Review
- Scalability and flexibility of MOOCs require SRL and SDL Skills
- The need for new metrics to measure persistence in MOOCs.

29. Work in Groups

30. Task
 Plan a design-based research or research-based design
process:
 Plan a design of an artefact/product
 Plan the development of a theoretical contribution
 Plan a research
 Present and convince

31. Instructions
 Identify: a research problem
 Where does your research problem come from?
 Frame: think of related literature and conceptual frameworks
that can contribute to your theoretical insights
 Ideate: think about a design idea (of a technological artefact)
and beneficiaries/users
 What does your design idea support?
 What problem do you want to address?
 What research cycles would you propose to investigate this
problem and the design idea?
 Would you align your research plan or framework with RDB or
DBR? And why?

32. Pitchyouridea!  Presentation of results - pitch your idea!

33. References &
Additional
Readings
Amiel, T. & Reeves, T. C. (2008). Design-based research and educational
technology: Rethinking technology and the research agenda. Educational
Technology & Society, 11(4), 29–40.
Bannan-Ritland, B. (2003). The role of design in research: The integrative learning
design framework. Educational Researcher, 32(1), 21-24.
Design-Based Research Collective. (2003). Design-based research: An emerging
paradigm for educational inquiry. Educational researcher, 32(1), 5-8.
Kelly, A. E. (2013). When is design research appropriate? In T. Plomp, & N.
Nieveen (Eds.), Educational design research-Part A: An introduction (pp. 134-151).
Retrieved from http://international.slo.nl/edr/
McKenney, S., & Reeves, T. (2012). Conducting Educational Design Research:
What it is, How we do it, and Why. London: Routledge.
Maina et ak, 2015, Art and Science of Learning Design
Holmberg, J. (2014). Studying the process of educational design–revisiting Schön
and making a case for reflective design-based research on teachers’‘conversations
with situations’. Technology, Pedagogy and Education, 23(3), 293-310.
Leinonen, T., Toikkanen, T., & Silfvast, K. (2008, October). Software as
hypothesis: research-based design methodology. In Proceedings of the tenth
anniversary conference on participatory design 2008 (pp. 61-70).

34. Contacts
Maka Eradze - maka.eradze@unifg.it
Twitter @MakaEradze
Maha Al Freih - maha.gmu@gmail.com