Even if the question of eLearning quality has been intensely discussed in the recent years, with several approaches and models arising, the implementation of concepts into practices remains contested (Elhers & Hilera, 2012 ) . Higher Education Institutions (HEI) are facing an important change:from the single institutional efforts to give answer to a very changing society and labour market to the transnational debates and pressure for HEI modernization, like the case of Bologna Process.In this context, eLearning is given different importance with regard to organizational innovation and the general HEI culture of quality (Ehlers & Schneckenberg, 2010). While it has been envisaged as the panacea to promote improvements in such different dimensions as cost-benefit ratio, access and inclusiveness, or the introduction of learner centered pedagogical approaches, very often the values and motivations entrenched in these dimensions clash and enter in more or less evident contradictions. As a result, the implementation of quality eLearning in HEI could be slowed down or blocked (Conole, Smith, & White, A critique of the impact of policy and funding, 2007). In this article the authors introduce the results of an initial exploratory phase undertaken as part of a participatory action research funded by the Italian Ministry of Education PRIN (Research Project of National Interest, “Progetto di Ricerca d’Interesse Nazionale”) namely, “Evaluation for the improvement of educational contexts. A research involving University and local communities in the participatory development of innovative assessment models”. On the basis of a qualitative epistemological approach (Creswell, 2007) (Denzin & Lincoln, 2011), several stakeholders from one University were interviewed, attempting to capture the several discourses on quality in HE and the embedded idea of quality eLearning . The results obtained were later conceptualized attempting to define quality as a complex object that requires mediation for the negotiation of the several perspectives.

1. 5th International Conference of Education, Research and
Innovation
Madrid, 19-21 November 2012
Patrizia Ghislandi – Juliana Raffaghelli
University of Trento

2. A) eLearning and change in higher
education: tensions and contradictions
ELearning evolution and its contribution to Higher
Education change
What is eLearning quality?
Quality as a complex object
B) The PRIN project case:
“Evaluation for the improvement of
educational contexts” C) Conclusions
The Methodological Approach Mediated quality. deep understanding, reflection
and contextualized design
First Findings from an exploratory phase

3. Are Universities fully
using the power of
technologies to rethink
pedagogical practices?
Web 2.0 and Social Media
3

4. Dimensions of The initial media 2.0 Future
change revolution (1.0)
Teachers Sage on the Guide on the Side The orchestra director:
Stage harmonizing the generation
of content
Pedagogical Individual Collaboration Open Knowledge and
Practices & assignments on flexible social networking:
Learning given contents networked learning
Institution campus Interinstitutional Beyond local and
Cooperation institutional barriers
Students Passive role Initial sense of Co-creation of contents
being part of the
educational process
4

5.  We all agree but… Organizational
 (O’ Hearn, 2000; Holley, 2000; Volery, 2000;
 Rosenblitt, 2006; ) Cultures
 the differential infrastructure and readiness of
different types of higher education institutions
to utilize the technologies’ potential ;
 the extent to which the ‘old’ distance
education technologies and the new
technologies replace teaching/learning
Digital Skills
Unprepared Pedagogical
practices in classrooms ; Institutions Beliefs
 the role of real problems, barriers and
obstacles in applying new technologies ;
 the impact of the new technologies on
different student clienteles ;
 information acquisition vs knowledge
construction in higher education ;
 cost considerations ;
 the human capacity to adapt to new learning
styles in face of the rapid development of the Infrasatructure
technologies ; and
 the organizational cultures of academic and
corporate worlds.

6. The reality today: Open questions…
 Access and prevention of drop outs
How can we with more people following studies at
University level;
conceive and
introduce • Transparency and usability of
teaching contents;
QUALITY
• Open relationships with the society
Elearning in and the world of work to improve
Higher young students’ transitions,
Education • Wider access through the use of
(QeLHE)? eLearning;
Building on Laurillard, 2002

7.  Diverse Cultures of Quality are
underpinned by diverse values:
 Exceptional/Original: the
value is on the uniqueness
 Distinctiveness: not for all
 Excellence: The highest levels
of performance
 Fitness for Purpose: doing
what has been planned
 Inclusive: all people can
participate
7

8. ISO/IEC 19796
Q UNESCO Quality for all
SLOAN-C MODEL –USA-
EFQUEL –UE-
Sistematic
Approaches CENTRO VIRTUAL PARA EL
Access? DESARROLLO DE ESTÁNDARES DE
Excellence? CALIDAD PARA LA EDUCACIÓN
Innovation SUPERIOR A DISTANCIA EN AMÉRICA
Inclusiveness?
LATINA Y EL CARIBE
8

9. Elements Dimensions
Multiperspective The teacher – the student – the institution, the
evaluators
Diverse Methods of Benchmarking – guidelines – standards –
Analysis quantitative or qualitative approaches
Diverse Time In itinere – ex ante – ex post
Diverse Meanings Pedagogical – Organizational – Technological –
Economical
Diverse Levels of Individual – Group – Institutional – Socio-cultural
Analysis
9

10.  Quality is not an intrinsic, universal
value
 It is very much about the methodology
of evaluation,
 And the substantial epistemological
principles and values underlying the
process of evaluation

11.  The selection of qualitative methods,
 a phenomenological approach based on narrative self-evaluation, peer-evaluation and
meta-evaluation,
 emphasizes the interest on processes and on the empowerment of
participants AS COMMITTED EVALUATED
 This logic studies the topic within its context, uses an emerging design that accounts for
reality as subjective and multiple, lessen the distance between “official” evaluators and
participants (Denzin & Lincoln, 2011).
As a result,
the evaluation process encompasses a transformational
(participatory/innovative) opportunity for the engaged
individuals/ institutions.(Creswell, 2007; Mertens, 2009).

12.  The transformational perspective is the kernel
of a quality learning culture:
 a human group that take part of a learning
experience as a deep, reflective experience,
connected to the own professional/ personal
identity
▪ For which purposes do I learn? What can I do with this
learning?
 not just for accomplishing activities, recalling
information, and obtaining credentials (course
diploma).
Teachers and students should become insiders
of the culture of quality.

13.  PRIN (Projects of National Relevance, Ministry of
University, Education and Research of Italian Republic)
project 2009
 “Evaluation for the improvement of educational contexts. A
research involving University and local communities in the
participatory development of innovative assessment
models”
 PRIN first exploratory phase:
 Analysis of the institutional culture
(values, meanings, beliefs) about quality in HE;
 Analysis the stakeholders approach to quality of HE and in
this context, to quality of eLearning in HE.

14. Exploratory Reflection &
•Understanding fieldwork •Pedagogical Feedback
QeLHE at different Innovation,
levels •Conflicts and participatory •Joint analysis on the
contraddictions in the evaluation educational impact of
discourses about the quality model
QeLHE within the
University
Preliminary Design &
Analysis Intervention
• Methodological Approach: Case Study, Participatory Action Research
• Main sources of data: documents, interviews, forum analysis,
observations, design workshops, use of tools for design (Conole, 2012)
• Method for data analysis:
• Exploratory phase: Discourse analysis, Semio-pragmatic Analysis
• Transformative phase: design based research (DBR)
14

15. Exploratory Reflection &
• Understanding fieldwork • Pedagogical Feedback
QeLHE at different Innovation,
levels • Conflicts and participatory • Joint analysis on the
contraddictions in evaluation educational impact
the discourses about of the quality model
QeLHE within the
University
Preliminary Design &
Analysis Intervention
Subphase 1 (A1-2 / A8-9) Subphase 2 (A3-A6 / A10)
Exploratory Phase Documental/Web Analysis Interviews / Observation
3 Courses A.A. 2011/2012 6 Students
Method: Case Study/PAR Near 500 students 6 Academics
3 Academics 1 NVA
1 Support to Didactics
4 Instructional Designers
2 eTutors
Transformative Phase Planning Interventions Intervening
Method: dbr 4 "Learning Design Workshops" 3 Courses A.A. 2012/2013
Near 500 students
3 Academics
2 eTutors 15

16.  Focus on the critical tensions and contradictions
within the institution between the several
stakeholders to implement concrete practices
linked to the own vision of educational quality
 Methodological approach
 Documental Analysis
 Semio-pragmatic Analysis
 Interviews
 Triangulation and Member-checking

17. Perspectives on eLearning inside an academic course
More than having access to materials: I didn't think they (materials) are
very useful, I had other files from other classes and other friends, they are
giving me some other stuffs to help me learn [S3]
Something very complementary: Due to my way of learning, I like to follow
the teacher’s lessons, then I organize my study. I don’t like technologies
Students
very much. It is ok if I can just receive communications, or have access to
materials (…) If I can choose, I take the FTF course. [S1]
Better possibilities of communicating among students and the teaching
staff: the teacher has to generate a sense of continuity between the FTF
activities and the online. I like that. Clearly a teacher with 100 students
cannot do this very well. But this year there was another assistant (the
eTutor) that accompanied us and it was very good [S2]

18. Perspectives on eLearning inside an academic course
A way of following the institution approach to learning: Our university
is not outside of the times, and eLearning came to stay. So it is better
to tackle the issue and be prompt to do what is our duty.[T2]
Facilitating the access to materials: well, I don’t use eLearning in an
advanced way; I have to recognize that it has facilitated the delivery of
Teachers
materials, but I never adopted collaborative ways, for I like to work
FTF if I can. So eLearning helps me to qualify my course in this sense
[T4]
Opening to continuing pedagogical innovation: eLearning is like a
Trojan horse…you introduce the technological frame, then you start to
rethink all your teaching practices and in the end the nature of the
knowledge that you teach. This should be a never ending process
[T1]

19. Perspectives on eLearning inside an academic course
Technologies are not all in the implementation and quality of eLearning: in
our initial phases of implementation of a project to introduce eLearning at
academic level, we adopted different technologies. The eLearning platform
adopted till today represented an important frame to support teachers in
their way of working with eLearning. But this is not all, this is (and continues to
Instructional Designers
be) the excuse to rethink the way of teaching (the pedagogical approach, our
comment) [ID1]
An invisible role: our role must be invisible, must be a base and a
springboard for the teacher that wants to adopt eLearning (…) but it would be
better if it was better recognized (…) no teacher likes to be told how to
teach. The problem is that in eLearning, the deep knowledge of your subject
do not necessarily take to the good delivery of online activities (…) sometimes
we are seen as the “text editors” [ID2]
Institutional context matters: the political context in the institution clearly
addresses what we can do or not in order to promote eLearning and the renew
of teaching methods [ID1]

20. Perspectives on eLearning inside an academic course
eLearning is the last concern in a process of quality
evaluation in HEI: I never really care about eLearning, even
Academic Secretariat
when I understand it importance. I see the importance of
technologies in what I do every day with students, but to me
(…) there are other important issues to solve. To be part of the
Bologna process, with the Dublin indicators (…) we have to
change the way we evaluate students (…). I think that an
important position to implement this process is that of the
coordination of academic courses, but now the role is
overwhelmed of bureaucracy and the academic in charge
cannot dedicate too much attention to institutional change.

21. Perspectives on eLearning inside an academic course
An excellent researcher is a good teacher. If you do real research
and you are an excellent researcher, you are able of being an
excellent teacher.
External Evaluator
The recognition of the teaching activity in HEIs: academics are not
really recognized by their teaching activity. Research counts, not
teaching, and teaching is a heavy work that they are not always open
to focus properly if it takes time from research.
Technologies can help the communication of your research field
into your teaching. I’m not an expert of eLearning, of course I
recognize the value. The technologies in my field of teaching are
important to show concepts/practices that in today’s crowded
universities you cannot always present.

22.  eLearning has a very different and rather
contradictory status among the interviewees.
 The main contradictions regard the
dimensions that matter for a overall quality
culture in HEI:
 some stakeholders concern is on the policy
context and institutional change
 other claim for the recognition of eLearning as
field of practice that is evidence based
 Concern on innovation vs. concern on tradition

23. Personal Approach Interest on Practices Quality
positioni eLearning values
ng
Outsiders Sense of duty with regard Secondary Implementing only Tradition
of QeLHE to a model that it is being issue official Outcomes
(1) implemented generally at Not aware programmes (*);
the University
Outsiders Solving specific problems eLearning as Delivery of content Tradition
of QeLHE on current practices support of facilitated by Outcomes
(2) what we eLearning
already do platforms
Insiders Clear personal conviction eLearning can Experimenting Transformation
of QeLHE on innovation, evidence be a mean to with eLearning Process
based driven transform
pedagogy
(*) In Italy there are only very few regulations regarding eLearning

24.  Raise awareness on the context as well as
specificities of pedagogical innovation within
higher education could lead to the
harmonization of quality discourses.
 We call this operation mediation of quality:
quality.
 From a socio-constructivist approach: means offering
tools that would support the processes of negotiating
the many values lying behing a quality culture
 Tools mediate learning of stakeholders to pass from a
position as outsiders of quality to a position as insiders,
or active agents of change.

25. On the incommensurability of Quality
Quality….you know what it is, yet you don’t know
what it is. But that’s self-contradictory. But some
things are better than others, that is, they have more
Quality. But when you try to say what Quality is,
apart from the things that have it, it all goes poof!.
There’s nothing to talk about. But if you can’t say
what Quality is, how do you know that it even exists?
If no one knows what it is, then for all practical
purposes it doesn’t exist at all. But for all practical
purposes it really does exist (Robert Pirsig. 1974)
On Learning Design
making the design process more explicit and
shareable (…) help learners to make more sense of
their educational provision and associated learning
pathways. (Grainne Conole 2011)

26. For communications:
patrizia.ghislandi@unitn.it