The document describes a study that investigated the effectiveness of a wiki-based learning activity on student performance in a human-computer interaction course. The study found that over half of students improved their test scores by at least 50% after participating in the wiki activity. The activity was most beneficial for students with lower initial scores. Additionally, the role students took on in their wiki teams did not affect their learning gains. The wiki activity was designed based on critical factors from prior research and included student collaboration on usability evaluation topics in their wiki groups.
Activity 2-unit 2-update 2024. English translation
Effectiveness of a framed wiki-based learning activity in the context of HCI education
1. Nikolaos Tselios* | nitse@ece.upatras.gr
Panagiota Altanopoulou* | galtanopoulou@gmail.com
Christos Katsanos# | ckatsanos@eap.gr
Effectiveness of a framed wiki-based learning
activity in the context of HCI education
PCI 2011 | Kastoria, Greece | 30 Sep -2 Oct 2011
*Educational Science and Early Childhood Education, University of Patras
# Internal Assessment and Education Unit, Hellenic Open University
2. Are the wikis suitable to
design learning activities?
Web 2 has brought much greater attention to collaboration
on the Web, and this has begun to flow into e-learning
Wikis are open-nature and can provide strong opportunities
for collaboration, critical thinking and reflection
Central tenets of sociocultural views of learning, and in particular of
project based learning (Duffy & Kirkley, 2004)
Wikis are also easy to learn and usable
Can be used both for distance and blended learning
2
However, without planning, design, and effective facilitation,
a wiki is no more than an empty webpage (West & West, 2009)
3. Research Goal & Questions
Investigate if (and to what extent) a properly designed
wiki based activity can enhance the learning process
Research Questions
Does students’ learning performance improve through a
properly designed wiki-based activity?
Are such activities more beneficial to students with lower initial
performance?
Does the student’s role in the team affect her/his learning
performance?
3
4. Using wikis in education
( West & West, 2009)
4
•Class encyclopedia
•Glossary
•Electronic book
Knowledge
construction
Critical
thought
•Case studies
•Structured debate
•Annotated bibliography
Contextual
application
•Organize design activities
•Story creation
•Organize real-world activities
5. Critical factors in the design of
wiki-based learning (West & West, 2009)
1) Establish a purpose for the wiki project
2) Define & classify the learning goals
3) Design a rich context & problem that support the
achievement of the purpose & goals
4) Prepare students for work in the new environment,
5) Promote a collaborative process through which active,
social learning can take place
5
6. Study Methodology
Participants
36 students
Dept. of Education & Early Childhood Education, Upatras
Compulsory project, Course: “Introduction to Web Science”
Within-subjects, Pretest-Posttest design
Demographic questionnaire (pre-)
5 questions, demographic data (e.g. age, ICT skills, wiki usage)
Knowledge assessment questionnaire (pre- & post-)
35 multiple-choice questions with 4 answer options
Questions on usability evaluation & heuristic evaluation
Students were not informed about the assessment
6
7. Activity design (1/4)
Framework: West & West (2009)
Goal: Study usability evaluation techniques in general and
heuristic evaluation in particular
Learning objectives
Understand impact and importance of usability
Understand usage of wikis (both in general and for education)
Study usability evaluation techniques
Be able to conduct a heuristic evaluation
Be able to form collaboration teams with specific roles assigned
Be able to organize and report the results of such a process
Be able to use a Wiki
7
8. Activity design (2/4)
Each group had to create a wiki (WikiSpaces service)
Assignment presented to the students through a wiki
initiated by the researchers (http://web-usability.wikispaces.com)
The wiki included 3 parts (Usability Evaluation, Heuristic
Evaluation, Actual Heuristic Evaluation of the Dept. Site)
Concise instructions
Indicative outline
Arguments to be developed
Supporting material (mainly in the form of hyperlinks)
8
Students were encouraged to use additional material
& arguments (discrete evaluation criterion)
9. Activity design (3/4)
Procedure
1. Instruction on the wiki’s basic functionality
2. Presentation of the compulsory project in the form of an already
initiated wiki
3. Students worked in freely-formed groups of 4
4. Each team member had a specific role in the group
Collector: obtain appropriate material relevant to the subject
Coordinator: organize collected material, check its consistency,
coordinate discussion on website heuristic evaluation
Editor: compose the outline of the topics according to the
objectives of the scenario
Verifier: evaluate content completeness, structure and
compliance with the objectives of the project.
9
10. Activity design (4/4)
Evaluation criteria (included in the wiki)
Coherence
Content and text originality
Text clarity, argumentation and reasoning
Compliance to the provided structure & format guidelines
Use of additional material & arguments
Material appropriateness and richness
Degree of collaboration
Appropriate use of references
10
12. Results (1/3)
12
Does students learning performance improve through a
properly designed wiki based activity?
More than half (56%)
improved their performance
by at least 50%
Highest improvement in
score 65.7 (14.3 => 80)
Some (22%) did not
improve or even did worse
13. Results (2/3)
13
Are such activities more beneficial to students with lower
initial performance?
Correlation
r=-0.586, p<0.001
14. Results (3/3)
14
Does the student’s role in the team affect her/his learning
performance?
ANOVA: F(3,32)=0.732, p=0.541
Students were benefited
regardless of the roles they
selected themselves
15. Summary & Questions
Through a properly-designed wiki-based activity (West & West, 2009)
h
Future work:
Compare learning gain with a control group (traditional teaching)
Design more wiki-based activities in a various educational settings
Investigate learners’ behavioral intention to use wiki technology using
technology acceptance models
Investigate to what extent students improve in other non-cognitive
aspects (e.g. self-organization, collaboration)
15
17. What about final scores in the
project?
We were interested in investigating the learning gain,
not the final outcome
Knowledge Assessment Questionnaire
Learning gain = Posttest_Score – Pretest_Score
18. References
Cress, U., & Kimmerle, J. (2008). A systemic and cognitive view on
collaborative knowledge building with wikis. Computer-Supported
Collaborative Learning, 3(2), 105-122.
Duffy, T., & Kirkley, J. (2004). Learner-Centred theory and practice
in distance education cases from higher education. Mahwah, N.J.:
Lawrence Erlbaum Associates.
Mindel, J., & Verma, S. (2006). Wikis for teaching and leading.
Communications of the Association for Information Systems, 18(1),
1-23.
Raman, M., Ryan, T., & Olfman, L. (2005). Designing knowledge
management systems for teaching and learning with wild
technology. Journal of Information Systems Education, 16(3), 311-
320.
West, J.A. & West, M.L. (2009). Using Wikis for Online
Collaboration. San Francisco:Jossey-Bass.