The document summarizes research that examined English language learners' comprehension of website information and perceptions of website usability. Participants analyzed a tourism website in English and completed questionnaires testing their understanding of content and assessing usability, interface, and navigation. Results from accuracy scores and correlation analyses provided insight into readers' abilities and impressions when interacting with an English-language website.

1. EFL Student’s Ability for Website
Information Comprehension and
Perceptions on Website Usability
s1170173
Makoto Yoshida
Supervised by
Prof. Debopriyo Roy

2. Overview
• Abstract
• Research Workflow
• Question what We Set (to Participants)
• Results and Discussion (1-9)
• Conclusion and Future Work

3. Abstract
The purpose of this study is to observe and consider the impression of L2
readers during website analysis in target language (English). At first,
participants completed an extensive website analysis using established web
design models with a closed-ended website information comprehension
questionnaire (testing readers’ understanding of the text content in the
website). In the second stage, they answered three usability questionnaires
(QUIS, CSUQ and MPRC questionnaires), recording their impression about
the interface, navigational efficiency, content comprehension, learning
aspects of the interface, basic comfort level with text-graphics content etc. I
used Pearson Correlation and non-parametric Friedman Test to analyze
user data. The self-reports on the questionnaires (QUIS, CSUQ and MPRC)
provided a general outlook about the website content and navigation and
might not have been specific to the accuracy scores (related to the text
content of the website).
This exploratory analysis as discussed in this article could help us obtain
initial data on how EFL readers in a typical context as this would perform
with English website information and the type of impression they have
about the website.

4. Research Workflow
• 59 participants completed the website information comprehension
questionnaire. Accuracy scores on the questionnaire were measured.
• Following the interaction with the website, readers completed the three
questionnaires on self-reporting.
• Collection their answers
• Data Analysis
• Consider the Results

5. Questionnaires
The website information comprehension questionnaire focused on the
efficiency with which L2 readers are able to search through
information from the Belize tourism website. The focus at this stage
was on readers’ ability to navigate through the pages, based on cues
from the questionnaire. The questionnaire was designed on the basis of
the following:
1. multiple-choice questions asking readers to accurately pinpoint the
information available in the website.
2. readers’ ability to sequence order steps in the correct order when
searching for an information from the webpage. (Q1 with 8 steps to
be correctly ordered).
10 questions were asked and each question (except Q1) could be scored
as 1(correct) or 0 (incorrect) (binary scale). The total accuracy scores
for each student were measured.
QUIS, CSUQ and MPRC questionnaires are standard software
usability questionnaires used for self-reporting users’ preference for
the website.

6. Results and Discussion
Ques%on N Minimum Maximum Mean Std.
Devia%on
Q1
59
0 10 8.66 2.496
Q2 0 2 1.97 .260
Q3 2 2 2.00 .000
Q4 0 2 1.19 .991
Q5 0 2 1.19 .991
Q6 0 2 1.73 .691
Q7 0 2 1.90 .443
Q8 0 2 1.64 .663
Valid
N
(List
wise)
Table 1: Descriptive Statistics for the 8 Website Information Comprehension Questions
0
0.5
1
1.5
2
2.5
1 2 3 4 5 6 7
MeanAccurateScore
Question Number (1 = Q2; 2 = Q3; 3 = Q4; 4 = Q5; 5 = Q6; 6 = Q7; 7 = Q8)
Mean Accuracy Score (Maxium Value =
2)
Mean Accuracy Score
(Maxium Value = 2)
Figure 1. Mean Accuracy Scores for Q2 – Q8 (Wrong Answer = 0; Correct Answer = 2)

7. Results and Discussion 2
Ques%on
Pearson
Correla%on
Sig.(2-­‐
tailed)
N=59
Q2-­‐Q7
.567**
.000
59
Q2-­‐Q8
.328*
.011
Q7-­‐Q4
.280*
.032
Q8-­‐Q7
.344*
.008
**.
Correla%on
is
significant
at
the
0.01
level
(2-­‐tailed)
*Correla%on
is
significant
at
the
0.05
level
(2-­‐tailed).
Table 2: Pearson Correlation Values (Statistically Significant Values only) among the Website Information Comprehension Questions

8. Results and Discussion 3
Ques%on
N
Minimum
Maximum
Mean
Std.
Devia%on
Q1
59
0
7
4.10
1.658
Q2
59
1
7
4.29
1.521
Q3
59
0
7
3.81
1.503
Q4
59
0
7
3.61
1.576
Q5
59
0
7
3.69
1.664
Q6
59
1
7
4.03
1.575
Q7
59
1
7
3.97
1.553
Q8
59
0
7
3.32
1.824
Q9
59
0
7
3.29
2.205
Q10
59
0
7
2.92
2.020
Q11
59
0
7
4.36
1.689
Q12
59
1
7
4.37
1.376
Q13
59
1
7
4.20
1.412
Q14
59
0
7
3.95
1.665
Q15
59
1
7
4.63
1.484
Q16
59
0
7
4.05
1.726
Q17
59
1
7
3.63
1.809
Q18
59
1
7
3.49
1.696
Q19
59
1
7
4.17
1.652
Valid
N
(listwise)
59
Table 3. Descriptive Statistics for Self-Reports on the CSUQ Questionnaire

9. Results and Discussion 4
Question Category
Number
Significant Correlation with other questions
Q1
All questions
Q2
All questions
Q3
All questions except Q9
Q4
All questions
Q5
All questions
Q6
All questions
Q7
All questions
Q8
All questions except Q9
Q9
All questions except Q3 and Q8 and Q15
Q10
All questions except Q12
Q11
All questions except Q12
Q12
All questions except Q9, Q10, Q11, Q12
Q13
All questions
Q14
All questions
Q15
All questions except Q9
Q16
All questions
Q17
All questions
Q18
All questions
Q19
All questions
Table 4: Significant Correlation between Self-Reports on CSUQ Questionnaire (N = 59)

10. Results and Discussion 5
Ques&ons
Minimum
Maximum
Mean
Std.
Devia&on
Q1
1
9
6.12
1.848
Q2
0
9
5.64
1.989
Q3
0
9
5.25
2.411
Q4
0
9
4.75
2.279
Q5
0
9
4.97
2.304
Q6
0
9
5.03
2.573
Q7
0
9
6.24
2.254
Q8
0
9
6.25
1.944
Q9
0
9
5.73
2.148
Q10
0
9
4.42
2.183
Q11
0
9
5.69
2.053
Q12
0
9
5.71
2.407
Q13
0
9
3.93
2.684
Q14
0
9
3.81
2.549
Q15
0
9
4.73
2.211
Q16
0
9
4.64
2.517
Q17
0
9
4.27
2.180
Q18
0
9
4.37
2.189
Q19
0
9
4.14
2.583
Q20
0
9
4.81
2.467
Q21
0
9
5.51
2.176
Q22
0
9
2.12
2.841
Q23
0
9
2.76
2.654
Q24
0
9
4.69
2.541
Table 5: Descriptive Statistics of the Self-reports on the QUIS
Questionnaire
59.4
64.8
54.2 49.2 44.2
0
10
20
30
40
50
60
70
PercentageofAgreementwiththe
Statement
Different Categories on the Questionnaire
Category 1 = Overall Reaction to the Website; Category
2 = Web Page; Category 3 = Terminology and Website
Information; Category 4 = Learning; Category 5 =
Percentage of Positive
Responses on Different
Categories in QUIS
Questionnaire
Percentage of
Positive Response on
Different Categories
Figure 2. Percentage Agreement with the Different Categories in QUIS
Questionnaire

11. Results and Discussion 6
Question
Number
Significant Correlation with other questions
Q1
All questions except Q2, Q10, Q14, Q16, Q23, Q24
Q2
No questions except Q22
Q3
All questions except Q2, Q10, Q11, Q17, Q22, Q23
Q4
All questions except Q2, Q7, Q8, Q9, Q10, Q11, Q12, Q15, Q16, Q17, Q18, Q20,
Q24
Q5
All questions except QQ2, Q5, Q11, Q12, Q19, Q20, Q22, Q23
Q6
All questions except Q2, Q6, Q10, Q13, Q14, Q15, Q16, Q17, Q22, Q23, Q24
Q7
All questions except Q2, Q4, Q10, Q13, Q16, Q17, Q22, Q23, Q24
Q8
All questions except Q2, Q4, Q13, Q14, Q16, Q19, Q20, Q22, Q23, Q24
Q9
All questions except Q2, Q4, Q13, Q14, Q19, Q20, Q22, Q23
Q10
All questions except Q1, Q2, Q3, Q4, Q6, Q7, Q12, Q13, Q14, Q19, Q20, Q22, Q23
Q11
All questions except Q2, Q3, Q4, Q5, Q11, Q13, Q21, Q22, Q23, Q24
Q12
All questions except Q2, Q4, Q5, Q10, Q12, Q13, Q15, Q16, Q19
Q13
All questions except Q2, Q6, Q7, Q8, Q9, Q10, Q11, Q12, Q13, Q15, Q16, Q17, Q18,
Q20, Q21, Q24
Q14
All questions except Q1, Q2, Q6, Q8, Q9, Q10, Q14, Q15, Q16, Q17, Q19, Q20, Q22,
Q23, Q24
Q15
All questions except Q1, Q2, Q4, Q6, Q12, Q13, Q14, Q15, Q19, Q20, Q22, Q23
Q16
All questions except Q1, Q2, Q4, Q6, Q7, Q8, Q12, Q13, Q14, Q16, Q20, Q21, Q22,
Q23
Q17
All questions except Q2, Q3, Q4, Q6, Q7, Q13, Q14, Q17, Q19, Q20, Q22, Q23, Q24
Q18
All questions except Q2, Q4, Q13, Q18, Q22
Q19
All questions except Q2, Q5, Q8, Q9, Q10, Q12, Q14, Q15, Q17, Q19, Q24
Q20
All questions except Q2, Q4, Q5, Q8, Q9, Q10, Q13, Q14, Q15, Q16, Q17, Q20, Q24
Q21
All questions except Q2, Q11, Q13, Q16, Q21, Q23
Q22
All questions except Q1, Q3, Q5, Q6, Q7, Q8, Q9, Q10, Q11, Q12, Q14, Q15, Q16,
Q17, Q18, Q22, Q24
Q23
No questions except Q4, Q13, Q18, Q19, Q20, Q22, Q24
Q24
No questions except Q3, Q5, Q9, Q10, Q15, Q16, Q18, Q21, Q23
Table 6: Significant Correlation between Self-Reports on QUIS Questionnaire (N = 59)

12. Results and Discussion 7
Mean
Rank
Chi-­‐
Square
Asymp.
Sig.
Q1
11.19
126.626
.000
Q2
11.62
Q3
9.54
Q4
8.3
Q5
8.88
Q6
10.4
Q7
10.14
Q8
7.2
Q9
8.14
Q10
7.08
Q11
12.19
Q12
11.83
Q13
11.12
Q14
10.4
Q15
13
Q16
10.95
Q17
8.69
Q18
7.71
Q19
11.62
Table 7: Friedman Test Values for the 19 Questions in CSUQ Questionnaire

13. Ques%on
Categories
Mean
Rank
Chi-­‐Square
Asymp.
Sig.
Q1
Comprehensive
evalua%on
3.690
21.532
.000
Q2
3.070
Q3
2.960
Q4
2.690
Q5
2.590
Q6
Web
Page
Design
1.580
24.818
.000
Q7
2.260
Q8
2.160
Q9
Terminology
and
Website
Informa%on
4.260
60.670
.000
Q10
2.920
Q11
4.110
Q12
4.220
Q13
2.860
Q14
2.640
Q15
Learning
3.210
5.769
.217
Q16
3.210
Q17
2.760
Q18
2.810
Q19
3.000
Q20
Website
Capabili%es
3.490
75.683
.000
Q21
3.900
Q22
1.950
Q23
2.350
Q24
3.310
Results and Discussion 8
Table 8: Friedman Test Statistics for the 5 Different Categories in the QUIS Questionnaire

14. Results and Discussion 9
Words
with
Maximum
Frequency
(Top
10)
Frequency
Convenient
32
Clean
29
Dull
29
Slow
28
Helpful
27
Useful
26
Accessible
24
Engaging
23
Fun
23
Usable
23
Table 9. Words chosen with Maximum Frequency

15. Conclusion and Future Work
• Production
-L2 readers have wide variability in the efficiency scale, when
analyzing an English website.
• Future Work
- This will help us judge their levels of proficiency and the types
of English websites they could be exposed to for various kinds of
assignments, task-based language learning etc.

16. Thank you