For more than two decades, the HCI community has elaborated numerous tools for user interface evaluation. Although the related tools are wide, the evaluation remains a difficult task. This paper presents a new approach for user interface evaluation. The proposed evaluation process focuses on utility and usability as software quality factors. It is based on the UI ergonomic quality inspection as well as the analysis and the study of the Human-Computer interaction. The proposed approach is mainly based on graphic controls dedicated to the user interface evaluation. These controls have, on the one hand, the role to compose graphically the interfaces. On the other hand, they contribute to the UI evaluation through integrated mechanisms. The evaluation is structured into two phases. The first consists of a local self-evaluation of the graphical controls according to a set of ergonomic guidelines. This set is specified by the evaluator. The second allows an electronic informer to estimate the interaction between the user interface (graphically composed by the evaluation based controls) and the user.

1. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Graphical controls based environment for user
interface evaluation
Selem CHARFI, Abdelwaheb TRABELSI, Houcine EZZEDINE
and Christophe KOLSKI
Laboratoire d’Automatique, de M´canique, et d’Informatique industrielles et
e
Humaines LAMIH CNRS 8201
Raisonnement Automatique et Interaction Homme-Machine
Toulouse. October 31th, 2012
4th international conference on Human-Centered Software Engineering
1/24

2. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Content
1 Introduction
2 UI Evaluation Tools
3 Evaluation Based Controls
4 Conclusion and Future work
2/24

3. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Introduction
UI evaluation is :
essential for interactive systems validation and test [Nielsen, 94].
important to create effective and usable interfaces and screens
[Galitz, 08].
UI evaluation essentially aims at :
identifying potential UI use problems leading to use problems
[Nielsen, 93].
improving UI acceptability [Zhang et al., 99].
establishing comparison between several design alternatives
[Mayhew, 99].
protecting user from erroneous actions (data lost, undesirable
situations) [Rubin, 08].
improving systems’ efficiency[Galitz, 08].
3/24

4. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
UI Evaluation Tools
Many works exist (tools, methods, technics, etc.). The UI
evaluation tools are so numerous. Although there are many
difficulties :
Ergonomic Guidelines exploitation is difficult to establish
[Keith, 05].
The evaluation results are difficult to analyze and to interpret
[Charfi et al., 11].
UI evaluation is generally a neglected process by many
designers [Grislin and Kolski, 96].
The evaluation results can vary from a method to another one
and from an evaluator to the other for the same UI [Nielsen, 93].
etc.
4/24

5. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
UI Evaluation Tools
An UI evaluation existing tools study revealed that [Charfi et al.] :
1 The majority of them are used at the final system design
phase (Test Phase).
2 UI evaluation tools should be easier and simpler to be
established to encourage designers to proceed to the
evaluation.
3 Most of existing tools are based on only one evaluation
method (Electronic Informer, Ergonomic Guideline inspection,
questionnaire, etc).
4 Most of the existing tools focus on a single aspect for the
evaluation (static UI display, the system functional kernel,
etc).
5 Most of existing tool offers automated capture and analyze
phases. The UI improvement suggestions and use problems are
handled manually by the evaluator.
5/24

6. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Global Overview
Purpose :
1 automating UI evaluation process.
2 supporting the evaluation process since early system design
phases.
3 couple between Design and Evaluation phases.
4 simplifying evaluation process that design can proceed to it
easily.
Concept : Integrate evaluation mechanism into graphical control.
6/24

7. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
7/24

8. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Evaluation Based Controls
Evaluation Based Controls
1 Controls dedicated to UI Usability Inspection (Static UI
display).
2 Controls dedicated to UI Utility Inspection (Interaction
between User and the GUI).
8/24

9. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Evaluation controls dedicated to Usability Inspection
”It may be 100 times more costly to proceed for UI
improvement at a late stage that early one” [Nielsen, 94].
The proposed controls ”evaluate” themselves according to a
set of EG : they check their coherency according to EG.
The EG are defined into XML files by the evaluator into XML
files through a tool entitled ”Guideline Manager”.
9/24

10. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
10/24

11. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
11/24

12. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
12/24

13. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
13/24

14. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Evaluation controls dedicated to Utility Inspection
The evaluation process :
consist in comparing between different actions sequences.
Planned action sequence : Referential Model.
User actions sequence : Object Model.
is based on CTT notation for task modeling [Paterno, 97].
is based on these controls and an Electronic Informer.
They have a client-server architecture.
14/24

15. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Evaluation controls dedicated to Utility Inspection
Roles :
Evaluation Controls
Communicating interaction data to the electronic informer :
elementary action execution time ;
action type (button click, check-list select, etc.) ;
associated form (the interface containing the graphical
control) ;
graphical control text ;
control type (button, text-box, label, combo-box, etc.) and,
IP machine address.
The Electronic Informer
Assist the evaluator to generate Referential Model ;
Capture interaction data ;
Generate Object Model ;
Comparing between Referential and Object Model and,
Establish comparison statistics.
15/24

16. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
The Electronic Informer
The EI is structured following a modular architecture
1 Referential model generator ;
2 Evaluated object model generator ;
3 Confrontation and,
4 Statistics generator.
16/24

17. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
(1) Referential Model Generator
This module is used to elaborate a description of the tasks
that are executed by the user during the test phase.
The task is expressed through its sub-tasks using CTT
notation
The task trees are specified by the evaluator.
The evaluator associates through this module the elementary
actions tasks to the task model in order to generate
Referential Model.
17/24

18. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
(2) Object Model Generator
This module is used to :
capture the user actions sequence while interacting with the
UI and,
store these captured data for the confrontation process.
Note : The interaction sequence is realized separately for each task.
18/24

19. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
(3) Confrontation Module
This module is used to compare between Referential and
Object model.
This comparison is elaborated in order to detect missing,
repetitive, useless and erroneous actions.
19/24

20. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
(4) Statistics Generator
The Statistics Generator module propose to the evaluator :
task execution rate ;
tasks and sub-tasks execution average and,
the comparison between the two models.
20/24

21. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Conclusion
UI Evaluation
is the object of numerous researches.
remains a difficult task.
A contribution for UI early evaluation to obtain useful and
usable UI.
The proposed UI evaluation approach limits :
Utility inspection controls do not identify clearly the design
problems
Difficulties to exploit EI provided statistics by the EI
The proposed usability inspection controls take into
consideration only simple EG
21/24

22. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Future work
1 Exploit web services to provide better operability for
evaluators ;
2 Couple between the two control families to get control
inspecting both utility and usability ;
3 Conceive ”Intelligent” controls that handle usability and utility
problems and,
4 Express Evaluation result using evaluation standards (EARL,
RDL, CIF, etc.).
22/24

23. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
23/24

24. Introduction UI Evaluation Tools Evaluation Based Controls Conclusion and Future work
Graphical controls based environment for user
interface evaluation
Selem CHARFI, Abdelwaheb TRABELSI, Houcine EZZEDINE
and Christophe KOLSKI
Laboratoire d’Automatique, de M´canique, et d’Informatique industrielles et
e
Humaines LAMIH CNRS 8201
Raisonnement Automatique et Interaction Homme-Machine
Toulouse. October 31th, 2012
4th international conference on Human-Centered Software Engineering
24/24