Track 14. 9th International Workshop on Software Engineering for E-learning (ISELEAR’18) Authors: Joaquín Gayoso-Cabada, Antonio Sarasa-Cabezuelo and José-Luis Sierra https://youtu.be/_-kkPLGPPPI

1. Document Annotation Tools: Annotation
Classification Mechanisms
Joaquín Gayoso, Antonio Sarasa, José Luis Sierra
Universidad Complutense de Madrid
Grupo ILSA

2. Documents annotation tools
• With these tools, students can associate annotations with fragments of
documents.
• They facilitate some activities of the learning process:
• Analysis of content.
• Develop the meta-reflective thinking.
• Collaborative annotation among students.
• Implement innovative interaction mechanisms.

3. Annotation Classification Mechanisms
• These tools provide mechanisms for classifying annotations which are
essential in order to :
• Retrieval of relevant annotations.
• Evaluate of work done by students.
• Promote the collaborative work in annotation communities.
• Help the students during annotation activities

4. Annotation Classification Mechanisms
• This paper presents a study about the mechanisms for classifying
annotations.
• Conditions:
• It is limited to tools oriented to the annotation of documents, where text is the
predominant medium.
• It was used recent surveys on the subject and free searches of primary scientific sources
(in particular, ScienceDirect, ACM Digital Library, IEEE xplore, and Google Scholar).
• It was selected 38 annotation tools.

5. Annotation Classification Mechanisms
• As results, it was identified 5 main annotation classification strategies:
• Tools that lacked specific classification mechanisms.
• Classification based on presentational attributes.
• Classification based on predefined semantic categories.
• Classification based on the use of folksonomies
• Classification based on the use of ontologies.

6. Tools that lacked specific classification
mechanisms.
• In this approach, there is no specific mechanism for classifying annotations.
In this way, these tools prioritize other aspects, such as more natural
interaction mechanisms, rather than the classification of annotations.
• Tools: Digital Reading Desk , Livenotes, WriteOn, PaperCP or u-Annotate.

7. Classification based on presentational
attributes.
• In this approach, the tools use the different ways of annotating a document
(e. g., underlining or highlighting fragments of the text, adding comments,
etc.) that it is used in order to define an implicit categorization of the
annotations (i. e., underlining, highlighting, explicit text, etc.). These
categories use the presentational attributes of the annotations.
• Tools: Adobe Reader, PDF Annotator , Diigo , CASE tool, CON2ANNO ,
Anozilla annotation plug-in, Vpen ,IIAF

8. Classification based on predefined semantic
categories.
• The tools that follow this approach uses a set of semantic tags for
annotation classification. These tags are related to the semantics of
annotations and not to their presentation characteristics.
• Tools: eLAWS, Highlight, PAMS 2.0 , MyNote, Tafannote, WCRAS-
TQAFM, CRAS-RAID, MADCOW

9. Classification based on the use of folksonomies
• In this approach, for the classification of annotations it is used folksonomies
(tags created by the users).
• Tools: HyLighter, Hypothe.sis, A.nnotate, Note-taking, OATS,
SpreadCrumbs, Tsaap-Notes.

10. Classification based on the use of ontologies.
• In this approach, the tools can be used explicit ontologies that represent
different aspects of the annotation process. Students can use these
ontologies to make the semantics of annotations explicit by associating one
or more concepts taken from the ontologies to the notes.
• Tools: loomp, DLNotes , MemoNote, WebAnnot, and @note

11. Discussion
• The study should showed the actual
tendencies in this domain because we
did not prioritize one type of tool
over another (however for tools based
on ontologies, we did an exhaustive
search and we found no more tools)
5
10
9
8
6
0
2
4
6
8
10
12
NONE ANNOTATION
MODES
PREDEFINED
SEMANTIC
CATEGORIES
FOLKSONOMIES ONTOLOGIES
numberoftools
Classification mechanism
Number of tools analyzed per annotation
classification method

12. Discussion
• The predominant tool type is the one based on annotation modes.
• The second most frequent type of tool is based on predefined sets of
semantic categories.
• Tools based on folksonomies are in third place.
• Finally, we have tools that do not include specific mechanisms for classifying
annotations and 6 tools that use mechanisms based on ontologies.

13. Educational point of view
• Tools lacking specific classification mechanisms or only enabling
presentational attributes are not very convenient from an educational point
of view:
• They imitate conventional pen and paper-based annotation mechanisms.
• It has a negative impact on learners.
• It is difficult to analyze the activities.

14. Educational point of view
• Tools supporting a set of predefined semantic categories for classification:
• It can use useful for students because they must reflect on the purpose of the
annotations in addition to deciding on the actual anchors and/or contents.
• They can offer meaningful criteria for retrieving annotations during the assessment of
the activities.
• The main limitation of the approach is the pre-defined character of these lists of
categories.
• They have the problem of lack of structure of simple lists.

15. Educational point of view
• Tools supporting folksonomies:
• They promote the reflection during the annotation process
• The main limitation is to delegate the collaborative design of these folksonomies to the
learners
• They have the problem of lack of structure of simple lists

16. Educational point of view
• Tools supporting ontologies:
• It is possible to capture specific knowledge about annotation.
• It provides a high degree of contextualization of the tool to each specific activity.
• The structural of ontologies solves the problem of lack of structure of simple lists
• The main limitation is the need of a background in computer science or knowledge
engineering.

17. Conclussions
• We have done an exploratory study of annotation classification mechanisms.
• It was analyzing 38 document annotation tools and we have found five basic
forms of classification.
• From the point of view of educational use, the tools that allow semantic
classification are more appropriate.
• We have found a high percentage of tools that do not pay attention to this
aspect of semantic classification, which can hinder their educational use.

18. Future work
• The study can be expanded by considering tools for the annotation of other
types of media (e.g., images or videos).
• The analysis can be useful when designing new annotation tools for
educational purposes.