This presentation consists of the storyboard of our CSCW2012 video submission. In the video's storiboard we demonstrate the practical application of research on human and machine annotation of online documents to support reflective reading and collective sensemaking of online documents. We present an innovative research prototype which integrate a discourse analysis software (XIP) with our Cohere Web Annotation and Knowledge-Mapping tool. We visualize an interactive scenario of use of the two integrated technologies in a unique user experience. This dynamic scenario will give an inspiring vision of future CSCW systems, which machine and human intelligence are combined to enhance reasoning power.

1. Story board for the CSCW2012:
Video Submission
Cohere+XIP: Human Annotation
Harnessing Machine Annotation
Power
Anna De Liddo Anna , Ágnes Sándor, Simon Buckingham Shum
KMi - Knowledge Media Institute- The Open University, Milton Keynes, UK
Xerox Research Centre Europe

2. Storyboard description
This storyboard aims at describing the main ‘moves’ that will be made in the
video submission. Each page of this storyboard is composed of an image
capturing the visual aspect of the video and a text describing the rationale of
the video step by step.
The video submission will consist of a continuous sequence of video clips and
images with background music and a narrating voice describing the video
story line.

3. First we describe the objective of the video: demonstrate the practical application of research
on human and machine annotation of online documents to support reflective reading and collective
sensemaking of online documents. We present an innovative research prototype which integrates a
textual analysis software (XIP) with an open source Web annotation and knowledge-mapping tool
(Cohere). …(short description of the tools)

4. Then we show the practical application of the new prototype by describing a use case: A user
browses a web resource on Open Education, this has been analyzed by XIP which highlighted and
classified salient content….see in the sidebar the XIP created annotations.
By clicking on each annotation the text of the document scrolls down and higlights just that specific
snippet of text extracted by XIP….

5. Extract
Connect & Create
The user reads the higlighted text and identifies one clip that is relevant for “Promoting Openness in Archeology and
Heritage” so he clicks on “connect” and “create” and connects XIP annotation to a new annotation and labels it as
“Promoting Openness in Archeology and Heritage”…..(video shows how to create and connect through the browser
sidebar)

6. The users keeps reading and finds another clip extracted by XIP that is relevant for “Promoting
Openness in Archeology and Heritage”…so he connects XIP machine annotation with his newly
created annotation…he keeps reading XIP extracted sentences and identifies a third relevant
sentence…..

7. Machine Annotation
Human Annotation
The user has used machine annotation to quickly identify key sentences for his investigation, and then
he has distilled and clustered them under a refined label (human annotation)
He can then explore his annotations by visualizing the network of relevant machine and human
annotations which have been connected to them…

8. Annotator’s Picture
Now the video move to a more Demo style it presents some specific features in action to give a proof
of context of the prototype tool potential.
What happen when there is more then one annotator?
Annotations in the side bar of the browser will show more faces (video shows an
example)….

9. What happen when there is more then one annotator?
The annotations’ network shows authors of the annotation…(see picture on the right hand
side of each node)

10. Human Annotation
Machine Annotation
What happen when there is more then one annotator?
The social network visualization will give a quick sense of the number and type (human/
machine) of annotators of a specific document (the document being the node at the center of the
star graph…)

11. One Document
100 Documents
20 Documents
What happen when there is more then one document?
XIP annotations can be visualized in COHERE as nodes clustered around a central document node…
Video shows the growing complexity of the network graph of machine annotations as soon as the
number of annotated documents increases…

12. 20 Human Annotation:
Documents
Connecting
Merging
?
Re-tagging
Summarising ?
? ?
What happen when there is more then one document?
….This exemplifies the important role of our mixed initiative approach which uses human intelligence
to spot connections between different documents, and to distill, summarize and cluster relevant
annotations from noise….
BUT the user needs support to EXPLORE FILTER AND MAKE SENSE of the growing number of
annotations

13. Filter by Author
Tag Cloud
Filter by Annotation Type
Support to SENSEMAKING:
The user can explore the annotations by tag (tag cloud)…or filter by annotation type and by
Annotation Author….
Video will show how to filter and explore…

14. Support to SENSEMAKING:
The user can also filter the annotation network (human and machine generated) by SEMANTIC
CONNECTION (i.e. To filter annotations which highlights QUESTIONS users can filter by “describes
an open question in” , “is raised in” , “suggests an open question in” etc ..)…video will show how to
perform semantic structure searches of network of annotations.
.

15. MACHINE
ANNOTATION: XIP
BLENDED ONLINE
OFFLINE
ANNOTATION:
UTOPIA TOOL
MACHINE-HUMAN
ANNOTATION ECOSYSTEM
COHERE:
SOCIAL
ANNOTATION
NETWORK INTEGRATION AND
VISUALIZATION INTEROPERABILITY: THE COHERE: HUMAN
AND ANALYSIS
(AO) ANNOTATION ANNOTATION
ONTOLOGY
THE VISION:
The video concludes by suggesting a vision of future CSCW systems, which combines human
annotation to harness machine analysis and reasoning power.
Those systems will use mixed-initiative approaches, will allow GLOBAL DISTRIBUTED human and
machine annotation across different annotation tools which share a common annotation language
(Exploring different annotation ontologies: i.e. AO or OA),will ENABLE BLENDED online and off line
annotation (Form PDF annotation to Web document annotation)….Video will also show initial results
of parallel research project to combine XIP Cohere and Utopia (a tool for PDF social annotation)…

16. Thanks for your time
Anna De Liddo Anna, Ágnes Sándor, Simon Buckingham Shum
http://kmi.open.ac.uk/people/anna/index.html