Presentation given at ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, September 2007 ABSTRACT: With the emergence of digital pen and paper interfaces, there is a need for gesture recognition tools for digital pen input. While there exists a variety of gesture recognition frameworks, none of them addresses the issues of supporting application developers as well as the designers of new recognition algorithms and, at the same time, can be integrated with new forms of input devices such as digital pens. We introduce iGesture, a Java-based gesture recognition framework focusing on extensibility and cross-application reusability by providing an integrated solution that includes tools for gesture recognition as well as the creation and management of gesture sets for the evaluation and optimisation of new or existing gesture recognition algorithms. In addition to traditional screen-based interaction, iGesture provides a digital pen and paper interface.

1. A General Gesture Recognition Framework
Beat Signer, Ueli Kurmann and Moira C. Norrie
Institute for Information Systems, ETH Zurich
8092 Zurich, Switzerland
http://vub.academia.edu/BeatSigner
ICDAR 2007, September 26

2. Interactive Paper
Digital Pen and Paper
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 2

3. Requirements
 Java API
 Different types of input devices
 digital pen and paper, mouse, etc.
 Support application and algorithm developers
 Definition and evaluation of new gesture sets
 Simple acquisition of gesture data
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 3

4. iGesture Components
Management Evaluation
Console Tools
Recogniser
Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 4

5. Management Evaluation
Console Tools
Recogniser
iGesture Admin Tab Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 5

6. Management Evaluation
Console Tools
Recogniser
Capture of Samples Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 6

7. Management Evaluation
Console Tools
Recogniser
iGesture Test Bench Tab Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 7

8. Management Evaluation
Console Tools
Recogniser
iGesture Test Data Tab Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 8

9. Management Evaluation
Console Tools
Recogniser
Evaluation Tools Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 9

10. Management Evaluation
Console Tools
Recogniser
Recogniser API Common Data Structures
Configuration configuration =
XMLTool.importConfiguration(new File(RUBINE));
GestureSet gestureSet =
XMLTool.importGestureSet(new File(RUBINE_SET));
configuration.addGestureSet(gestureSet);
recogniser = new Recogniser(configuration);
...
ResultSet result = recogniser.recognise(note);
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 10

11. Management Evaluation
Console Tools
Recogniser
Gesture Representation Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 11

12. Management Evaluation
Console Tools
Recogniser
Recognition Algorithm Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 12

13. Management Evaluation
Console Tools
Recogniser
Persistence Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 13

14. Current Status
 Algorithms
 Rubine, SiGeR, E-Rubine, SiGrid
 Gesture Sets
 Graffiti numbers, Graffiti letters, MS application
gestures, customised multi-stroke gestures
 Input Devices
 Anoto digital pen (e.g. Magicomm G303), mouse
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 14

15. Conclusions
 Simple gesture recogniser Java API
 Experimental platform for implementing and
testing new gesture recognition algorithms
 Definition and evaluation of new gesture sets
 Extensibility in terms of
 input devices
 algorithms
 gestures sets
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 15

16. References
 iGesture Framework,
Apache License, V2.0
 http://www.igesture.org
 B. Signer, M. C. Norrie and U. Kurmann. iGesture: A
Java Framework for the Development and Deployment
of Stroke-Based Online Gesture Recognition Algorithms.
Technical Report ETH Zurich TR 561, September 2007
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 16