Eye-based head gestures
•Descargar como PPSX, PDF•
2 recomendaciones•1,743 vistas
Mardanbegi, D., Hansen, D.W., and Pederson, T. “Eye-based head gestures: Head gestures through eye movements”. Proceedings of the ACM symposium on Eye tracking research & applications ETRA '12, ACM Press, California, USA, 2012. (Awarded as best full paper + best student paper)
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (7)
Similar a Eye-based head gestures
Similar a Eye-based head gestures (14)
Último
Último (20)
Eye-based head gestures
- 1. Eye-based head gestures Diako Mardanbegi Dan Witzner Hansen Thomas Pederson IT University of Copenhagen
- 2. This paper is about… IT University of Copenhagen
- 3. Gaze pointing Click Dwell-time Blink Fixation+Saccade (Fixation) DbClick Two steps Fixation+Saccade DbBlink Dwell-time More commands Gaze gestures
- 4. Gaze gestures Off-screen targets Eye-writer Mostly used for eye typing [Isokoski 2000] ! [Wobbrock, et.al 2008] Complex patterns are needed Unnatural Cognitive load Loosing focus on object IT University of Copenhagen
- 5. Gaze pointing Click Dwell-time Fixation+Saccade Wink (Fixation) (e.g., context switching) Head gestures DbClick Two steps Fixation+Saccade measured by DbWink Dwell-time an eye tracker More commands Gaze gestures
- 6. Video-based head gesture recognition [Nonaka 2003] [Kapoor and Picard 2002] IT University of Copenhagen
- 7. IT University of Copenhagen
- 8. Yaw Linear eye movements Pitch Roll Rotational eye movements Pt = [L, R] IT University of Copenhagen
- 9. Basic movements P = [L, R] P = [(x, y),(Dx, Dy),(r1, r2,.., r8)] P Basic movement classifier Hi IT University of Copenhagen
- 10. GESTURES Continuous gestures Discrete gestures Repetitive gestures Sweep gesture
- 11. Application Examples IT University of Copenhagen
- 12. Classification of gestures Character: Discrete gestures: repeatable and recognizable sequence of characters, Cij=Ci Cj Cit Gesture Gesture classifier Application state IT University of Copenhagen
- 13. Fixation on objects + Head movements i. fixed-head eye movements ii. fixed-gaze eye movements Remote eye trackers (Only eye image): Using the reflection of a fixed light source (glint) Head-mounted eye trackers (eye image + scene image): Using the information obtained from the scene image IT University of Copenhagen
- 14. Eye tracker Method implemented on a head-mounted eye tracker: Accuracy of about 1.5° Eye/scene images resolution: 640x480 25 frames per second in real time Pupil detection: feature-based method Gaze estimation: homography mapping Detecting the display corners in the scene image IT University of Copenhagen
- 15. Testing the classifier 14 predefined gestures have been tested ` 8 participants (6 male and 2 female, mean=35.6, SD=9.7) Task: Looking at a marker on the screen and then asking the user to do the gestures Each gesture has been shown 2 times for each participant IT University of Copenhagen
- 16. 4 participants were not All participants able to perform were able 4 False trials 3 2 1 0 ` Gestures False trials because: Unable to perform predefined gestures Simplicity of the classifier Unable to fixate on the marker during gesture IT University of Copenhagen
- 17. Experimental applications iRecipe iiPhone Read and follow recipes by Controlling the iPhone emulator looking and head gestures by head gestures
- 18. Investigating the participants experience in terms of physical effort and the level of difficulty difficulty physical effort High 5 4 Answer 3 2 Low 1 IT University of Copenhagen
- 19. Summary Detecting head movements using eye and gaze information using both pupil and iris pattern. Can keep gaze at the object while interacting Found reliable gestures that were comfortable and easy to do Showed two examples of potential use IT University of Copenhagen
- 20. Future work Improving the accuracy of the classifier by learning user specific gestures Apply method to control everyday real world objects Use the method in a remote eye tracker IT University of Copenhagen
- 21. ?
Notas del editor
- AVALESH AROOM SHOO KON
- In the keynote on wendsday Andrew talked about accelerometers in the smartphones . I’m gona talk about the natural accelerometer that we have in our body call vestibular system that sencethe head movement. The basic idea of this paper is that when you look at an object and move your head your eyes move in the oposit direction because of the vistibulo ocular reflexThis reflexive eye-movements stabilize the eyes in space when the head moves, providing a stable image on the retina..
- gaze based interaction basicly started by gaze pointing. However The point of regard does not provide sufficient information for interactingwith interfaces, and this is known as the midas touch problemIntentional blinking and dwell time are typical ways of sending 1 bit information (that can be used for clicking). Two steps dwell time and Blinking twice are also used when we need some more commands (such as dbclickRclick).However, interaction with blinking especially repetitive blinking for long-term use is not very cofrtable. Beside that in some applications we need more command and these methods do not provide more commands. Another method is to use saccades+fixations.Using the intentional eye movements for interaction is known as gaze gestures which can be defined as definable patterns of eye movements performed within a limited time interval
- Gaze gestures are mostly used for eye typing.As far as we know it strated by isokosi by the off-screen targets. Where ehe eye gaze has to visit the off-screen targets in a certain order to select characters. Eye writer…In terms of interaction gaze gestures are facing several limitations, for example:Simple gaze gestures can not be distinguished from natural eye patterns. Therefore Complex gaze gestures consist of several simple gaze gestures are needed for robust results. Using the perceptual channel such as vision for motor control may be considered unnatural Besides, the user needs to memorize combinations of gaze gestures,and therefore takes the focus away from the actual interaction task, and increases the cognitive load.The other limitation is that when performing the gaze gestures, the point of regard leaves the object while interacting. So it is not a good method for selecting the icons or interaction with real objectsPEOPLE CAN EASILY GET UNCOMFORTABLE BY DOING GESTURE WHEN I WANT to control an object, I DON’T LIKE TO LOOK SOMEWHERE ELSE
- In contrast we sugest to use the head gestures.We offtenuse the head gesturesin our daily life for communication. WE DO NOT USE GAZE GESTURES FOR COMMUNICATIONmost gaze based interactive applications are intended for desabled people who can only move their eyes. Why shouldn’t eye trackers be used for the general population. For example People can move their heads and this can be used for interaction.
- Head gesture recognition is well known in computer vision. The general problem are not able to separate the head gestures from the natural head movements There have been attepmts to use the eye information….Only shakes and nodsIn another work they have used an additional sensor (motion tracker) instead of the natural accelerometer. They have used motion tracker together with eye tracker. Eye tracker for … and motion tracker for ….
- In contrast, the presented method in this paper, allows for identifying a wide range of head movements even the smallmovemnts accurately and in real time, by only using an eye tracker. What we do is only use the information observed by the cameras of the eye tracker. Here I am showing the example of using the head mounted, but the method applies TO remote eye trackers as well.
- There are Three canals in our inner ear that sences the roll, picth and yaw of the head and reflect these movements to the eyes. (for stabilizing the image on the retina)Yaw and pitch create what we call linear movements and roll create what we call rotational..Therefore we can extract a set of two types of feature vectors from the eye image. One part of the the feature vectors are for detecting the rotational eye movements (R) and the other one for linear (L) (pi at time t)
- The first part of the feature vector is the pupil center and velosity of the center. the second part of the feature vector which is for detecting the eye rotations. IN OUR IMPLEMENTATION WE HAVE USEd THE MEAN OF THE OPTIC FLOW WITHIN 8 regions around the pupilNow we have the feature vector. We have used a classifier for detecting the basic head movements between two frames of the video sequence by using the feature vector.In this image we the basic movements of head & corresponding Basic reflexive movements of the eyeWe have defined different head gesture using the basic head movements
- We have defined two types of gestures:Continuous gestures are defined as the sequences of basic eye movements along an axis Repetitive gestures are where you move the head twice in a same directionSweep when you move the head in one direction and return in the oposit direction. Discrete gstures can be horizental, vertical or diagonal
- You can change the volume by continiusly moving the head up and down
- Continiuos…Measuring sequences of eye movement are usually influenced by noise. We define a character, Ci, as a sequence of N small eye movements between each two frames where the majority of movements are the same Simple discretegestures,Gij=CiCj are 2 character wordsThere are in total 64 but some of them are not phisicly possible so we only use 14
- There are Two types eye movements:So we need a method for detecting whether the gaze is fixed on an object ORFor RT we already have the reflection of a fixed light sourceFor the HMET we need to analize the scene image. In this paper we have used this idea in slitly more dificult setting by using a head mounted eye tracker. The intended use is to apply this technique for gaze based interaction in highly mobile settings.The main advantage of this method with compare to the gaze gestures is that the gaze does not leave the object during the interationWe assume that there is a computer display….
- In our own eye tracker
- WE HAVE MADE AN EXPERIMENT TO INVESTIGATE THE ACCURACY O FTHE CLASSIFIER. We defined 14 …. As shown here (describe up down diagonal…)We use 8 participants.The method and gestures were introduced to participants and they had the chance of practicing the gestures for 10 minutes before the experiments. 14 simple gestures were shown on the screen by a simple figure, two times one by one and randomly. The shown gesture remains on the screen until the user performs the same gesture or pressing a key in the case when the user was not able to perform that gesture. Each time that a participant performs a gesture but it is not recognized correctly, it will considered as a false trial.
- Horizental:..Vertical: average number of false trials of all participants for each gesture.Our general observations were4 participants were not able to perform the diagonal gestures meaning that they were unnaturalRepetitive down gesture is inconvenient
- We made two examples to show the potential use of this method for interaction. Four different sweep gestures including Up, Down, Left and Right together with the continuous vertical head movements were used for controlling two applications. Each gesture is interpreted differently based on the gazed object. iRecipe, is intended for a hands-free interaction with a recipe when cooking and when the hands are occupied.iiPhonewhich is an iPhone emulator running on the screen that can be controlled by head gestures to show the potential of thismethod for the mobile devices. WE HAVE THREE REGIONS. A MUSIC IS PLYING DURING COOKING. TO CONTROL THE VOLUME THE USER FIXATES ON THE ….IN THE IIPHONE SWEEPING BETWEEN THE PAGES..We did not meature the false trials but what we observed was that all participants did the tasks susefuly.It is easier to keep the gaze fixed on a meaningful object Participants more liked the volume control by continuous gesture because of the real time sound feedback
- After the test, the participants were given a questionnaire consists of questions with the range of the answers from 1 to 5 to investigate the participants experience in terms of physical effort and the level of difficulty of THREE DIFERENT TYPES OF GESTURESthis is the avarage score of different questions which all were about the level of difficulty…..
- Thank you very much