SlideShare una empresa de Scribd logo
1 de 20
Descargar para leer sin conexión
Statechart modeling of interactive gesture-based applications




                   Statechart modeling of interactive
                       gesture-based applications

                         Romuald Deshayes                       Tom Mens

                                       Software Engineering Lab




                                                   2010-2011
  Deshayes Romuald – UMONS                                                 1 / 20
Statechart modeling of interactive gesture-based applications




Table des matières


      1   Problem Statement

      2   Proof-of-concept application

      3   Modeling interactive behaviour

      4   Statechart models

      5   The application framework

      6   Conclusion and Future Work




  Deshayes Romuald – UMONS                                      2 / 20
Statechart modeling of interactive gesture-based applications
  Problem Statement




Outline : Problem Statement




                                Problem Statement




  Deshayes Romuald – UMONS                                      3 / 20
Statechart modeling of interactive gesture-based applications
  Problem Statement




Problem Statement

      Challenge
      Develop intuitive interactive applications that are easy to maintain

      Why is it a challenge ?
              Complex behaviour
              Nondeterministic user actions
              Same events lead to different actions

      Contributions
          Gestural user interface for 3D objects manipulation
              Modeling executable behaviour using a visual formalism
              Validation with an application framework

  Deshayes Romuald – UMONS                                                   4 / 20
Statechart modeling of interactive gesture-based applications
  Proof-of-concept application




Outline : Proof-of-concept application




                    Proof-of-concept application




  Deshayes Romuald – UMONS                                      5 / 20
Statechart modeling of interactive gesture-based applications
  Proof-of-concept application




Proof-of-concept application




              3D visual drawing tool
              Uses gestures to create and manipulate 3D objects
              OpenGL graphical library
              Microsoft Kinect + NITE




  Deshayes Romuald – UMONS                                        6 / 20
Statechart modeling of interactive gesture-based applications
  Proof-of-concept application




Small video : www.youtube.com/watch?v=lVcqzWTnpMY




  Deshayes Romuald – UMONS                                      7 / 20
Statechart modeling of interactive gesture-based applications
  Modeling interactive behaviour




Outline : Modeling interactive behaviour




                 Modeling interactive behaviour




  Deshayes Romuald – UMONS                                      8 / 20
Statechart modeling of interactive gesture-based applications
  Modeling interactive behaviour




Modeling interactive behaviour

      Context
          Highly reactive event-driven systems
              Gesture-based interaction

      Proposed solutions
              Visual modeling language
                     Statecharts
                     Petri nets
              Amenable to formal analysis
              Easier to evolve
              Reduced complexity

  Deshayes Romuald – UMONS                                      9 / 20
Statechart modeling of interactive gesture-based applications
  Statechart models




Outline : Statechart models




                                  Statechart models




  Deshayes Romuald – UMONS                                      10 / 20
Statechart modeling of interactive gesture-based applications
  Statechart models




Statechart models - Hand




  Deshayes Romuald – UMONS                                      11 / 20
Statechart modeling of interactive gesture-based applications
  Statechart models




Statechart models - Component




  Deshayes Romuald – UMONS                                      12 / 20
Statechart modeling of interactive gesture-based applications
  Statechart models




Statechart models - Scene




  Deshayes Romuald – UMONS                                      13 / 20
Statechart modeling of interactive gesture-based applications
  The application framework




Outline : The application framework




                      The application framework




  Deshayes Romuald – UMONS                                      14 / 20
Statechart modeling of interactive gesture-based applications
  The application framework




The application framework


              Framework for developing interactive applications
              Executable behaviour specified using statecharts
              Client-server architecture




  Deshayes Romuald – UMONS                                        15 / 20
Statechart modeling of interactive gesture-based applications
  The application framework




The application framework II
      Features
          Generic
                     Abstract classes
                     Observer design pattern
              Communication between statecharts
              Uses Java SwingStates library




  Deshayes Romuald – UMONS                                      16 / 20
Statechart modeling of interactive gesture-based applications
  The application framework




The application framework III

                                                                                                                                                                          <<library>>
                                                                          Framework                                                                                      SwingStates
              Server extends Thread
        -serverSocket : DatagramSocket                                                                                                                                     StateMachine

                                                          Controller                                     left   0..1
                            1..*              -machine : StateMachine                                                                Pointer
                <<Interface>>                                                                           right             -open : boolean
                                              +addComponent(c : Component)                                      0..1
            BodyListener extends                                                                                          -machine : StateMachine
                                              +addPointer(p : Pointer)
                EventListener                 +addScene(s : Scene)
                                                                                                                                   Scene
                                                                                                                                                                      Main Application
        +handMovedX(e : BodyEvent)                                                                               1..*
        +handMovedY(e : BodyEvent)                                                                                        -machine : StateMachine
                                                                                                                                                                            MainScene
        +handMovedZ(e : BodyEvent)                1
        +handClosed(e : BodyEvent)
        +handOpened(e : BodyEvent)
                                                                                                                        selectedModel
                                                                         1..*
                                                  *                                                                                   0..1
                                                                                     Component                                         Model
                            BodyAdapter                                                                                                                                      3DModel
                                                                     -boundingBox : Rectangle                           -selected : boolean
                   +addAsListenerOf(s : Server)                      -machine : StateMachine
                                                                                                                                                  1..*
                                                                                                                                               models



                      BodyAdapters                                                                        OpenGLWidgets
                             Hand                                GLButton                                 ExtendableMenu
                   -machine : StateMachine            +changeActiveTexture() : void        +extend() : void
                                                      +drawButton(gl : GL) : void          +retract() : void
                                                      +startCounter() : void               +drawMenu(gl : GL) : void


                                                                            SpawningMenu
                                                                 +addButton(button : GLButton) : void                                                          ImageMenu
                                                                 +addText(text : String) : void                       ButtonMenu
                                                                                                                                                         +drawMenu(gl : GL) : void
                                                                 +drawMenu(gl : GL) : void                +addButton(button : GLBUtton) : void           +onModel() : void
                                                                 +spawn() : void                          +drawMenu(gl : GL) : void
                                                                 +reduce() : void




  Deshayes Romuald – UMONS                                                                                                                                                                17 / 20
Statechart modeling of interactive gesture-based applications
  Conclusion and Future Work




Outline : Conclusion and Future Work




                    Conclusion and Future Work




  Deshayes Romuald – UMONS                                      18 / 20
Statechart modeling of interactive gesture-based applications
  Conclusion and Future Work




Conclusion and Future Work



      Statecharts
           Appropriate and scalable formalism for modeling the
           interactive gesture-based behaviour ?

      Improvements
              Externalise/decouple statechart representation from framework
              Compare statecharts with Petri nets




  Deshayes Romuald – UMONS                                               19 / 20
Statechart modeling of interactive gesture-based applications
  Conclusion and Future Work




Future Work II




              Apply our approach to other HCI applications
              Other types of natural interfaces
              Compare/combine with other UIML
              Controlled experiments, carry out user studies, empirical
              studies




  Deshayes Romuald – UMONS                                                20 / 20

Más contenido relacionado

Similar a Statechart modeling of interactive gesture-based applications

Ectel nods v2
Ectel nods v2Ectel nods v2
Ectel nods v2
nodenot
 
Mobile Enterprise Applications
Mobile Enterprise ApplicationsMobile Enterprise Applications
Mobile Enterprise Applications
Jason Conger
 
MICE: Monitoring and modelIng of Context Evolution
MICE: Monitoring and modelIng of Context EvolutionMICE: Monitoring and modelIng of Context Evolution
MICE: Monitoring and modelIng of Context Evolution
Luca Berardinelli
 

Similar a Statechart modeling of interactive gesture-based applications (20)

Controlling Computer using Hand Gestures
Controlling Computer using Hand GesturesControlling Computer using Hand Gestures
Controlling Computer using Hand Gestures
 
Windows 8.1 term paper
Windows 8.1 term paperWindows 8.1 term paper
Windows 8.1 term paper
 
Android 101 Session @thejunction32
Android 101 Session @thejunction32Android 101 Session @thejunction32
Android 101 Session @thejunction32
 
Mvi an architecture for reactive programming
Mvi an architecture for reactive programmingMvi an architecture for reactive programming
Mvi an architecture for reactive programming
 
Project Gesture & RealSense: gestures in a simple way!!
Project Gesture & RealSense: gestures in a simple way!!Project Gesture & RealSense: gestures in a simple way!!
Project Gesture & RealSense: gestures in a simple way!!
 
Reactive programming with cycle.js
Reactive programming with cycle.jsReactive programming with cycle.js
Reactive programming with cycle.js
 
Ectel nods v2
Ectel nods v2Ectel nods v2
Ectel nods v2
 
Advance ui development and design
Advance ui  development and design Advance ui  development and design
Advance ui development and design
 
Responsive app design
Responsive app designResponsive app design
Responsive app design
 
A Generic Framework for Executable Gestural Interaction Models
A Generic Framework for Executable Gestural Interaction ModelsA Generic Framework for Executable Gestural Interaction Models
A Generic Framework for Executable Gestural Interaction Models
 
Mobile Enterprise Applications
Mobile Enterprise ApplicationsMobile Enterprise Applications
Mobile Enterprise Applications
 
Project Gesture & Real Sense: il potere nelle mani!!
Project Gesture & Real Sense: il potere nelle mani!!Project Gesture & Real Sense: il potere nelle mani!!
Project Gesture & Real Sense: il potere nelle mani!!
 
Next generation of frontend architectures - Luca Mezzalira - Codemotion Milan...
Next generation of frontend architectures - Luca Mezzalira - Codemotion Milan...Next generation of frontend architectures - Luca Mezzalira - Codemotion Milan...
Next generation of frontend architectures - Luca Mezzalira - Codemotion Milan...
 
Implementation
ImplementationImplementation
Implementation
 
Prosciutto Palermo Valley Mobile Sessions - English
Prosciutto Palermo Valley Mobile Sessions - EnglishProsciutto Palermo Valley Mobile Sessions - English
Prosciutto Palermo Valley Mobile Sessions - English
 
Interactive Screen
Interactive ScreenInteractive Screen
Interactive Screen
 
Develop an app for Windows 8 using HTML5
Develop an app for Windows 8 using HTML5Develop an app for Windows 8 using HTML5
Develop an app for Windows 8 using HTML5
 
Mie presentation
Mie presentationMie presentation
Mie presentation
 
MICE: Monitoring and modelIng of Context Evolution
MICE: Monitoring and modelIng of Context EvolutionMICE: Monitoring and modelIng of Context Evolution
MICE: Monitoring and modelIng of Context Evolution
 
From Event-Driven Business Process Management to Ubiquitous Complex Event Pr...
From Event-Driven Business Process Management to Ubiquitous Complex Event Pr...From Event-Driven Business Process Management to Ubiquitous Complex Event Pr...
From Event-Driven Business Process Management to Ubiquitous Complex Event Pr...
 

Más de Tom Mens

Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and RubygemsComparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Tom Mens
 
Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)
Tom Mens
 
Empirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package ManagersEmpirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package Managers
Tom Mens
 

Más de Tom Mens (20)

How to be(come) a successful PhD student
How to be(come) a successful PhD studentHow to be(come) a successful PhD student
How to be(come) a successful PhD student
 
Recognising bot activity in collaborative software development
Recognising bot activity in collaborative software developmentRecognising bot activity in collaborative software development
Recognising bot activity in collaborative software development
 
A Dataset of Bot and Human Activities in GitHub
A Dataset of Bot and Human Activities in GitHubA Dataset of Bot and Human Activities in GitHub
A Dataset of Bot and Human Activities in GitHub
 
The (r)evolution of CI/CD on GitHub
 The (r)evolution of CI/CD on GitHub The (r)evolution of CI/CD on GitHub
The (r)evolution of CI/CD on GitHub
 
Nurturing the Software Ecosystems of the Future
Nurturing the Software Ecosystems of the FutureNurturing the Software Ecosystems of the Future
Nurturing the Software Ecosystems of the Future
 
Comment programmer un robot en 30 minutes?
Comment programmer un robot en 30 minutes?Comment programmer un robot en 30 minutes?
Comment programmer un robot en 30 minutes?
 
On the rise and fall of CI services in GitHub
On the rise and fall of CI services in GitHubOn the rise and fall of CI services in GitHub
On the rise and fall of CI services in GitHub
 
On backporting practices in package dependency networks
On backporting practices in package dependency networksOn backporting practices in package dependency networks
On backporting practices in package dependency networks
 
Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and RubygemsComparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
 
Lost in Zero Space
Lost in Zero SpaceLost in Zero Space
Lost in Zero Space
 
Evaluating a bot detection model on git commit messages
Evaluating a bot detection model on git commit messagesEvaluating a bot detection model on git commit messages
Evaluating a bot detection model on git commit messages
 
Is my software ecosystem healthy? It depends!
Is my software ecosystem healthy? It depends!Is my software ecosystem healthy? It depends!
Is my software ecosystem healthy? It depends!
 
Bot or not? Detecting bots in GitHub pull request activity based on comment s...
Bot or not? Detecting bots in GitHub pull request activity based on comment s...Bot or not? Detecting bots in GitHub pull request activity based on comment s...
Bot or not? Detecting bots in GitHub pull request activity based on comment s...
 
On the fragility of open source software packaging ecosystems
On the fragility of open source software packaging ecosystemsOn the fragility of open source software packaging ecosystems
On the fragility of open source software packaging ecosystems
 
How magic is zero? An Empirical Analysis of Initial Development Releases in S...
How magic is zero? An Empirical Analysis of Initial Development Releases in S...How magic is zero? An Empirical Analysis of Initial Development Releases in S...
How magic is zero? An Empirical Analysis of Initial Development Releases in S...
 
Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)
 
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
 
SecoHealth 2019 Research Achievements
SecoHealth 2019 Research AchievementsSecoHealth 2019 Research Achievements
SecoHealth 2019 Research Achievements
 
SECO-Assist 2019 research seminar
SECO-Assist 2019 research seminarSECO-Assist 2019 research seminar
SECO-Assist 2019 research seminar
 
Empirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package ManagersEmpirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package Managers
 

Último

➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men 🔝jhansi🔝 Escorts S...
➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men  🔝jhansi🔝   Escorts S...➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men  🔝jhansi🔝   Escorts S...
➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men 🔝jhansi🔝 Escorts S...
amitlee9823
 
怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证
怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证
怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证
eeanqy
 
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
amitlee9823
 
Peaches App development presentation deck
Peaches App development presentation deckPeaches App development presentation deck
Peaches App development presentation deck
tbatkhuu1
 
➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men 🔝dehradun🔝 Escor...
➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men  🔝dehradun🔝   Escor...➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men  🔝dehradun🔝   Escor...
➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men 🔝dehradun🔝 Escor...
amitlee9823
 
How to Build a Simple Shopify Website
How to Build a Simple Shopify WebsiteHow to Build a Simple Shopify Website
How to Build a Simple Shopify Website
mark11275
 
ab-initio-training basics and architecture
ab-initio-training basics and architectureab-initio-training basics and architecture
ab-initio-training basics and architecture
saipriyacoool
 

Último (20)

WhatsApp Chat: 📞 8617697112 Call Girl Baran is experienced
WhatsApp Chat: 📞 8617697112 Call Girl Baran is experiencedWhatsApp Chat: 📞 8617697112 Call Girl Baran is experienced
WhatsApp Chat: 📞 8617697112 Call Girl Baran is experienced
 
Call Girls Jalgaon Just Call 8617370543Top Class Call Girl Service Available
Call Girls Jalgaon Just Call 8617370543Top Class Call Girl Service AvailableCall Girls Jalgaon Just Call 8617370543Top Class Call Girl Service Available
Call Girls Jalgaon Just Call 8617370543Top Class Call Girl Service Available
 
Hingoli ❤CALL GIRL 8617370543 ❤CALL GIRLS IN Hingoli ESCORT SERVICE❤CALL GIRL
Hingoli ❤CALL GIRL 8617370543 ❤CALL GIRLS IN Hingoli ESCORT SERVICE❤CALL GIRLHingoli ❤CALL GIRL 8617370543 ❤CALL GIRLS IN Hingoli ESCORT SERVICE❤CALL GIRL
Hingoli ❤CALL GIRL 8617370543 ❤CALL GIRLS IN Hingoli ESCORT SERVICE❤CALL GIRL
 
➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men 🔝jhansi🔝 Escorts S...
➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men  🔝jhansi🔝   Escorts S...➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men  🔝jhansi🔝   Escorts S...
➥🔝 7737669865 🔝▻ jhansi Call-girls in Women Seeking Men 🔝jhansi🔝 Escorts S...
 
怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证
怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证
怎样办理伯明翰大学学院毕业证(Birmingham毕业证书)成绩单留信认证
 
💫✅jodhpur 24×7 BEST GENUINE PERSON LOW PRICE CALL GIRL SERVICE FULL SATISFACT...
💫✅jodhpur 24×7 BEST GENUINE PERSON LOW PRICE CALL GIRL SERVICE FULL SATISFACT...💫✅jodhpur 24×7 BEST GENUINE PERSON LOW PRICE CALL GIRL SERVICE FULL SATISFACT...
💫✅jodhpur 24×7 BEST GENUINE PERSON LOW PRICE CALL GIRL SERVICE FULL SATISFACT...
 
Top Rated Pune Call Girls Saswad ⟟ 6297143586 ⟟ Call Me For Genuine Sex Serv...
Top Rated  Pune Call Girls Saswad ⟟ 6297143586 ⟟ Call Me For Genuine Sex Serv...Top Rated  Pune Call Girls Saswad ⟟ 6297143586 ⟟ Call Me For Genuine Sex Serv...
Top Rated Pune Call Girls Saswad ⟟ 6297143586 ⟟ Call Me For Genuine Sex Serv...
 
Q4-W4-SCIENCE-5 power point presentation
Q4-W4-SCIENCE-5 power point presentationQ4-W4-SCIENCE-5 power point presentation
Q4-W4-SCIENCE-5 power point presentation
 
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
 
Sector 104, Noida Call girls :8448380779 Model Escorts | 100% verified
Sector 104, Noida Call girls :8448380779 Model Escorts | 100% verifiedSector 104, Noida Call girls :8448380779 Model Escorts | 100% verified
Sector 104, Noida Call girls :8448380779 Model Escorts | 100% verified
 
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️
 
Hire 💕 8617697112 Meerut Call Girls Service Call Girls Agency
Hire 💕 8617697112 Meerut Call Girls Service Call Girls AgencyHire 💕 8617697112 Meerut Call Girls Service Call Girls Agency
Hire 💕 8617697112 Meerut Call Girls Service Call Girls Agency
 
Peaches App development presentation deck
Peaches App development presentation deckPeaches App development presentation deck
Peaches App development presentation deck
 
call girls in Kaushambi (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝...
call girls in Kaushambi (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝...call girls in Kaushambi (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝...
call girls in Kaushambi (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝...
 
UI:UX Design and Empowerment Strategies for Underprivileged Transgender Indiv...
UI:UX Design and Empowerment Strategies for Underprivileged Transgender Indiv...UI:UX Design and Empowerment Strategies for Underprivileged Transgender Indiv...
UI:UX Design and Empowerment Strategies for Underprivileged Transgender Indiv...
 
The hottest UI and UX Design Trends 2024
The hottest UI and UX Design Trends 2024The hottest UI and UX Design Trends 2024
The hottest UI and UX Design Trends 2024
 
➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men 🔝dehradun🔝 Escor...
➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men  🔝dehradun🔝   Escor...➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men  🔝dehradun🔝   Escor...
➥🔝 7737669865 🔝▻ dehradun Call-girls in Women Seeking Men 🔝dehradun🔝 Escor...
 
How to Build a Simple Shopify Website
How to Build a Simple Shopify WebsiteHow to Build a Simple Shopify Website
How to Build a Simple Shopify Website
 
ab-initio-training basics and architecture
ab-initio-training basics and architectureab-initio-training basics and architecture
ab-initio-training basics and architecture
 
Pooja 9892124323, Call girls Services and Mumbai Escort Service Near Hotel Gi...
Pooja 9892124323, Call girls Services and Mumbai Escort Service Near Hotel Gi...Pooja 9892124323, Call girls Services and Mumbai Escort Service Near Hotel Gi...
Pooja 9892124323, Call girls Services and Mumbai Escort Service Near Hotel Gi...
 

Statechart modeling of interactive gesture-based applications

  • 1. Statechart modeling of interactive gesture-based applications Statechart modeling of interactive gesture-based applications Romuald Deshayes Tom Mens Software Engineering Lab 2010-2011 Deshayes Romuald – UMONS 1 / 20
  • 2. Statechart modeling of interactive gesture-based applications Table des matières 1 Problem Statement 2 Proof-of-concept application 3 Modeling interactive behaviour 4 Statechart models 5 The application framework 6 Conclusion and Future Work Deshayes Romuald – UMONS 2 / 20
  • 3. Statechart modeling of interactive gesture-based applications Problem Statement Outline : Problem Statement Problem Statement Deshayes Romuald – UMONS 3 / 20
  • 4. Statechart modeling of interactive gesture-based applications Problem Statement Problem Statement Challenge Develop intuitive interactive applications that are easy to maintain Why is it a challenge ? Complex behaviour Nondeterministic user actions Same events lead to different actions Contributions Gestural user interface for 3D objects manipulation Modeling executable behaviour using a visual formalism Validation with an application framework Deshayes Romuald – UMONS 4 / 20
  • 5. Statechart modeling of interactive gesture-based applications Proof-of-concept application Outline : Proof-of-concept application Proof-of-concept application Deshayes Romuald – UMONS 5 / 20
  • 6. Statechart modeling of interactive gesture-based applications Proof-of-concept application Proof-of-concept application 3D visual drawing tool Uses gestures to create and manipulate 3D objects OpenGL graphical library Microsoft Kinect + NITE Deshayes Romuald – UMONS 6 / 20
  • 7. Statechart modeling of interactive gesture-based applications Proof-of-concept application Small video : www.youtube.com/watch?v=lVcqzWTnpMY Deshayes Romuald – UMONS 7 / 20
  • 8. Statechart modeling of interactive gesture-based applications Modeling interactive behaviour Outline : Modeling interactive behaviour Modeling interactive behaviour Deshayes Romuald – UMONS 8 / 20
  • 9. Statechart modeling of interactive gesture-based applications Modeling interactive behaviour Modeling interactive behaviour Context Highly reactive event-driven systems Gesture-based interaction Proposed solutions Visual modeling language Statecharts Petri nets Amenable to formal analysis Easier to evolve Reduced complexity Deshayes Romuald – UMONS 9 / 20
  • 10. Statechart modeling of interactive gesture-based applications Statechart models Outline : Statechart models Statechart models Deshayes Romuald – UMONS 10 / 20
  • 11. Statechart modeling of interactive gesture-based applications Statechart models Statechart models - Hand Deshayes Romuald – UMONS 11 / 20
  • 12. Statechart modeling of interactive gesture-based applications Statechart models Statechart models - Component Deshayes Romuald – UMONS 12 / 20
  • 13. Statechart modeling of interactive gesture-based applications Statechart models Statechart models - Scene Deshayes Romuald – UMONS 13 / 20
  • 14. Statechart modeling of interactive gesture-based applications The application framework Outline : The application framework The application framework Deshayes Romuald – UMONS 14 / 20
  • 15. Statechart modeling of interactive gesture-based applications The application framework The application framework Framework for developing interactive applications Executable behaviour specified using statecharts Client-server architecture Deshayes Romuald – UMONS 15 / 20
  • 16. Statechart modeling of interactive gesture-based applications The application framework The application framework II Features Generic Abstract classes Observer design pattern Communication between statecharts Uses Java SwingStates library Deshayes Romuald – UMONS 16 / 20
  • 17. Statechart modeling of interactive gesture-based applications The application framework The application framework III <<library>> Framework SwingStates Server extends Thread -serverSocket : DatagramSocket StateMachine Controller left 0..1 1..* -machine : StateMachine Pointer <<Interface>> right -open : boolean +addComponent(c : Component) 0..1 BodyListener extends -machine : StateMachine +addPointer(p : Pointer) EventListener +addScene(s : Scene) Scene Main Application +handMovedX(e : BodyEvent) 1..* +handMovedY(e : BodyEvent) -machine : StateMachine MainScene +handMovedZ(e : BodyEvent) 1 +handClosed(e : BodyEvent) +handOpened(e : BodyEvent) selectedModel 1..* * 0..1 Component Model BodyAdapter 3DModel -boundingBox : Rectangle -selected : boolean +addAsListenerOf(s : Server) -machine : StateMachine 1..* models BodyAdapters OpenGLWidgets Hand GLButton ExtendableMenu -machine : StateMachine +changeActiveTexture() : void +extend() : void +drawButton(gl : GL) : void +retract() : void +startCounter() : void +drawMenu(gl : GL) : void SpawningMenu +addButton(button : GLButton) : void ImageMenu +addText(text : String) : void ButtonMenu +drawMenu(gl : GL) : void +drawMenu(gl : GL) : void +addButton(button : GLBUtton) : void +onModel() : void +spawn() : void +drawMenu(gl : GL) : void +reduce() : void Deshayes Romuald – UMONS 17 / 20
  • 18. Statechart modeling of interactive gesture-based applications Conclusion and Future Work Outline : Conclusion and Future Work Conclusion and Future Work Deshayes Romuald – UMONS 18 / 20
  • 19. Statechart modeling of interactive gesture-based applications Conclusion and Future Work Conclusion and Future Work Statecharts Appropriate and scalable formalism for modeling the interactive gesture-based behaviour ? Improvements Externalise/decouple statechart representation from framework Compare statecharts with Petri nets Deshayes Romuald – UMONS 19 / 20
  • 20. Statechart modeling of interactive gesture-based applications Conclusion and Future Work Future Work II Apply our approach to other HCI applications Other types of natural interfaces Compare/combine with other UIML Controlled experiments, carry out user studies, empirical studies Deshayes Romuald – UMONS 20 / 20