SlideShare una empresa de Scribd logo
1 de 31
Descargar para leer sin conexión
Neuromechanics
Antti Oulasvirta, Sunjun Kim, and Byungjoo Lee bit.do/neuromechanics
of a Button Press
Related papers at CHI 2018:
1. Control-theoretic Models of Pointing Tue 9-11.30 517C
2. Impact Activation Improves Rapid Key Pressing Mon 16.30-17.50 514AB
3. Moving Target Selection: A Cue Integration Model This session
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics10x slowdown
Long-term goal:
A theory of input
Design
Feedback
Goals
Anatomy
Skill
Movement
Efficiency
Efficacy
Effort
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Existing theories
Information theory
Human performance
Control theory
Cognitive models
Characterize
Intrapolate
Poor transfer
Change in device or task
insists on data collection
or manual task modeling
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
The brain’s point-of-view
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Design
Anatomy
Goals
Feedback
Kinematics
Dynamics
Precision
Effort
Human-like
responses
Adapt
A generative
approach
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
The
physical
The
neural
What is neuromechanics?
Neural principles of motor control in biomechanical systems
The
physiological
[Enoka 2009]
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Input is control of mediated sensations
The
brain
Peripheral
nervous system
Sensory
nervous system
Action
potential
Physical
stimulation
Feedforward
Feedback
Limbs
Sensory organs
Device
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Probabilistic
internal model
Sensory
signals
Activation
signal
Cue
integration
Integrated
percept
Perceptual
control task
Prediction
error
Prediction
Muscle, bone,
tissue, device,
sensory organs
Overview of the theory
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Theory: Elements
Biomechanical simulation
Cue integration
Probabilistic internal model
Perceptual control
[Enoka 2009]
[Sung-Hee Lee 2009]
[Ernst 2004, 2006]
[Powers 1973, 2009]
[Clark 2013]
[Hohwy 2016]
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Too brief for closed-loop control
Button mechanism not perceived
Ephemeral sensations
Noisy neuromuscular system
10x slowdown
Precise time and force
Effective skill transfer
Ability to adapt and recover
Button-
pressing:
A miracle
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Random
movements are
unsuccessful
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Biomechanics
simulation
Noisy muscle
activation signal
Mechanoreceptive
sensor
Proprioceptive
sensor
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Silicone
finger tip
Pressure
sensor
Hill type
muscle
Joint angle
sensor
Real button
Robotic
implementation
+ Noise
+ Noise+ Noise
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
No motor noise
à super-human performance
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Sensory signals
Cue integration
Perceptual center
eeds a threshold value. This
ls: visual and auditory (beep).
o-beep delays are assumed to
utation of p-Centers
m is connected to four extero-
ion, proprioception, audition,
ity i produces a p-center pci.
sfer of a neural signal evoked
hanoreceptors. We are espe-
noreceptors on the finger pad
ming of a button press. Slowly
ensitive to coarse spatial struc-
flat top surface of the button),
ers respond to motion. Kim
in signals from the fingertip
ion, and jerk from the finger
t force and indentation have
d force correlates highly with
use maximum likelihood estimation (MLE) to obtain
estimate of pco. For another implementation of cue inte
see [35]. In MLE, assuming that a single-cue estim
unbiased but corrupted by Gaussian noise, the optimal s
for estimating pco is a weighted average [16, 17]:
pco = Â
i
wi pci where wi =
1/s2
i
Âi 1/s2
i
with wi being the weight given to the ith single-cue es
and s2
i being that estimate’s variance. Figure 6 sho
emplary p-center calculations: signal-specific (pci) an
grated p-centers (pco) from 100 simulated runs of NEU
CHANIC pressing a tactile button. Note that absolute
ences among pci do not affect pco, only signal varian
The integrated timing estimate is robust to long delays
auditory or visual feedback. This assumption is base
study showing that physiological events that take place q
within a few hundred milliseconds, do not tend to be
over- nor underestimations of event durations [14].
Maximum likelihood estimator
“When was the
button activated?”
Modality-specific
noise variances
Proprio-
ceptive
Tactile
Integration is
sensitive to how
reliable the cues are
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
No mechanoreceptive
feedback signal
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Perceptual control task
Choose (1) a motor command and (2) expected perception that
minimizes this objective:
The
oned
ntin-
prio-
ion),
Gaus-
form
error
tiva-
cuss
esult
ptual
enter
com-
f cue
(GP)
and
e the
with signal offset µ, signal amplitude t, and duration s of the
agonist (A+) muscle. We have set physiologically plausible
extrema (min and max) for the activation parameters. Note
that this formulation assumes that the antagonist muscle re-
sists motion passively. More determinate pull-up motion can
be achieved by adding similar parameters for the antagonist
muscle (A-).
The objective is to determine motor command (q) and as-
sociated estimate of perceived button activation (pce) that
minimize error:
min
q,pce
EP(q, pce)+EA(q)+EC(q)+wFM(q) (2)
where EP is perceptual error, EA is error in activating the
button, and EC is error in making contact (button cap not
touched). FM is muscle force expenditure computed from the
Hill muscle model (see below), and w is a tuning factor. We
assume that activation and contact errors are trivial to perceive.
Therefore, EA and EC are binary: 1 in the case of error and 0
otherwise. Perceptual error EP is defined as distance (in time)
between expected p-center pce and observed p-center pco:
Objective function
Perceptual
error
Activation
failure
(binary)
Contact
failure
(binary)
Muscle
force
User goal:
Lightness
vs.
precision
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Perceptual error
Minimize error between expected and perceived activation time
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
No button-
activation term
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
No muscle
effort term
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
A probabilistic internal model
learns motor outputs that minimize the perceptual objective
A Bayesian optimizer with a Gaussian Process prior
Approximate Bayesian Computation (ABC)Approximate Bayesian Computation (ABC) Approximate Bayesian Computat
...
Finds a good button
press after 10-20 trials
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Model
parameters
Table 1. Model parameters. Button parameters here given for physical
buttons. Task parameters (e.g., finger starting height) are given in text.
f denotes function
Variable Description Value, Unit Ref.
fr Radius of finger cone 7.0 mm
fw Length of finger 60 mm
rf Density of finger 985 kg/m3
cf Damping of finger pulp 1.5 N·s/m [64]
kf Stiffness of finger pulp f, N/m [65]
wb Width of key cap 14 mm
db Depth of key cap 10 mm
rb Density of key cap 700 kg/m3
cb Damping of button 0.1 N·s/m
ks Elasticity of muscle 0.8·PCSA [38]
kd Elasticity of muscle 0.1·ks [38]
kc Damping of muscle 6 N·s/m [38]
PCSA Phys. cross-sectional area 4 cm2
L0ag, L0an Initial muscle length 300 mm
sn Neuromuscular noise 5·10 2
sm Mechanoreception noise 1·10 8
sp Proprioception noise 8·10 7
sa Sound and audition noise 5·10 4
sv Display and vision noise 2·10 2
system. We have set these parameters manually in order to
reproduce certain basic effects: Neuromuscular noise, which
reflects the joint additive contribution of neural and muscular
Physically
measurable
Tuned based on
literature
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Simulation workbench (MATLAB)
bit.do/neuromechanics
But are the
outputs
realistic?
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Precision & success: Predictions
Figure 7. Data collection on press kinematics: A single-subject study.
High-fidelity optical motion tracking was used to track a marker on
the finger nail. A custom-made single-button setup was created using
switches and key caps from commercial keyboards.
SIMULATIONS: COMPARING BUTTON DESIGNS
Most precise Less precise Least precise
High success High success Low success
Push-
button
Touch Mid-air
Order supported by literature
Data collection: A single-subject study
“Press rhythmically in a
manner natural for you”
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Mid-air button: Kinematics
[Torre & Balasubramaniam 2009]
Human Model
Similar to our data and literature
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Dynamics
Force-displacement curves
Predicted
peak forces
1.5-2.9N
Force ranges similar
to literature
cle force–displacement behavior for a tactile
with an effort-minimizing term in the objective
task performance (performance in button activation). We co
clude, that although much work remains to be done, the resul
support the ’optimal black box’ assumption. And many mor
analyses could done, such as looking at the effect of unreliab
feedback, oscillation of the finger tip, such as when walkin
or the effects that impairments like essential tremor have.
FUTURE WORK
Modeling latent neural and cognitive constructs, such as nois
poses a scientific challenge for future research. Change i
noise parameters has a large and poorly understood effect o
dynamics downstream. However, without noise, a button ca
be activated with arbitrary precision. For example, cuttin
sensory noise parameters to 10 9 reduces perceptual error t
the order of 1.5·10 6 s. Our noise model was tuned manuall
to reproduce some standard findings on sensory modalities. T
“Light touch”
Force(N)
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Touch buttons: Kinematics
Displacement-velocity curves
e 8. Displacement–velocity curves for four button types from single-subject recordings (top) and simulations (bottom
Shape similar except in release
Human Model
Similar result for push-buttons
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Signal-dependent
noise
Muscle and
joint models
Force
perception
Noise parameter
identification
Limitations
Neuromechanics of a Button Press
Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics
Neuro-
science PhysicsPhysiology
Machine
learning
Biomech.
simulation
EE and signal
processing
A unifying account
A generative simulation
Can it be made to work beyond buttons?
bit.do/neuromechanics
Antti Oulasvirta, Sunjun Kim, Byungjoo Lee
Acknowledgements: Jong-In Lee, Aleksi
Pesonen, Yunfei Xiu, and Crista Kaukinen
Related papers at CHI 2018:
1. Control-theoretic models of Pointing Tue 9.00 517C
2. Impact Activation Improves Rapid Key Pressing Mon 16.30 514AB
3. Moving Target Selection: A Cue Integration Model This session
Figure 8. Displacement–velocity curves for four button types from single-subject recordings (top) and simulations (bottom).
Data Matlab codeRobot

Más contenido relacionado

Similar a Neuromechanics of a Button Press: A talk at CHI 2018, April 2018

IRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android PhonesIRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android PhonesIRJET Journal
 
45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-report45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-reportkapilpanwariet
 
IRJET -Stress Detection using Arduino
IRJET -Stress Detection using ArduinoIRJET -Stress Detection using Arduino
IRJET -Stress Detection using ArduinoIRJET Journal
 
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...IJECEIAES
 
Non-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal ProcessingNon-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal ProcessingAtul Kumar Sharma
 
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...IRJET Journal
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacingKirtan Shah
 
Full Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface ParadigmFull Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface ParadigmTakumi Kodama
 
Biorobotics Abstract
Biorobotics AbstractBiorobotics Abstract
Biorobotics AbstractAbida Zama
 
IRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEGIRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEGIRJET Journal
 
Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...IJECEIAES
 
Spinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal CordSpinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal CordIRJET Journal
 
Leap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic LimbLeap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic LimbPrasanna Venkataramanan
 
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...Takumi Kodama
 
IRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG SignalsIRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG SignalsIRJET Journal
 
A method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEGA method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEGIRJET Journal
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...IJRTEMJOURNAL
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...journal ijrtem
 
IRJET- Survey on EEG Based Brainwave Controlled Home Automation
IRJET-  	  Survey on EEG Based Brainwave Controlled Home AutomationIRJET-  	  Survey on EEG Based Brainwave Controlled Home Automation
IRJET- Survey on EEG Based Brainwave Controlled Home AutomationIRJET Journal
 

Similar a Neuromechanics of a Button Press: A talk at CHI 2018, April 2018 (20)

IRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android PhonesIRJET- Wireless Healthcare Monitoring using Android Phones
IRJET- Wireless Healthcare Monitoring using Android Phones
 
45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-report45891026 brain-computer-interface-seminar-report
45891026 brain-computer-interface-seminar-report
 
IRJET -Stress Detection using Arduino
IRJET -Stress Detection using ArduinoIRJET -Stress Detection using Arduino
IRJET -Stress Detection using Arduino
 
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...Improved feature exctraction process to detect seizure  using CHBMIT-dataset ...
Improved feature exctraction process to detect seizure using CHBMIT-dataset ...
 
Non-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal ProcessingNon-Contact Health Monitoring System Using Image and Signal Processing
Non-Contact Health Monitoring System Using Image and Signal Processing
 
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...IRJET-  	  Automation of a Prosthetic Limb using Shared Control of Brain Mach...
IRJET- Automation of a Prosthetic Limb using Shared Control of Brain Mach...
 
Pain in the neck - Bernard Martin
Pain in the neck - Bernard MartinPain in the neck - Bernard Martin
Pain in the neck - Bernard Martin
 
Neural interfacing
Neural interfacingNeural interfacing
Neural interfacing
 
Full Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface ParadigmFull Body Spatial Vibrotactile Brain Computer Interface Paradigm
Full Body Spatial Vibrotactile Brain Computer Interface Paradigm
 
Biorobotics Abstract
Biorobotics AbstractBiorobotics Abstract
Biorobotics Abstract
 
IRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEGIRJET-Estimation of Meditation Effect on Attention Level using EEG
IRJET-Estimation of Meditation Effect on Attention Level using EEG
 
Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...Recognition of new gestures using myo armband for myoelectric prosthetic appl...
Recognition of new gestures using myo armband for myoelectric prosthetic appl...
 
Spinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal CordSpinesense: Advanced Pain Detection System for Spinal Cord
Spinesense: Advanced Pain Detection System for Spinal Cord
 
Leap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic LimbLeap - Learning Adaptive Prosthetic Limb
Leap - Learning Adaptive Prosthetic Limb
 
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
Tactile Brain-Computer Interface Using Classification of P300 Responses Evoke...
 
IRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG SignalsIRJET- Emotion Analysis for Personality Inference from EEG Signals
IRJET- Emotion Analysis for Personality Inference from EEG Signals
 
A method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEGA method for detection and reduction of stress using EEG
A method for detection and reduction of stress using EEG
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
 
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
ApplyingElectroencephalography inEstablishing SafetyCriterion byMeasuring Men...
 
IRJET- Survey on EEG Based Brainwave Controlled Home Automation
IRJET-  	  Survey on EEG Based Brainwave Controlled Home AutomationIRJET-  	  Survey on EEG Based Brainwave Controlled Home Automation
IRJET- Survey on EEG Based Brainwave Controlled Home Automation
 

Más de Aalto University

"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paper"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paperAalto University
 
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...Aalto University
 
"Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI..."Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI...Aalto University
 
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...Aalto University
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Aalto University
 
Computational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti OulasvirtaComputational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti OulasvirtaAalto University
 
HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides] HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides] Aalto University
 
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...Aalto University
 
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...Aalto University
 
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...Aalto University
 
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...Aalto University
 
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...Aalto University
 
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...Aalto University
 
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...Aalto University
 
Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)Aalto University
 
Improving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen DevicesImproving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen DevicesAalto University
 
Studying interaction with 3D mobile maps
Studying interaction with 3D mobile mapsStudying interaction with 3D mobile maps
Studying interaction with 3D mobile mapsAalto University
 

Más de Aalto University (17)

"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paper"Computational rationality as a theory of interaction" - CHI'22 paper
"Computational rationality as a theory of interaction" - CHI'22 paper
 
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
Observations on typing from 136 million keystrokes - Presentation by Antti Ou...
 
"Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI..."Computational Support for Functionality Selection in Interaction Design" CHI...
"Computational Support for Functionality Selection in Interaction Design" CHI...
 
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
User Interfaces that Design Themselves: Talk given at Data-Driven Design Day ...
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"
 
Computational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti OulasvirtaComputational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
Computational Rationality I - a Lecture at Aalto University by Antti Oulasvirta
 
HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides] HCI Research as Problem-Solving [CHI'16, presentation slides]
HCI Research as Problem-Solving [CHI'16, presentation slides]
 
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
Can Computers Design? Presented at interaction16, March 2, 2016, Helsinki by ...
 
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
Model-Based User Interface Optimization: Part V: DISCUSSION - At SICSA Summer...
 
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
Model-Based User Interface Optimization: Part IV: ADVANCED TOPICS - At SICSA ...
 
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At...
 
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
Model-Based User Interface Optimization: Part II: LETTER ASSIGNMENT - At SICS...
 
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
Model-Based User Interface Optimization: Part I INTRODUCTION - At SICSA Summe...
 
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
CHI 2014 talk by Antti Oulasvirta: Automated Nonlinear Regression Modeling fo...
 
Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)Information Capacity of Full-body Movements (CHI'13)
Information Capacity of Full-body Movements (CHI'13)
 
Improving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen DevicesImproving Two-Thumb Text Entry on Touchscreen Devices
Improving Two-Thumb Text Entry on Touchscreen Devices
 
Studying interaction with 3D mobile maps
Studying interaction with 3D mobile mapsStudying interaction with 3D mobile maps
Studying interaction with 3D mobile maps
 

Último

layers of the earths atmosphere.ppt slides for grade 9
layers of the earths atmosphere.ppt slides for grade 9layers of the earths atmosphere.ppt slides for grade 9
layers of the earths atmosphere.ppt slides for grade 9rolanaribato30
 
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...dkNET
 
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest ManagementPests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest ManagementPirithiRaju
 
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITYRHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITYDnyandaBopche
 
Introduction to Green chemistry ppt.pptx
Introduction to Green chemistry ppt.pptxIntroduction to Green chemistry ppt.pptx
Introduction to Green chemistry ppt.pptxMuskan219429
 
Skin: Structure and function of the skin
Skin: Structure and function of the skinSkin: Structure and function of the skin
Skin: Structure and function of the skinheenarahangdale01
 
Zoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptxZoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptx2019n04898
 
The GIS Capability Maturity Model (2013)
The GIS Capability Maturity Model (2013)The GIS Capability Maturity Model (2013)
The GIS Capability Maturity Model (2013)GregBabinski
 
Advances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative BiolabsAdvances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative BiolabsCreative-Biolabs
 
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...Sharon Liu
 
1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...
1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...
1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...klada0003
 
Solid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptxSolid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptxkrishuchavda31032003
 
Development of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in DrivingDevelopment of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in Drivingstudiotelon
 
INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...
INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...
INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...Ajay kamboj
 
Advance pharmacology presentation..............
Advance pharmacology presentation..............Advance pharmacology presentation..............
Advance pharmacology presentation..............SIMRAN VERMA
 
structure of proteins and its type I PPT
structure of proteins and its type I PPTstructure of proteins and its type I PPT
structure of proteins and its type I PPTvishalbhati28
 
Science9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptxScience9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptxteleganne21
 
Basics Of Computers | The Computer System
Basics Of Computers | The Computer SystemBasics Of Computers | The Computer System
Basics Of Computers | The Computer SystemNehaRohtagi1
 
Project report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docxProject report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docxpriyanshimanchanda4
 

Último (20)

layers of the earths atmosphere.ppt slides for grade 9
layers of the earths atmosphere.ppt slides for grade 9layers of the earths atmosphere.ppt slides for grade 9
layers of the earths atmosphere.ppt slides for grade 9
 
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
dkNET Webinar "The Multi-Omic Response to Exercise Training Across Rat Tissue...
 
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest ManagementPests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
Pests of Maize_Dr.UPR_Identification, Binomics, Integrated Pest Management
 
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITYRHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
RHEOLOGY MODIFIERS: ENHANCING PERFORMANCE AND FUNCTIONALITY
 
Introduction to Green chemistry ppt.pptx
Introduction to Green chemistry ppt.pptxIntroduction to Green chemistry ppt.pptx
Introduction to Green chemistry ppt.pptx
 
Skin: Structure and function of the skin
Skin: Structure and function of the skinSkin: Structure and function of the skin
Skin: Structure and function of the skin
 
Zoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptxZoogeographical regions In the World.pptx
Zoogeographical regions In the World.pptx
 
The GIS Capability Maturity Model (2013)
The GIS Capability Maturity Model (2013)The GIS Capability Maturity Model (2013)
The GIS Capability Maturity Model (2013)
 
Proof-of-Concept Publicly Accessible Data Dashboards from the US-EPA.pptx
Proof-of-Concept Publicly Accessible Data Dashboards from the US-EPA.pptxProof-of-Concept Publicly Accessible Data Dashboards from the US-EPA.pptx
Proof-of-Concept Publicly Accessible Data Dashboards from the US-EPA.pptx
 
Advances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative BiolabsAdvances in Oncolytic Virotherapy - Creative Biolabs
Advances in Oncolytic Virotherapy - Creative Biolabs
 
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
20240315 ACMJ Diagrams Set 2.docx . With light, motor, coloured light, and se...
 
1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...
1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...
1David Andress - The Oxford Handbook of the French Revolution-Oxford Universi...
 
Solid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptxSolid waste management_13_409_U1_2024.pptx
Solid waste management_13_409_U1_2024.pptx
 
Development of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in DrivingDevelopment of a Questionnaire for Identifying Personal Values in Driving
Development of a Questionnaire for Identifying Personal Values in Driving
 
INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...
INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...
INFLUENCE OF PREHARVEST PRACTICES, ENZYMATIC AND TEXTURAL CHANGES, RESPIRATIO...
 
Advance pharmacology presentation..............
Advance pharmacology presentation..............Advance pharmacology presentation..............
Advance pharmacology presentation..............
 
structure of proteins and its type I PPT
structure of proteins and its type I PPTstructure of proteins and its type I PPT
structure of proteins and its type I PPT
 
Science9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptxScience9 Quarter 3:Latitude and altitude.pptx
Science9 Quarter 3:Latitude and altitude.pptx
 
Basics Of Computers | The Computer System
Basics Of Computers | The Computer SystemBasics Of Computers | The Computer System
Basics Of Computers | The Computer System
 
Project report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docxProject report on Fasciola hepatica.docx
Project report on Fasciola hepatica.docx
 

Neuromechanics of a Button Press: A talk at CHI 2018, April 2018

  • 1. Neuromechanics Antti Oulasvirta, Sunjun Kim, and Byungjoo Lee bit.do/neuromechanics of a Button Press Related papers at CHI 2018: 1. Control-theoretic Models of Pointing Tue 9-11.30 517C 2. Impact Activation Improves Rapid Key Pressing Mon 16.30-17.50 514AB 3. Moving Target Selection: A Cue Integration Model This session
  • 2. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics10x slowdown Long-term goal: A theory of input Design Feedback Goals Anatomy Skill Movement Efficiency Efficacy Effort
  • 3. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Existing theories Information theory Human performance Control theory Cognitive models Characterize Intrapolate Poor transfer Change in device or task insists on data collection or manual task modeling
  • 4. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics The brain’s point-of-view
  • 5. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Design Anatomy Goals Feedback Kinematics Dynamics Precision Effort Human-like responses Adapt A generative approach
  • 6. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics The physical The neural What is neuromechanics? Neural principles of motor control in biomechanical systems The physiological [Enoka 2009]
  • 7. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Input is control of mediated sensations The brain Peripheral nervous system Sensory nervous system Action potential Physical stimulation Feedforward Feedback Limbs Sensory organs Device
  • 8. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Probabilistic internal model Sensory signals Activation signal Cue integration Integrated percept Perceptual control task Prediction error Prediction Muscle, bone, tissue, device, sensory organs Overview of the theory
  • 9. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Theory: Elements Biomechanical simulation Cue integration Probabilistic internal model Perceptual control [Enoka 2009] [Sung-Hee Lee 2009] [Ernst 2004, 2006] [Powers 1973, 2009] [Clark 2013] [Hohwy 2016]
  • 10. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Too brief for closed-loop control Button mechanism not perceived Ephemeral sensations Noisy neuromuscular system 10x slowdown Precise time and force Effective skill transfer Ability to adapt and recover Button- pressing: A miracle
  • 11. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Random movements are unsuccessful
  • 12. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Biomechanics simulation Noisy muscle activation signal Mechanoreceptive sensor Proprioceptive sensor
  • 13. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Silicone finger tip Pressure sensor Hill type muscle Joint angle sensor Real button Robotic implementation + Noise + Noise+ Noise
  • 14. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No motor noise à super-human performance
  • 15. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Sensory signals Cue integration Perceptual center eeds a threshold value. This ls: visual and auditory (beep). o-beep delays are assumed to utation of p-Centers m is connected to four extero- ion, proprioception, audition, ity i produces a p-center pci. sfer of a neural signal evoked hanoreceptors. We are espe- noreceptors on the finger pad ming of a button press. Slowly ensitive to coarse spatial struc- flat top surface of the button), ers respond to motion. Kim in signals from the fingertip ion, and jerk from the finger t force and indentation have d force correlates highly with use maximum likelihood estimation (MLE) to obtain estimate of pco. For another implementation of cue inte see [35]. In MLE, assuming that a single-cue estim unbiased but corrupted by Gaussian noise, the optimal s for estimating pco is a weighted average [16, 17]: pco = Â i wi pci where wi = 1/s2 i Âi 1/s2 i with wi being the weight given to the ith single-cue es and s2 i being that estimate’s variance. Figure 6 sho emplary p-center calculations: signal-specific (pci) an grated p-centers (pco) from 100 simulated runs of NEU CHANIC pressing a tactile button. Note that absolute ences among pci do not affect pco, only signal varian The integrated timing estimate is robust to long delays auditory or visual feedback. This assumption is base study showing that physiological events that take place q within a few hundred milliseconds, do not tend to be over- nor underestimations of event durations [14]. Maximum likelihood estimator “When was the button activated?” Modality-specific noise variances Proprio- ceptive Tactile Integration is sensitive to how reliable the cues are
  • 16. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No mechanoreceptive feedback signal
  • 17. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Perceptual control task Choose (1) a motor command and (2) expected perception that minimizes this objective: The oned ntin- prio- ion), Gaus- form error tiva- cuss esult ptual enter com- f cue (GP) and e the with signal offset µ, signal amplitude t, and duration s of the agonist (A+) muscle. We have set physiologically plausible extrema (min and max) for the activation parameters. Note that this formulation assumes that the antagonist muscle re- sists motion passively. More determinate pull-up motion can be achieved by adding similar parameters for the antagonist muscle (A-). The objective is to determine motor command (q) and as- sociated estimate of perceived button activation (pce) that minimize error: min q,pce EP(q, pce)+EA(q)+EC(q)+wFM(q) (2) where EP is perceptual error, EA is error in activating the button, and EC is error in making contact (button cap not touched). FM is muscle force expenditure computed from the Hill muscle model (see below), and w is a tuning factor. We assume that activation and contact errors are trivial to perceive. Therefore, EA and EC are binary: 1 in the case of error and 0 otherwise. Perceptual error EP is defined as distance (in time) between expected p-center pce and observed p-center pco: Objective function Perceptual error Activation failure (binary) Contact failure (binary) Muscle force User goal: Lightness vs. precision
  • 18. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Perceptual error Minimize error between expected and perceived activation time
  • 19. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No button- activation term
  • 20. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics No muscle effort term
  • 21. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics A probabilistic internal model learns motor outputs that minimize the perceptual objective A Bayesian optimizer with a Gaussian Process prior Approximate Bayesian Computation (ABC)Approximate Bayesian Computation (ABC) Approximate Bayesian Computat ... Finds a good button press after 10-20 trials
  • 22. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Model parameters Table 1. Model parameters. Button parameters here given for physical buttons. Task parameters (e.g., finger starting height) are given in text. f denotes function Variable Description Value, Unit Ref. fr Radius of finger cone 7.0 mm fw Length of finger 60 mm rf Density of finger 985 kg/m3 cf Damping of finger pulp 1.5 N·s/m [64] kf Stiffness of finger pulp f, N/m [65] wb Width of key cap 14 mm db Depth of key cap 10 mm rb Density of key cap 700 kg/m3 cb Damping of button 0.1 N·s/m ks Elasticity of muscle 0.8·PCSA [38] kd Elasticity of muscle 0.1·ks [38] kc Damping of muscle 6 N·s/m [38] PCSA Phys. cross-sectional area 4 cm2 L0ag, L0an Initial muscle length 300 mm sn Neuromuscular noise 5·10 2 sm Mechanoreception noise 1·10 8 sp Proprioception noise 8·10 7 sa Sound and audition noise 5·10 4 sv Display and vision noise 2·10 2 system. We have set these parameters manually in order to reproduce certain basic effects: Neuromuscular noise, which reflects the joint additive contribution of neural and muscular Physically measurable Tuned based on literature
  • 23. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Simulation workbench (MATLAB) bit.do/neuromechanics But are the outputs realistic?
  • 24. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Precision & success: Predictions Figure 7. Data collection on press kinematics: A single-subject study. High-fidelity optical motion tracking was used to track a marker on the finger nail. A custom-made single-button setup was created using switches and key caps from commercial keyboards. SIMULATIONS: COMPARING BUTTON DESIGNS Most precise Less precise Least precise High success High success Low success Push- button Touch Mid-air Order supported by literature
  • 25. Data collection: A single-subject study “Press rhythmically in a manner natural for you”
  • 26. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Mid-air button: Kinematics [Torre & Balasubramaniam 2009] Human Model Similar to our data and literature
  • 27. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Dynamics Force-displacement curves Predicted peak forces 1.5-2.9N Force ranges similar to literature cle force–displacement behavior for a tactile with an effort-minimizing term in the objective task performance (performance in button activation). We co clude, that although much work remains to be done, the resul support the ’optimal black box’ assumption. And many mor analyses could done, such as looking at the effect of unreliab feedback, oscillation of the finger tip, such as when walkin or the effects that impairments like essential tremor have. FUTURE WORK Modeling latent neural and cognitive constructs, such as nois poses a scientific challenge for future research. Change i noise parameters has a large and poorly understood effect o dynamics downstream. However, without noise, a button ca be activated with arbitrary precision. For example, cuttin sensory noise parameters to 10 9 reduces perceptual error t the order of 1.5·10 6 s. Our noise model was tuned manuall to reproduce some standard findings on sensory modalities. T “Light touch” Force(N)
  • 28. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Touch buttons: Kinematics Displacement-velocity curves e 8. Displacement–velocity curves for four button types from single-subject recordings (top) and simulations (bottom Shape similar except in release Human Model Similar result for push-buttons
  • 29. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Signal-dependent noise Muscle and joint models Force perception Noise parameter identification Limitations
  • 30. Neuromechanics of a Button Press Oulasvirta, Kim, & Lee Proc. CHI 2018 bit.do/neuromechanics Neuro- science PhysicsPhysiology Machine learning Biomech. simulation EE and signal processing A unifying account A generative simulation Can it be made to work beyond buttons?
  • 31. bit.do/neuromechanics Antti Oulasvirta, Sunjun Kim, Byungjoo Lee Acknowledgements: Jong-In Lee, Aleksi Pesonen, Yunfei Xiu, and Crista Kaukinen Related papers at CHI 2018: 1. Control-theoretic models of Pointing Tue 9.00 517C 2. Impact Activation Improves Rapid Key Pressing Mon 16.30 514AB 3. Moving Target Selection: A Cue Integration Model This session Figure 8. Displacement–velocity curves for four button types from single-subject recordings (top) and simulations (bottom). Data Matlab codeRobot