This is AR/VR presentation. Also, I'd like you to visit https://sigmally.com/ and play

1. October 1, 2018
October 1, 2018
Mark F. Bocko | Professor
Department: Electrical and Computer Engineering
Focus: Spatial audio
Pilot Project: “Development of a quantitative framework for spatial audio characterization”
Project Goals
• Develop quantitative methods to assess spatial audio rendering systems
• Incorporate quantitative binaural hearing models into audio system design tools
• Predict what listeners will report hearing (locations, spatial extent of sources, diffusiveness)

2. October 2018
Geunyoung Yoon | Professor
Department: Ophthalmology, The Institute of Optics, Center for Visual Science, Biomedical Engineering
Focus: Physiological Optics, Vision Correction, Visual Psychophysics, Optical Imaging, Biomechanics, Eye Diseases
Lab website: http://www.cvs.rochester.edu/yoonlab/
OCULAR OPTICS & CUSTOMIZED
VISION CORRECTION
• Eye’s aberration and visual quality
• Ocular wavefront sensing
• Advanced ophthalmic lenses
• Sport vision
• Optical metrology
OCULAR OPTICS and VISION
• Adaptive optics vision simulator
• Adaptation to habitual optics
• Neural processing and perception
• Binocular integration
• Neural plasticity
• Stereopsis
ANTERIOR SEGMENT IMAGING
• Mechanisms of pathologic cornea
diseases
• Ocular surface diseases and dry eye
• Corneal biomechanics
• Multimodal high-resolution ocular
imaging
• Advanced cataract surgery
RESEARCH TOPICS:
ACOOMMODATION & PRESBYOPIA
• Vergence-Accommodation conflict
under VR/AR environments
• Extended depth of focus technology
• Accommodating intraocular lens
• Peripheral vision and optics
• Binocular accommodation
• Emmetropization / Refractive error

3. October 2018
Zhiyao Duan | Assistant Professor
Department: Electrical and Computer Engineering
Focus: Computer Audition, Music Information Retrieval, Audio-Visual Analysis, Audio for AR/VR
Lab: Audio Information Research (AIR) Lab
RESEARCH TOPICS:
MUSIC INFORMATION RETRIEVAL
• Music transcription
• Audio-score alignment
• Music source separation
• Music generation
AUDIO-VISUAL PROCESSING
• AV analysis of music performances
• Visually informed source separation
• Music performance generation
• Talking face generation from speech
ENVIRONMENTAL SOUND
UNDERSTANDING
• Sound search by vocal imitation
• Sound event detection
• Source localization and tracking
SPEECH PROCESSING
• Speech emotion classification
• Speaker recognition and diarization
• Speech enhancement

4. October 2018
Andrew White| Assistant Professor
Department: Chemical Engineering
Focus: Role of AR/VR in higher education, computational chemistry
Courses: CHE 116 – Numerical Methods & Statistics | CHE 477 – Advanced Numerical Methods
Role of AR and VR in Higher Education
• Collaborative project exploring the use of
AR and VR in STEM curriculum
• Emphasis on tactile, collaborative,
interactive replacement for traditional
hands-on labs
• Intended for topics that are abstract or
impossible to have labs, e.g. quantum
mechanics or solving ODEs
• Provides a new tool for outreach to
generate enthusiasm for STEM careers
RESEARCH TOPICS:
Computational Chemistry
• Computer simulation of dynamics of
molecules at the level of atoms
• Provides insight at a length-scale inaccessible
to experiments
• Requires careful design of scientifically
accurate, highly-parallel, algorithms and code
• Use techniques like multi-scale modeling to
study complex phenomena like protein
adsorption
• High accuracy through novel methods to
incorporate data derived from experiments in
simulations

5. October 2018
Yuhao Zhu | Assistant Professor
Department: Computer Science, Goergen Institute for Data Science
Focus: Architecting next-generation computer hardware for an AR/VR-driven future!
RESEARCH TOPICS: Co-Design Computing Systems with Non-computing Systems for AR/VR
• Optical system + image sensor + imaging + computer vision (w/ Jannick Rolland)
• Does the optimal design of optical system change with the specific vision task?
• Does the optimal design of vision hardware change with optical systems?
• How to design an end-to-end system with specific tasks and quality metrics in mind?
• How to dynamically reconfigure both optical systems and computer systems on the fly?

6. October 2018
Michele Rucci | Professor
Department: Brain & Cognitive Sciences
Focus: Action & perception, visual perception in humans and machines, human behavior, sensory
processing
COMPUTATIONAL MECHANISMS
• Computational goals of visual
processing
• Establishment of spatial
representations
• Multimodal integration
HUMAN BEHAVIOR
• Action properties
• Identification of visuomotor
strategies
• Limits of oculomotor control
• Natural head-eye coordination
VISUAL DEVELOPMENT
• Consequences of eye movements in
visual maturation
• Abnormal eye movements
• Myopia
RESEARCH TOPICS:
ACTIVE VISION
• Vision as sensorimotor integrated
process
• Dependence of visual functions on
eye movements
• Disruption of oculomotor cycle via
gaze-contingent control

7. Building Virtual Concert Halls with Spatial Audio
Ming-Lun Lee, Matthew Brown, Zhiyao Duan, and Steven Philbert
Department of Electrical and Computer Engineering
Eastman School of Music
Building Virtual Concert Halls with Spatial Audio
Ming-Lun Lee, Matthew Brown, Zhiyao Duan, and Steven Philbert
Department of Electrical and Computer Engineering
Eastman School of Music

8. zbai@cs.rochester.edu http://zhenbai.io October 2018
Zhen Bai| Assistant Professor
Department: Computer Science
Focus: Human-Computer Interaction, Augmented Reality, Tangible User Interface, Embodied Conversational Agent,
Education Technology, Computer-Supported Collaboration, Design for Diversity
RESEARCH TOPICS: Augmented Reality - Theory of Mind, Symbolic Play, Children with Autism Spectrum Condition

9. UR AR/VR Pilot Project:
Real-time synthesis of a virtual talking face from acoustic speech
• Chenliang Xu, Assistant Professor of Computer Science
• Collaborators:
• Ross Maddox, Assistant Professor of Biomedical Engineering
• Zhiyao Duan, Assistant Professor of Electrical and Computer Engineering
[Chen, Li, Maddox, Duan, and Xu, ECCV 2018]

10. October 2018
Ross Maddox | Assistant Professor
Department: Neuroscience & Biomedical Engineering
Focus: Audio-visual integration, selective attention, sound localization
AUDIO-VISUAL INTEGRATION
• Multisensory binding and object
formation
• Impact of “uninformative” visual
stimuli on auditory perception
VISUAL HEARING AID
• Generate an artificial talking face
from speech audio in real-time
• Improve listening abilities for
people with hearing impairments,
attention disorders
RESEARCH TOPICS:
GAZE EFFECTS ON SPATIAL HEARING
• Interaction between eye
movements and auditory spatial
acuity
• Benefits of directed eye gaze on
speech-in-noise understanding
BRAINSTEM CODING OF SPEECH
• Use electroencephalography and
novel signal processing schemes to
study how brainstem codes speech
• Investigate subcortical effects of
attention and cognition

11. October 2018
Martina Poletti| Assistant Professor
Department: Neuroscience
Focus: Visual perception, eye movements, attention, eyetracking.
RESEARCH TOPICS:
VISUOSPATIAL ATTENTION
• Resolution of attention in the fovea
• Attention contribution to fine
spatial vision
• Pre-microsaccadic enhancements of
foveal vision
HIGH ACUITY VISION
• Fine control of eye movements
during high acuity tasks
• Distribution of high acuity
capabilities across the fovea
• Oculomotor strategies in fine spatial
vision
SPATIAL REPRESENTATIONS
• Multimodal integration
• Spatial updating across saccades
• Spatial updating during fixation
FOVEAL PRIORITY MAPS
• Driving factors
• Perceptual benefits
• Spatiotemporal dynamics
• Visual exploration at the foveal
scale

12. October 2018
River Campus Libraries
Presenter: Lauren Di Monte, Director of Research Initiatives
Focus: Support AR/VR research, teaching, and learning
AR/VR Creation and Exploration Space
Enhance access and support Provide on-ramps Grow a community of practice

13. UR Medicine Health Lab (Hasselberg, Mitten, Dasilva)
• Expertise in technology innovation to improve delivery of care
Department of Psychiatry (Cross, Hasselberg)
• Expertise in cognitive behavior therapy, and implementation science
Eastman School of Music (Brown, Winders)
• Expertise in visual and audio therapeutic functions
Art, Science, & Engineering (Luo)
• Expertise in computer science and smartphone sensors
13
Cognitive Behavior Therapy Mobile App with
Embedded Virtual Reality

14. October 2018
Edmund Lalor | Associate Professor
Department: Biomedical Engineering and Neuroscience
Focus: Human neuroscience, sensory processing, perception, cognition
HUMAN SENSORY PROCESSING
• Hierarchical processing of natural
audio and visual stimuli
• The effects of knowledge and
prediction on early sensory
processing
MULTISENSORY INTEGRATION
• Audiovisual speech processing
• The effect of visual input on
auditory scene segregation
NEURAL SIGNAL DECODING
• Methods for decoding multivariate
neural data
• Decoding representations of
acoustic space in the cortex
• Decoding selective attention in real-
time
RESEARCH TOPICS:
ATTENTION
• Visual spatial attention
• Auditory selective attention –
particularly to speech (i.e., the
cocktail party problem)

15. October 2018
Ania Busza | Assistant Professor
Department: Neurology (Stroke division)
Focus: Stroke Rehabilitation
KEY ISSUES IN STROKE REHABILITATION AND RECOVERY
• What factors predict stroke recovery?
• What is the best timing/dose for rehabilitation therapies?
• How can we use new technologies to create more effective therapeutics?
RESEARCH TOPICS:
(1) Surface-EMG controlled
Virtual Arm
(2) Using superficial sensors to
quantify rehab “dose”
(3) EMG analysis of Motor
System fatigue during learning

16. October 2018
Michael Jarvis| Associate Professor
Director, Digital Media Studies Program
Departments: History & ATHS
RESEARCH: VR, Visualization & Analysis of Cultural Heritage Sites; VR and Public History
UR/University of Ghana
Digital Archaeology & Structural Analysis
Field Research, 2017-present:
VR applications

17. Aims 1 & 2: Self-regulation (executive functioning & autonomic regulation) à social motivation
Aim 3: Self-regulation (autonomic & behavioral reactivity/regulation) during immersive, 360 social filmà social motivation
Measuring: autonomic
reactivity/regulation & head orientation
• Custom-made
video (~6 min)
• Spatial audio
• Progressively
more directive
social overtures
made toward
participant
Self-regulation processes underlying social motivation in ASD: The influence of social context
1. Decision
Making
2. Inhibitory
Control
3. Cognitive
Flexibility
Social Motivation:
Executive Functioning Tasks (done with and without social noise):
Collecting
HRV & EDA
during tasks
Jessica M. Keith| Doctoral Candidate
Department: Clinical & Social Sciences in Psychology; Research Lab: Dev. Neuropsychology Lab (PI: Loisa Bennetto, PhD)

18. October 2018
Benjamin T. Crane| Associate Professor
Department: Otolaryngology, Biomedical Engineering and Neuroscience
Focus: Human visual-vestibular multisensory integration, motion perception, rehabilitation of vestibular
lesions
VISUAL-VESTIBULAR INTEGRATION
• Common causality between visual
and inertial stimuli
• Disambiguation of extern vs. self
motion
VISUAL-VESTIBULAR ADAPTATION
• Use of visual-inertial heading offsets
to adapt heading
• Exposure to a rotating environment
for heading adaptation
NEW METHODS OF VESTIBULAR
REHABILITATION
• Develop method for rehabilitation
of heading deviation
• Decoding representations of
acoustic space in the cortex
• Decoding selective attention in real-
time
RESEARCH TOPICS:
PATHOLOGICAL MOTION PERCEPTION
• Rotation perception
• Heading offset
• Integration with visual motion
• Effect of unilateral lesions
• Migraine
Acute Chronic
I
V
I
V

19. October 2018
Duje Tadin | Professor
Department: Brain & Cognitive Science, Center for Visual Science, Neuroscience, Ophthalmology
Co-Director, Center for Augmented and Virtual Reality (Neuroscience)
Focus: Visual perception, Multisensory processing, Brain plasticity
Conflict of interest declaration: Senior Scientist, NeuroTrainer
VISUAL PERCEPTION & COGNITION
• Motion perception
• Visual attention & awareness
• Binocular vision
• Visual adaptation
• Cognitive assessment in VR
BRAIN PLASTICITY
• Perceptual learning
• Cognitive training
• Brain training in VR
• Used in heathy adults, dementia,
stroke, corneal disease, concussion,
schizophrenia.
METHODS
• Psychophysics
• Computational modeling
• Eye tracking
• Brain stimulation
• Neuroimaging
RESEARCH TOPICS:
MULTISENSORY PROCESSING
• Audiovisual interactions
• Vision & proprioception
• Synesthesia