Development for Virtual Reality is a hunt for each millisecond of latency – it’s the one metric that needs to be perfect for an immersive and comfortable experience. In Dominic Eskofier’s talk, you’ll learn how to maximize both framerate and visual quality of your Virtual Reality app by using sophisticated rendering techniques built into popular game engines, NVIDIA’s SDKs and modern GPUs.

1. EVERY MILLISECOND COUNTS
How to render faster for Virtual Reality
AWE Europe, Berlin 2016
Dominic Eskofier
Virtual Reality
Manager EMEAI
deskofier@nvidia.com

2. Gaming
Movies
Sports
Concerts
Travel Retail
PERSONAL
ENTERTAINMENT
VR

3. NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
QUADRO PROFESSIONAL VR/AR
MEDICAL ARCHITECTURE
DESIGN MANUFACTURING COLLABORATION
DIGITAL CONTENT CREATION

4. COMPUTING CHALLENGES
IN SIMULATING REALITY
GRAPHICS
PHYSICAL
SIMULATION
AUDIO TOUCH

5. 5
TRADITIONAL = 60 MP/S
(1920 X 1080 @ 30 FPS)
VIRTUAL REALITY = 450 MP/S
(3024 X 1680* @ 90 FPS)
1920
1080
1512
1680
*VR render resolution
1512
VR PERFORMANCE DEMANDS
Ultra-High Resolution and Frame Rate

6. Motion to Photon: ≤ 20 ms
VR PERFORMANCE DEMANDS
Ultra-Low Latency

7. GRAPHICS 101
CREATE 3D
GEOMETRY
PROJECT TO
2D VIEW
CALCULATE
PIXELS

8. Warped ImageRendered Image
VR RENDERING 101

9. Rendered Image Warped Image
VR RENDERING 101
GPU Renders Many Pixels that Never Make It to the Screen

10. NVIDIA Confidential – Do Not Distribute
NVIDIA MULTI-RES SHADING
Introduced in Maxwell

11. LENS MATCHED SHADING
Renders to a Lens Corrected Surface

12. LENS MATCHED SHADING
Renders to a Lens Corrected Surface
Original Image LMS Image

13. TRADITIONAL STEREO RENDERING
Requires 2 Geometry Passes
Left Eye (Pass 1) Right Eye (Pass 2)

14. SINGLE PASS STEREO
Renders Left & Right Eye in One Geometry Pass
Left Eye
Right Eye

15. ‘MOTION TO PHOTON’ LATENCY KEY
TO COMFORTABLE VR

16. VR LATENCY WITHOUT TIMEWARP
CPU
GPU
Flip
Flash backlight
Scanout
Sample head pose
Submit to GPU
Latency

17. Timewarp
based on latest
head pose
CPU
GPU
Latency
Scanout
Flip
Submit to GPU
Flash backlight
Sample head pose
VR LATENCY WITH TIMEWARP

18. PREEMPTION IN VR
Asynchronous Time Warp
11 ms (90 Hz)
GRAPHICS OR COMPUTE
Conservative
Preemption
Request
ATW
Display
Refresh
Display
Refresh

19. PREEMPTION IN VR - PASCAL
Improved Latency & Speed
11 ms (90 Hz)
GRAPHICS OR COMPUTE
Pascal
Preemption
Request
ATW
Display
Refresh
Display
Refresh

20. VRWORKS GRAPHICS
Bringing Modern Visuals to VR
GREATER GEOMETRIC DETAIL
W/ SINGLE PASS STEREO
FASTER PIXEL SHADING
W/ LENS MATCHED SHADING
FASTER TIMEWARP
WITH PREEMPTION

21. DIRECTION PROPAGATION
SIMULATING AUDIO IN VR
SYNTHESIS
Creation of Source
Sounds
Location of
Incoming Sound
How Sound Moves in
Space

22. DIRECTIONAL AUDIO
‘HRTF’ MODELS THE AUDIO CUES THAT YOUR BRAIN USES TO DETERMINE THE POSITION OF SOUND
Level L
Level R
Interaural Level DifferenceInteraural Time Difference
Time L
Time R
Spectral Cues

23. Models Direction and Propagation Using Ray Tracing

24. Models Direction and Propagation Using Ray Tracing

25. VR TOUCH
Realistic Collision Detection for Haptics
FORCE FRICTION
PhysX API
PhysX Constraint Solver
Haptics Layer
RESTITUTION

26. PHYSX FLEX
HAIRWORKS FLOW
DESTRUCTION
CLOTH
NVIDIA PHYSX
Simulating behavior in VR

27. NVIDIA VR FUNHOUSE
Full Source Code Available on Github

29. Comprehensive SDK for VR Developers
GRAPHICS HEADSET AUDIOTOUCH & PHYSICS
NVIDIA VRWORKS
PROFESSIONAL
VIDEO

30. “ Forward thinking features in PASCAL VRWorks enable us to push the limits of
photoreal graphics. ”
— Kjartan Pierre Emilsson, CEO, Sólfar Studios
DEVELOPERS EMBRACE
THE NEW VRWORKS
“ The new VRWorks Lens Matched Shading and Single Pass Stereo features
are smart, novel approaches to the performance challenges VR applications
are faced with. ”
— Brandon Laatsch, Co-founder, Stress Level Zero
“ We took the performance headroom we gained by adding VRWorks
technologies to Mars 2030 and we used it to bring new levels of immersion
and realism to experience. ”
— Julian Reyes, lead VR producer for FUSION
“ We are looking forward to bringing NVIDIA's new PASCAL VRWorks features
to Valkyrie to take the game’s visuals and performance to another level. ”
— Hilmar Veigar Pétursson, CEO, CCP Games

31. GROWING VRWORKS ADOPTION
APPLICATIONS
HEADSETS ENGINES

32. VRWORKS GAME ENGINE INTEGRATION
VRWorks Graphics Available Now
on Github!
VRWorks is Coming to Unity
Soon!

33. FOR MORE INFO ON VR
Website
www.nvidia.com/virtualreality
VRWorks SDK
https://developer.nvidia.com/vrworks
GameWorks SDK
https://developer.nvidia.com/gameworks

34. Thank you and have
a great AWE Europe!
Questions?
Dominic Eskofier
Virtual Reality
Manager EMEAI
deskofier@nvidia.com