Presented at SIGGRAPH 2010.

1. Photorealistic Models for Pupil Light Reflex and Iridal Pattern Deformation Vitor F. Pamplona Manuel M. Oliveira Gladimir V. G. Baranoski

2. The most important feature in facial animation Eyes © Marco Ruiz - http://bit.ly/bba1qw

3. Pupil Light Reflex (PLR) Involuntary movement of the pupil Deforms the iris patterns

4. Close-Ups Require Natural Looking Eyes 4 Image: http://wallpaper-s.org

6. First to simulate pupil dynamics under variable lighting

7. All parameters derived from experimental data

8. Real-time predictable animations

9. Support for individual variability

11. 7 Our Models in Action

14. Static9 Moon, P. and Spencer, D.. Onthe stiles-crawford effect. J. Opt. Soc. Am. 1944

15. 10 The Dynamics of PLR Retina Brain

16. 11 Retina Brain The Dynamics of PLR

17. 12 Retina Brain The Dynamics of PLR

18. Retina 13 Brain The Dynamics of PLR

19. Retina 14 Brain The Dynamics of PLR

20. Retina 15 Brain The Dynamics of PLR

21. Retina 16 Brain The Dynamics of PLR

22. 17 Retina Brain The Dynamics of PLR

24. Uses a delay differential equation18 Longtin, A. and Milton, J. G.. Modellingautonomousoscilations in thehumanpupil light reflex usingnon-lineardelay-differentialequations. BulletinofMath. Bio. 1989.

26. Uses a delay differential equationPupil Area Longtin, A. and Milton, J. G.. Modellingautonomousoscilations in thehumanpupil light reflex usingnon-lineardelay-differentialequations. BulletinofMath. Bio. 1989.

28. Uses a delay differential equationPupil Area

30. Uses a delay differential equationLatency Retinal Light Flux Pupil Area Longtin, A. and Milton, J. G.. Modellingautonomousoscilations in thehumanpupil light reflex usingnon-lineardelay-differentialequations. BulletinofMath. Bio. 1989.

32. Uses a delay differential equationRetinal Light Flux Longtin, A. and Milton, J. G.. Modellingautonomousoscilations in thehumanpupil light reflex usingnon-lineardelay-differentialequations. BulletinofMath. Bio. 1989. 22 Related Work (Cont.) Latency Muscular Activity Pupil Area

34. Outline Physiologically-based Model for PLR Iridal Pattern Deformation Model Results Summary 24

35. Summary Physiologically-based model from Longtin & Milton Static model of Moon & Spencer 25

36. Pupil Light Reflex Model Longtin & Milton’s (L&M) Model 26

37. Pupil Light Reflex Model L&M model under constant illumination condition becomes 27 Longtin & Milton

38. Pupil Light Reflex Model 28 L&M model under constant illumination condition becomes Longtin & Milton Moon & Spencer

39. Pupil Light Reflex Model 29 L&M model under constant illumination condition becomes Longtin & Milton Moon & Spencer Rewritten

40. Pupil Light Reflex Model 30 L&M model under constant illumination condition becomes Longtin & Milton Moon & Spencer Rewritten

41. Pupil Light Reflex Model 31 L&M model under constant illumination condition becomes Longtin & Milton Moon & Spencer Rewritten

42. Pupil Light Reflex Model 32 L&M model under constant illumination condition becomes Moon & Spencer Rewritten

43. Pupil Light Reflex Model 33 L&M model under constant illumination condition becomes Moon & Spencer Rewritten

44. Pupil Light Reflex Model 34 From Moon & Spencer model L&M model under constant illumination condition becomes

45. Pupil Light Reflex Model 35 L&M model under constant illumination condition becomes

46. Pupil Light Reflex Model 36 L&M model under constant illumination condition becomes

47. Pupil Light Reflex Model 37 L&M model under constant illumination condition becomes

48. Pupil Light Reflex Model 38 L&M model under constant illumination condition becomes

49. Static Model Comparison 39 Moon & Spencer Our Model

50. Static Model Comparison 40 Moon & Spencer Our Model Just for the equilibrium case

51. Pupil Light Reflex Model 41 L&M model under constant illumination condition becomes

52. Pupil Light Reflex Model 42 L&M model under constant illumination condition becomes

53. Pupil Light Reflex Model The Dynamic Model for PLR 43

54. Pupil Light Reflex Model The Dynamic Model for PLR 44

55. PLR Model: Individual Sensitivity to Light Mapped as iso-curves in Moon & Spencer’s data 45 Moon, P. and Spencer, D.. On the stiles-crawford effect. J. Opt. Soc. Am. 1944

56. PLR Model: Individual Sensitivity to Light Mapped as iso-curves in Moon & Spencer’s data 46 Ct Cm Cb Moon, P. and Spencer, D.. On the stiles-crawford effect. J. Opt. Soc. Am. 1944

57. PLR Model: Individual Sensitivity to Light Mapped as iso-curves in Moon & Spencer’s data 47 Ct Cm Cb Subject X Moon, P. and Spencer, D.. On the stiles-crawford effect. J. Opt. Soc. Am. 1944

58. PLR Model Validation Videos captured while changing the illumination The illumination was measured (lux meter) The pupil size in mm was estimated for each frame

59. PLR Model: Individual Variability 49

60. PLR Model: Individual Variability 50

61. PLR Model: Individual Variability 51

62. PLR Model: Individual Variability 52

63. Outline Physiologically-based Model for PLR Iridal Pattern Deformation Model Results Summary 53

64. 54 Iris Texture

65. Feature Tracking 55

66. Feature Tracking 56

67. Feature Tracking 57

68. Feature Tracking 58

69. Feature Tracking 59

70. Tracking Results 60

71. Relative Position to the Iris Border 61

72. Relative Position to the Iris Border 62

73. Relative Position to the Iris Border 63

74. Relative Position to the Iris Border 64

75. Relative Position to the Iris Border 65

76. Simulation of Iris Deformation

77. Comparison Against Photographs Orig. Texture Rendering Picture 67

78. Comparison Against Photographs Orig. Texture Rendering Picture 68

79. Final Result 69

80. Summary A Physiologically-based Pupil Light Reflex (PLR) Model First to simulate pupil dynamics under variable lighting All parameters derived from experimental data Real-time predictable animations Support for individual variability Iridal Pattern Deformation Model Produce photorealistic animations in real time 70

81. Possible Applications Screening tool for Eye diseases such as … XXXX Intoxication by alcohol/drugs An average healthy subject model for comparison Ophthalmologic tool for simulations and training Iris recognition withoutcontrolled illumination 71

82. Acknowledgments Volunteers and Collaborators Dr. JacoboMelamedCattan, MD Prof. Roberto Silva Prof. Luis A. V. Carvalho Leandro Fernandes, Marcos Slomp and RenatoSilveira CNPq-Brazil fellowship (305613/2007-3) NSERC-Canada grant (238337) Microsoft Brazil

83. Photorealistic Models for Pupil Light Reflex and Iridal Pattern Deformation Vitor F. Pamplona Manuel M. Oliveira Gladimir V. G. Baranoski

85. Photorealistic Models for Pupil Light Reflex and Iridal Pattern Deformation Vitor F. Pamplona Manuel M. Oliveira Gladimir V. G. Baranoski

86. PLR Model: Light Bulb Validation 76

87. PLR Model: Light Bulb Validation 77

88. PLR Model: Flash Light Validation 78 * Light intensity obtained from the inverse of Moon and Spencer

89. Applications Photorealistic animations Non-adhoc automatic animation of the iris-pupil system Tool for diagnostics An average healthy subject model for comparison Ophthalmologic simulations Iris recognition Controlled illumination is not needed anymore 79

91. Under constant illumination conditions

92. Frequency of 0.05 a 0.3Hz

93. Aplitude about 0.2mm

94. Implementation

95. Random perturbations in the light source

96. From -100.3 to 100.3 Blondels

97. In a frequency range of 0.05 a 0.3Hz80 Stark, L. W., Sherman, P. M. A servoanalytic study of consensual pupil reflex to light. J. Neurophysiol, 1959. Hachol, A. et al. Measurement of pupil reactivity using fast pupillometry. Physiol. Meas., 2007.

98. Wyatt’s model for Iris Structure Model for iris’ collagen fibbers Linear and radial deformation plus a non linear component. 81 Wyatt, H. J. A minimum-wear-and-tear meshwork for the iris. Vision Research. 2000.

99. Static Model Comparison 82

100. PLR Model: Velocities Included in the numerical step Constriction is 3 times faster than dilation S is a constant that may change among individuals 83 Ellis, C. J. The pupillary light reflex in normal subjects. British J. of Ophthalmology, 1981. Bergamin, O. et al. The influence of iris color on the pupillary light reflex. Graefes Arch. Clin. Exp. Ophthalmol., 1998.

101. PLR Model: Individual Variability Mapped as iso-curves in Moon and Spenser’s data 84 Moon, P. and Spencer, D.. On the stiles-crawford effect. J. Opt. Soc. Am. 1944

102. Defformation model: Individual Variability 85

103. Defformation Model: Feature Tracking 86

104. Modelo 2: Tracking das Saliências 87

105. The problem of an static latency Model Copyright Vitor Pamplona 88

106. Photorealistic Facial Animations 89

108. Challenges Under variable lighting conditions: How does the pupil change? How does the iris pattern deform? 91 © Marco Ruiz - http://bit.ly/bba1qw

109. Eyes and Facial Animation © Marco Ruiz - http://bit.ly/bba1qw

110. Physiologically-based models [Animar o fluxo] 93 Privitera, C. M., Stark L. W. A. Binocular PupilModel for SimulationofRelative AfferentPupilDefectsandtheSwingingFlashlight Test. Bio. Cyber. 2006.

111. Contributions Pupil Light Reflex (PLR) model First practical model to simulate pupil dynamics under variable lighting conditions First integrated model with support to individual variability, latency, and constriction/dilation velocities Iridal pattern deformation model First model validated with human eyes. 94

112. Summary Physiologically-based model from Longtin & Milton Dynamic Constantsandfunctionsundefined Experiment-based model from Moon & Spencer Static Averagesubject 95

113. Summary Physiologically-based model from Longtin & Milton Dynamic Constants and functions (g) undefined Experiment-based model from Moon & Spencer Static Averagesubject 96

114. Eyes and Emotions