9. NETRA = Inverse of Shack-Hartmann
Spot Diagram on LCD
Cell Eye
Phone Piece
Display
9
10. Inverse of Shack-Hartmann
User interactively creates the Spot Diagram
Spot Diagram on LCD
Displace 25
points but
3 parameters 10
11. Inverse of Shack-Hartmann
User interactively creates the Spot Diagram
Spot Diagram on LCD
Displace 25
points but
3 parameters 11
12. Relaxed Eye with Myopia
Eye
Red point Blurred
at infinity point
Focusing Range
perfect vision
myopia
hyperopia
infinity ~10cm
12
13. Relaxed Eye with Myopia
Eye
Pinholes
Red point Distinct
at infinity image
points
Focusing Range
perfect vision
myopia
Scheiner’s Principle
hyperopia
infinity ~10cm
13
14. Relaxed Eye with Myopia
Eye
Display
A
Virtual red point Distinct
at infinity image
B
points
Focusing Range
perfect vision
myopia
hyperopia
infinity ~10cm
14
15. Relaxed Eye with Myopia
Eye
Display
Move spots
towards each
other A
Distinct
image
Virtual red point B points
at finite distance
Focusing Range
perfect vision
myopia
hyperopia
infinity ~10cm
15
16. Relaxed Eye with Myopia
Eye
Display
Move spots
towards each
other A
Points
overlap
Virtual red point B
at finite distance
Focusing Range
perfect vision
myopia
hyperopia
infinity ~10cm
16
17. Relaxed Eye with Myopia
Eye
Display
Move spots
towards each
other A
Points
overlap
Virtual red point B
at finite distance
d
Focusing Range
perfect vision
myopia
hyperopia
infinity ~10cm
17
18. Relaxed Eye with Myopia
Eye
1
d
Point at Points
infinity overlap
d
Focusing Range
perfect vision
myopia
hyperopia
infinity ~10cm
18
23. Interactive Method
Farthest Focal Point
(myopia, hyperopia, astigmatism) 24
24. Limitations
• Children
• Ability to align lines
• Resolution is a function of the display DPI
– Samsung Behold II – 160 DPI – 0.35D
– Google Nexus One – 250 DPI – 0.2D
– Apple iPhone 4G – 326 DPI – 0.14D
25
32. CATRA: Quantitative
Cataract Maps
Unique, quantitative lens mapping
with: Erick Passos, Jan Zizka, Everett Lawson, Esteban Clua , Manuel M. Oliveira, Ramesh Raskar
33. Testing the Presence of Cataracts
Blinking patterns on Screen
Pinhole
Lens
Cell
Phone
Display
34
34. 36 years old female with Posterior Sub-Capsular (10%)
Retro-illumination
Slit-lamp CATRA’s Opacity Map
35. For the Future
• Ophthalmology for Masses
– >.5 billion URE. > 2.5 billion RE. -> Devices for all
• Quality of phones (resolution) will increase exponentially
• New features (recently cataracts, next Retinal NETRA)
• Smart phones will take over the market in developing world
countries like India in next 5 years.
• Hardware store
44. Needs expert, Moving parts, Shining lasers
Retino Auto- Chart In-Focus: Solo-
scope w/ refracto- with Focometer Optiopia health: NETRA
Lenses meter Lenses EyeSite
Technology Shining Light Fundus Moving lenses Moving Reading Cellphone
plus lenses Camera + target lenses + chart on + eyepiece
target monitor
Cost to buy $2,000* ~$10,000 ~$100 ~$495 ~$200 -- $30
Cost per test ~$36 ~$36 ~$5 -- -- -- ~$1
Data capture No Comp. No No No Comp. Phone
Mobility <500g >10Kg 2kg 1kg <5kg >10Kg <100g
Speed Fast Fast Medium Medium -- Fast Fast
Scalability No No No Yes Probably No Yes
Accuracy 0.15 0.15 0.5 0.75 -- -- <0.5
Self evaluation No No Yes Yes Yes Yes Yes
Electricity Req No Yes No No -- Yes No
Astigmatism Yes Yes Yes/No No -- Yes Yes
Network No Yes No No No Yes Yes
Training High High High Medium Medium Low Low
* Phoropter-based: $5,000.00
45. Human Eye
Retina
Human Eye
Cornea
(~40D) Crystalline lens
(10~20D) 46
46. Human Eye
Accommodation Retina
Human Eye
Cornea
(~40D) Crystalline lens
(10~20D) 47
47. Perfect Vision System
Infinity
Subject
can focus
at infinity
Human Eye
Accommodation Range
Normal Vision
Infinity 10cm 48
48. Myopia (nearsightedness)
Infinity
Subject
cannot
focus
Wrong at far
focal point distances
Human Eye
Accommodation Range
Normal Vision
Myopia
Infinity 10cm 49
49. Myopia Correction
Infinity
Subject
can focus
at infinity
Divergent Lens
Human Eye
Accommodation Range
Normal Vision
Corrected Myopia
Myopia
Infinity 10cm 50
50. Hyperopia (farsightedness)
Infinity
Wrong
focal point
Human Eye
Accommodation Range
Normal Vision
Myopia
Hyperopia
Infinity 10cm 51
51. Hyperopia Correction
Infinity
Convergent Lens
Human Eye
Accommodation Range
Normal Vision
Myopia
Corrected Hyperopia
Hyperopia
Infinity 10cm 52
52. Refractive Errors and Shifted Range
Need to Perfect vision
measure
Myopia
Hyperopia
Infinity 1m 33cm 10cm Distance
54
60. Measuring the Accommodation Range
Perfect vision
Myopia
Hyperopia
Infinity ~10cm
Step 1: Far limit Step 2: Near limit
62
61. Measuring the Accommodation Range
Perfect vision
Myopia
Hyperopia
Infinity ~10cm
Step 1: Far limit Step 2: Near limit
63
62. Measuring the Accommodation Range
Perfect vision
Myopia
Hyperopia
Infinity ~10cm
Step 1: Far limit Step 2: Near limit
64
63. Relaxed Eye
Display
A
Virtual Point at Points
the far limit B
overlap
65
64. Accommodated Eye
Display
Move points towards
each other
A
Points
B overlap
Virtual point
getting closer
Subject Accommodates
to fix the “blur”
66
65. Accommodated Eye
Display
Move points towards
each other
A
Points
B overlap
Virtual point
getting closer
Subject Accommodates
to fix the “blur”
67
66. Accommodated Eye
Display
Move points towards
each other
A
Points
B overlap
Virtual point
getting closer
Subject cannot
accommodate more
than the previous point
68
67. Patterns for Alignment Task
A B A B A B A B A B
Displayed
Subject view
A B A B A B A B A B
Displayed
Subject view
Visual
69
Cryptography [NaorShamir94]
68. Patterns for Alignment Task
A B A B A B A B A B
Displayed
Subject view
A B A B A B A B A B
Displayed
Subject view
Visual
70
Cryptography [NaorShamir94]
69. Summary of Interaction
Accommodation Range
Farthest Point Nearest Point
(myopia, hyperopia, astigmatism) (presbyopia) 71
Notas del editor
Thanks, xxx
Everybody knows this machine, right? They call it thermometer. For me, this is the most amazing device medicine has ever used. And you know, nobody teaches on how to use a thermometer. We just somehow learned when we were kids. We started using and seeing that after the red mark something bad is going on and we need to see a doctor. It’s cheap, simple to use, no language barriers, no versions for rich and poor, it has a global spread and provides the first screen for a lot of diseases. I would guess that this guy has saved more life than anything else by just telling people when they should see a doctor.
Now Imagine if you have a 2 dollar plastic device that you put close to your eye, press a button that says compute, and it gives you numbers that represent your nearsightedness, farsightedness and astigmatism, presbyopia and cataracts. Sort of a thermometer for vision. In the sense that since you have your number you know when you need to see a doctor. Well, what we have been working on is called NETRA, and it does exactly what I just said
It is a clip-on for phones that you put it on, run the app, look close, align those red and green lines and the result is your refractive error, or cataract condition. You can test your eyesight how many times you want, by yourself, anywhere, with the same accuracy an optometric tool would do today.
2 billion people have refractive errors And half a billion in developing countries worldwide have uncorrected vision that affects their daily livelihood. They don’t have access to an optometrist or it simply too expensive. While making and distributing of lenses has become quite easy now, surprisingly there is still no easy solution for measuring eyesight. So our solution involves using the 4.5B phones out there to reach the whole 6B people.
The most accurate method is based on a so called SH WS. It involves shining a laser at the back of the retina and observing the wavefront using a sophisticated sensor. We ask user to generate a spot diagram. But navigating in a high dimensional space is challenging so we come up with a strikingly simple approach to let the user interactively create the spot diagram. We are first to make connection between Shack Hartmann and Lightfields (and it goes well with recent work in computational photography about ALF and Zhang/Levoy). Connection to Adaptive optics/ Astronomy. The way that this device works is that, it shines a lasers in the eye, the laser is reflected in the retina and comes out of the eye being distorted by the cornea. These light rays reaches an array of lenses that focus them to dots in a sensor. The device measures how much this dots deviate from the ideal case. Since it uses lasers, the device is expensive and requires trained professionals
For a normal eye, the light coming out of the eye forms a parallel wavefront. The sensor has a lenslet array and we get a spot diagram of uniform dots. This lenslet should remind you of a lightfield camera, and in fact Levoy and others showed last year that there is a close relationship between the two. In addition, Zhang and Levoy, plus our grp has shown the relationship between wavefront sensing and lightfield sensing.
When the eye has a distortion, the spot diagram is not uniform. And the displacement of the spots from the center indicates the local slope of the wavefront. From the slope one can integrate and recover the wave shape.
NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user. And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently. But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently. But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
So, lets start with an eye with myopia. Remember, they cannot see far, so a red point at infinity for them will look like a red blur.
Using Shceiner’s principle, if we put two pinholes in the field, this will instead create two distinct dots.
Instead of a distant point source, we put an LCD display behind the pinholes. If we draw two spots exactly under these pin-holes, we create a virtual point at infinity.
So, as we move the two red circles toward each other, the virtual point gets closer to the subject and he sees the two red dots getting closer.
When this two red circles overlaps for the subject, we can compute d based on the spot displacements
Which is the distance between the eye and this virtual point.
Turns out that the inverse of D is the refractive power required for this person to see clearly objects at infinity. In other words, the lens that will shift the accommodation range of this subject back to the regular one.
In case of a perfect eye using the system, since the subject can see far, he will see the two points overlapping in his retina, meaning that he does not need glasses.
So, we do the alignment task for a few meridians
By showing oriented lines on the display.
In the end, we best fit the sinusoidal curve over the four measured values to estimate the astigmatic parameters.
In the end, we best fit the sinusoidal curve over the four measured values to estimate the astigmatic parameters.
Since we are relying on the user interaction, the subject has to be aware of the alignment tasks. So, very young Children may not be able to run the test. Instead of just one eye, one may use both eyes to exploit convergence. And of course, the resolution of NETRA itself is a function of the resolution of the display. With a 326 dpi display, resolution is 0.14 diopters and presciption glasses come in increments of 0.25 diopters. So our system is already sufficiently accurate.
Reading charts appear to be an easy solution, this method has too many problems. Sharpness of legible text is very subjective. The brightness of the chart has to be very carefully chosen otherwise the pupil size will change, increasing depth of field, and allowing user to recognize even lower rows. The trial lenses + the lens frame the doctor will use also cost over $150 % Reading chart tests involve using a frame or a phoropter. The doctor will swing a sequence of lenses in front of your eye and ask for which lens allows you to see the lower rows on the reading chart.
NETRA matches Retinoscopy
About 8 million worldwide are blind (worse than 3/60 vision) because of uncorrected refractive error, mostly from the developing world…3million from India. About 22.5 million worldwide are blind because of cataracts, 19 million in the developing world, 14 million in India. From lowered productivity to less independence in conducting simple tasks, the burden of blindness is well known. This can be solved if they had access to a diganostic test and glasses, but they don’t have access to an optometrist or it simply too expensive. While making and distributing of lenses has become quite easy now, surprisingly there is still no easy solution for measuring eyesight. Can we use a fraction of the 4.5B cellphone displays to address this problem?
NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user. And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
Thanks XXX NETRA is a clip-on device that you attach to your cell phone. You look close, press some buttons, you hit calculate and it gives you the prescription for glasses. It’s a 2-dollar device that measures nearsightedness, farsightedness and astigmatism with the same accuracy that doctors have in their clinic. To understand what happened here, let’s think about the evolution of photography.
In 1960s, photography equipment was really crappy. They were expensive and bulky equipment, require specialized training , with high maintenance costs and they were not smart at all. But the worse thing about photography in that time is that you must go to a specific place to take the picture and then go back to get the results.
Well, today things changed. Each one of us carries at least 3 cameras: two eyes and a cell phone camera. Cameras are everywhere. They became cheap, accessible and easy-to-use without losing in accuracy.
Now, if you think about optometry today, the devices are expensive and bulky , they require specialized training , have high maintenance costs and they are not smart at all. Some of them do not even communicate with facebook. But the worse thing is that you must go to a specific place to take the eye exam and then go back to get the results.
So, we propose the increase of accessibility for optometry solutions by using high end scientific devices: cell phones. An smartphone screen today has the pixel size of 30 micrometers. At this resolution, the smartphone is not a phone anymore it is a scientific tool. With 4.5 billion phones out there, we can scale optometry and find half a billion people that today do not know that they need glasses.
Power for user intelligence can overcome very cumbersome and expensive devices. But unlike other condition eye screening is quite challenging. Modern solutions may provide students a fighting charge is a very rewarding.
Reading charts appear to be an easy solution, this method has too many problems. Sharpness of legible text is very subjective. The brightness of the chart has to be very carefully chosen otherwise the pupil size will change, increasing depth of field, and allowing user to recognize even lower rows. The trial lenses + the lens frame the doctor will use also cost over $150 % Reading chart tests involve using a frame or a phoropter. The doctor will swing a sequence of lenses in front of your eye and ask for which lens allows you to see the lower rows on the reading chart.
New wireless eyecare ecosystem Anybody can take netra to patients, load .. Mobile partners, Deliver .. Because it is mobile and deskilled, breaks the barrier to entry, takes eyecare to remote areas Decouple diagnostics from delivery
For better precision, there are many kinds of solutions, some really clever. The beauty of netra is that it avoids moving parts or shining lasers, and all intelligence is in the software.
The human eye is like a camera. It has lenses, sensors and also aberrations. The human eye is composed of two main lenses: the cornea, which is main responsible for converging light rays to the retina; and the crystalline lenses, which is responsible for our ability of focus far and close by changing its shape.
So, in a perfect vision system, the light coming from a point at infinity will converge to a single point at the retina. A subject with perfect vision see clearly from infinity to up to 10cm.
Myopes cannot see far. Therefore, all the rays coming from a point at infinity, converges before the retina. The Accommodation range for those people is shifted to close, so they can closer than regular individuals.
The correction for myopia includes a divergent lens, which brings the focal point back to the retina by shifting the Accommodation range.
Hyperopes cannot see close. All the rays coming from a point at infinity, converges behind the retina. The Accommodation range for those people is shifted to the far field, so they can actually see “beyond infinity”. This remembers-me some other story, but let keep the focus here.
The correction for myopia includes a convergent lens, which shifts the Accommodation range back to the regular indivudial.
The correction for myopia includes a convergent lens, which shifts the Accommodation range back to the regular indivudial.
We need to measure the difference between the subject’s farthest focal point wrt infinity.
And this is measured in diopters which is 1 divided by this distance.
which is an angle-dependent refractive error. An astigmatic subject has two main focal lengths in perpendicular meridians. One …
Stronger and one weaker
Think of a cornea with the shape of an american football creating a cylindrical aberration with unknown focal length and axis.
The required correction is now a function of measured angle. In order to measure the farthest point for these guys, we need to evaluate Cylindrical component, the Spherical component, and the angle theta on the equation. However, the interpolation of refractive powers between C and S leads to a s ituation where the pattern drawn on the screen matters.
As you can see in this video, the astigmatic lenses create a deviation on the path of the pattern, and they may never overlap, turning the alignment task into a 2D search for some angles.
However, if we drawn lines perpendicular to the measured angle, the alignment task is again an 1D search. The deviation still exists, but the pattern makes the task easier.
Ours is the only system where one can estimate not only the farthest point
one can focus but also
the nearest point without any mechanically moving parts. So, in order to measure the closest reading point
We draw a pattern on the screen that induces accommodation. In this way, when we move A and B closer on the screen,
the user will try to focus on a closer object. We can move this virtual point all the way to the nearest discernable point.
When the user is not able to focus anymore, the visual system give up and the user start seeing more than one pattern.
As I sad before, this is possible because we can draw whatever we want in the display. We tested many patterns, static and dynamic, including visual cryptography.
Turns out that the best pattern to induce accommodation is the sinosoidal curves aligned perpendicular to the measurement angle.
As a summary, our method has two steps. First measures the farthest point in focus in many angles using lines and the second step measures the nearest point using sinusoidals oriented on the angle of astigmatism.