1. Outline
Introduction
Triangulation methods
Practical examples
Conclusion
Triangulation Methods
Seminar work
Robotics and Medicine SS 09
Institut f¨r Prozessrechentechnik, Automation und Robotik
u
(IPR)
Zlatka Mihaylova
Supervisor: M.Phys. Matteo Ciucci
July 13, 2009
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

2. Outline
Introduction
Triangulation methods
Practical examples
Conclusion
Introduction
The human visual perception system
Epipolar geometry
Triangulation methods
3D point reconstruction
Computation of the Fundamental matrix F
Practical examples
Active triangulation
Conclusion
Appliance in the medical robotics
Closing words
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

3. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
Stereovision
Why are we able to percept the relative distance to all objects?
Why is it so important to measure the distance to and
between the objects?
How can another point of view help in solving this problem?
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

4. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
Stereovision principle
x1
b f d
f
Disparity p = b d , where f represents the lens focal length
p is proportional to the stereoscopic base b and inversely
proportional to d - the distance to the measured object.
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

5. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
Basics of the epipolar geometry
C
�
epipolar plane
c c’
ep
e
ipo
in
rl
lar e e’
a
lin
ol
A baseline B
ip
e
ep
The baseline connects camera centers A and B and intersects
the image planes in the epipoles e and e .
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

6. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
Basics of the epipolar geometry
C
�
epipolar plane
c c’
ep
e
ipo
in
rl
lar e e’
a
lin
ol
A baseline B
ip
e
ep
The baseline connects camera centers A and B and intersects
the image planes in the epipoles e and e .
The epipolar plane π is defined by the camera centers and the
3D object point C .
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

7. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
Basics of the epipolar geometry
C
�
epipolar plane
c c’
ep
e
ipo
in
rl
lar e e’
a
lin
ol
A baseline B
ip
e
ep
The baseline connects camera centers A and B and intersects
the image planes in the epipoles e and e .
The epipolar plane π is defined by the camera centers and the
3D object point C .
The ambiguous projection of C .
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

8. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
The Fundamental Matrix F and the camera matrices P, P
F is a 3 × 3 matrix representing the mapping between a point
in the first image and epipolar line in the second image.
For all pairs of image points c and c the correspondence
condition holds:
T
c Fc = 0 (1)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

9. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
The Fundamental Matrix F and the camera matrices P, P
F is a 3 × 3 matrix representing the mapping between a point
in the first image and epipolar line in the second image.
For all pairs of image points c and c the correspondence
condition holds:
T
c Fc = 0 (1)
The camera matrices P and P satisfy the conditions c = PC
and c = P C for every point correspondence
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

10. Outline
Introduction
The human visual perception system
Triangulation methods
Epipolar geometry
Practical examples
Conclusion
The Fundamental Matrix F and the camera matrices P, P
F is a 3 × 3 matrix representing the mapping between a point
in the first image and epipolar line in the second image.
For all pairs of image points c and c the correspondence
condition holds:
T
c Fc = 0 (1)
The camera matrices P and P satisfy the conditions c = PC
and c = P C for every point correspondence
In the case, when we deal with calibrated cameras, it is
cleverer to compute the Essential Matrix E , which is
specialization of F :
−T
F =P EP −1 (2)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

11. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
General approach
Algorithm:
Take two images of the scene, separated by a baseline
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

12. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
General approach
Algorithm:
Take two images of the scene, separated by a baseline
Identify the point correspondences in the images
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

13. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
General approach
Algorithm:
Take two images of the scene, separated by a baseline
Identify the point correspondences in the images
Apply the triangulation rules: compute F , P and P
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

14. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
General approach
Algorithm:
Take two images of the scene, separated by a baseline
Identify the point correspondences in the images
Apply the triangulation rules: compute F , P and P
Find these two lines, which intersection defines the searched
world point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

15. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
Identification the point correspondences in the images
The most difficult part is finding the point correspondences
automatically! Robust pattern matching algorithm needed!
Harris corner detector: simple but scales dependent
Successful combination of Harris and Laplacian detectors:
www.robots.ox.ac.uk/∼vgg/research/affine/det eval
files/mikolajczyk ijcv2004.pdf
Laplacian and Difference of Gaussian (DoG) ”points of
interest” detectors
Salient region detector: www.robots.ox.ac.uk/∼vgg/research/
affine/det eval files/kadir04.pdf
Maximally stable extremal regions (MSER)
(http://www.robots.ox.ac.uk/∼vgg/research/affine/det eval files/
matas bmvc2002.pdf - specially developed for the stereo
problem analysis)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

16. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
Algorithms for computing F
Having F computed gives us the possibility to estimate the scene
points. There are some algorithms available:
Eight point algorithm: F has 8 degrees of freedom, therefore
we need 8 unique point pairs to compute it. Every pair defines
equation, which solution contains the nine coefficients of F
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

17. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
Algorithms for computing F
Having F computed gives us the possibility to estimate the scene
points. There are some algorithms available:
Eight point algorithm: F has 8 degrees of freedom, therefore
we need 8 unique point pairs to compute it. Every pair defines
equation, which solution contains the nine coefficients of F
Algebraic minimization algorithm: based on the eight point
algorithm, but tries to minimize the algebraic error caused by
noisy measurement.
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

18. Outline
Introduction
3D point reconstruction
Triangulation methods
Computation of the Fundamental matrix F
Practical examples
Conclusion
Algorithms for computing F
Having F computed gives us the possibility to estimate the scene
points. There are some algorithms available:
Eight point algorithm: F has 8 degrees of freedom, therefore
we need 8 unique point pairs to compute it. Every pair defines
equation, which solution contains the nine coefficients of F
Algebraic minimization algorithm: based on the eight point
algorithm, but tries to minimize the algebraic error caused by
noisy measurement.
Gold standard algorithm: dealing with the problem of
Gaussian noise. This approach uses statistical methods for
solving the triangulation puzzle, namely computing F by
minimizing the Likelihood function. (proposed in the book:
”Multiple View Geometry in Computer Vision” - Richard
Hartley and Andrew Zisserman)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

19. Outline
Introduction
Triangulation methods Active triangulation
Practical examples
Conclusion
Light spot technique
Simple construction: laser ray, lens, detector (CCD or PSD)
Advantages: fast, accurate, independent from surface color
Disadvantages: the surface should be no ideal mirror
laser
Ө
p’ PSD
q’
h
q
p
measured
object
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

20. Outline
Introduction
Triangulation methods Active triangulation
Practical examples
Conclusion
Stripe projection
The object’s surface manipulates the scan line
Resulting displacement in the light stripe ∼ to obj. distance
camera
laser
d
measured object
h
Ө
reference surface
(a) (b)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

21. Outline
Introduction
Triangulation methods Active triangulation
Practical examples
Conclusion
Projection of encoded patterns
Disadvantage of the stripe projection: too slow
Correspondence problem by static line pattern projection
Solutions: Binary coding, Grey coding, Phase shifted pattern
projection, Colored pattern (the picture is taken from the
book ”Digitale Bildverarbeitung” - Bernd J¨hne)
a
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

22. Outline
Introduction
Appliance in the medical robotics
Triangulation methods
Closing words
Practical examples
Conclusion
Polaris R , NDI
Standard optical tracking system in medicine, produced by
Northern Digital Inc. (NDI)
Offers passive, active and hybrid tracking.
The triangulated points are fixed on the surgical instrument.
http://www.ndigital.com/medical/polarisfamily.php
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

23. Outline
Introduction
Appliance in the medical robotics
Triangulation methods
Closing words
Practical examples
Conclusion
A.R.T. R Systems
Advanced Realtime Tracking GmbH (A.R.T. GmbH)
Multiple camera systems - 3, 4, 5 cameras for better results
Example system: smARTtrack - two ARTtrack2 cameras
mounted on a rigid bar, so that no calibration needed.
different configurations depending on focal length, angle
between both cameras, baseline
http://www.ar-tracking.de/smARTtrack.49.0.html
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

24. Outline
Introduction
Appliance in the medical robotics
Triangulation methods
Closing words
Practical examples
Conclusion
Da Vinci R Surgical System
a) A high-resolution 3D endoscope coupled with two 3-chip
cameras take the surgeon ”inside” the patient
b) The console helps by visualizing the camera records and by
repositioning the surgical camera inside the patient.
www.intuitivesurgical.com/products/davinci surgicalsystem
(c) (d)
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

25. Outline
Introduction
Appliance in the medical robotics
Triangulation methods
Closing words
Practical examples
Conclusion
Conclusion
Through the methods of triangulation the robots similar to
humans process the visual information.
For triangulation the following prerequisites are needed:
at least 2 points of view (implemented either with cameras or
mixed with light sources)
object point, placed on a comparably closer distance (not at
infinity)
statistically stable algorithms for computing the point
correspondences, respectively the distance to the world point
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods

26. Outline
Introduction
Appliance in the medical robotics
Triangulation methods
Closing words
Practical examples
Conclusion
Questions time
Thank You for Your attention!
Zlatka Mihaylova Supervisor: M.Phys. Matteo Ciucci Triangulation Methods