图像处理之ORB特征，主要讲ORB特征的理论

1. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
ORB 特征
高洪臣
cggos@outlook.com
2020 年 7 月 10 日
高洪臣 ORB 特征 2020 年 7 月 10 日 1 / 12

2. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Image Features
feature types
feature extraction(detection) & matching
• feature detector
• feature descriptor
高洪臣 ORB 特征 2020 年 7 月 10 日 2 / 12

3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Corner / Keypoint / Interest Point
• keypoint detector
• Harris (1988)
• Shi-Tomas (1994)
• FAST1
(Features from Accelerated and Segments Test) (2006)
• AGAST (2010)
• keypoint descriptor
• BRIEF2
(Binary robust independent elementary feature) (2010)
• keypoint detector & descriptor
• SIFT (1999, 2004)
• SURF (2006)
• BRISK (Binary Robust Invariant Scalable Keypoints) (2011)
• ORB3
(2011)
• FREAK (2012)
• KAZE (2012)
1Edward Rosten and Tom Drummond. “Machine Learning for High-Speed Corner Detection.”. In: ECCV (1).
Ed. by Ales Leonardis, Horst Bischof, and Axel Pinz. Vol. 3951. Lecture Notes in Computer Science. Springer,
2006, pp. 430–443. isbn: 3-540-33832-2. url: http://dblp.uni-trier.de/db/conf/eccv/eccv2006-1.html#RostenD06.
2Michael Calonder et al. “BRIEF: Binary Robust Independent Elementary Features”. In:
Proceedings of the 11th European Conference on Computer Vision: Part IV. ECCV’10. Heraklion, Crete,
Greece: Springer-Verlag, 2010, pp. 778–792. isbn: 364215560X.
3Ethan Rublee et al. “ORB: An efficient alternative to SIFT or SURF”. In:
2011 International conference on computer vision. Ieee. 2011, pp. 2564–2571.
高洪臣 ORB 特征 2020 年 7 月 10 日 3 / 12

4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
ORB Overview
ORB = Oriented FAST + Rotated BRIEF
• oFAST (ORB Detector)
• keypoint position - FAST
• keypoint orientation - image patch moment
• keypoint response/score - NMS(Non-Maximal Supression)
• scale-invariant - image pyramid
• rBRIEF (ORB Descriptor)
• keypoint descriptor - BRIEF
• rotation-invariant - keypoint orientation
高洪臣 ORB 特征 2020 年 7 月 10 日 4 / 12

5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Multiscale Image Pyramid
Parameters
• level: 8
• scale: 1.2
• downsample:
bilinear interpolation
produce FAST features and compute descriptors at each level in the pyramid
高洪臣 ORB 特征 2020 年 7 月 10 日 5 / 12

6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
FAST
fast-9
• 9 contiguous pixels in a circle (of 16 pixels) brighter than Ip + t or darker than Ip − t
• Rapid rejection by testing 1, 9, 5 then 13
NMS (Non-Maximal Suppression) 1
remove corners which have an adjacent corner with higher score
Uniform Distribution
DistributeOctTree() 2
1all called non-maximum suppression
2use Quad-Tree to iteratively segment image regions in ORB-SLAM2
高洪臣 ORB 特征 2020 年 7 月 10 日 6 / 12

7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Orientation by Intensity Centroid (IC)
the moments of a patch:
mpq =
∑
x,y
xp
yq
I(x, y)
the first order moment of a patch 1
I (radius = 15):
m10 =
15∑
x=−15
15∑
y=−15
xI(x, y) =
15∑
y=0
15∑
x=−15
x [I(x, y) − I(x, −y)]
m01 =
15∑
x=−15
15∑
y=−15
yI(x, y) =
15∑
y=1
15∑
x=−15
y [I(x, y) − I(x, −y)]
the intensity centroid:
C =
(
m10
m00
,
m01
m00
)
the orientation (from the corner’s center to the centroid):
θ = atan 2 (m01, m10)
1a circular patch
高洪臣 ORB 特征 2020 年 7 月 10 日 7 / 12

8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Image Gaussian Filtering/Blurring/Smoothing 1
why filtering
start by smoothing image using a Gaussian kernel at each level in the pyramid in order to
prevent the descriptor from being sensitive to high-frequency noise
Gaussian Kernel
G(u, v) =
1
2πσ2
e
− u2+v2
2σ2
Gn(u, v) =
1
s
· e
− u2+v2
2σ2 , s =
w∑
u=−w
w∑
v=−w
e
− u2+v2
2σ2
(a) (u,v) (b) kernel (σ=1.5) (c) normalized
Image Gaussian Filtering: I′(i, j) =
∑w
u=−w
∑w
v=−w
I(i + u, j + v)Gn(u, v)
1low-pass filter
高洪臣 ORB 特征 2020 年 7 月 10 日 8 / 12

9. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Separability of the Gaussian filter
s =
w∑
u=−w
w∑
v=−w
g(u) · g(v) =
( w∑
u=−w
g(u)
)
·
( w∑
v=−w
g(v)
)
= s′
· s′
Gn(u, v) =
1
s
· e− u2
2σ2
· e− v2
2σ2
=
1
s
· g(u) · g(v) =
g(u)
s′
·
g(v)
s′
Separable Kernel Matrix:
G(2w+1)×(2w+1) =
1
s








g(−w)g(−w) . . . g(−w)g(0) . . . g(−w)g(w)
...
...
...
g(0)g(−w) . . . g(0)g(0) . . . g(0)g(w)
...
...
...
g(w)g(−w) . . . g(w)g(0) . . . g(w)g(w)








=
1
s′








g(−w)
...
g(0)
...
g(w)








·
1
s′
[
g(−w) . . . g(0) . . . g(w)
]
高洪臣 ORB 特征 2020 年 7 月 10 日 9 / 12

10. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Separability of the Gaussian filter
Image Gaussian Filtering:
I′
(i, j) =
w∑
u=−w
w∑
v=−w
I(i + u, j + v)Gn(u, v)
=
w∑
u=−w
w∑
v=−w
I(i + u, j + v)
1
s
g(u)g(v)
=
w∑
u=−w
w∑
v=−w
I(i + u, j + v)
1
s′
g(u)
1
s′
g(v)
=
w∑
u=−w
[ w∑
v=−w
I(i + u, j + v)
g(v)
s′
]
g(u)
s′
=
w∑
u=−w
S(i + u)
g(u)
s′
• kernel size: (2w + 1) × (2w + 1), w = 3
1 int gaussKernel[4] = { 224, 192, 136, 72 };
高洪臣 ORB 特征 2020 年 7 月 10 日 10 / 12

11. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BRIEF
• vector dim: 256 bits (32 bytes)
• each vector ←→ each keypoint
for each bit, select a pair of points in a patch I which centered a corner p and compare
their intensity
S =
( p1, . . . , pn
q1, . . . , qn
)
∈ R(2×2)×256
τ(I; pi, qi) :=
{
1 : I(pi) I(qi)
0 : I(pi) ≥ I(qi)
the descriptor (each bit ←→ each pair of points (pi, qi)):
f(n) =
n
∑
i=1
2i−1
τ(I; pi, qi), (n = 256)
高洪臣 ORB 特征 2020 年 7 月 10 日 11 / 12

12. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
rBRIEF
• construct a lookup table of precomputed BRIEF patterns
1 static int ORB_pattern[256*4] = {
2 8, -3, 9, 5,
3 4, 2, 7,-12,
4 -11, 9, -8, 2,
5 7,-12, 12,-13, // ...
6 }
• steered BRIEF, for each bit of the descriptor
p′
i = p + Rθ(pi − p)
q′
i = p + Rθ(qi − p)
, Rθ =
[
cos θ − sin θ
sin θ cos θ
]
τ(I; p′
i, q′
i) :=
{
1 : I(p′
i) I(q′
i)
0 : I(p′
i) ≥ I(q′
i)
1 float angle = (float)kpt.angle*factorPI;
2 float a = (float)cos(angle), b = (float)sin(angle);
3 #define GET_VALUE(idx)
4 center[cvRound(pattern[idx].x*b + pattern[idx].y*a)*step +
5 cvRound(pattern[idx].x*a - pattern[idx].y*b)]
高洪臣 ORB 特征 2020 年 7 月 10 日 12 / 12