Mathematics (from Greek μάθημα máthēma, “knowledge, study, learning”)

1. Signal Processing
and
Representation Theory
Lecture 1

2. Outline:
• Algebra Review
– Numbers
– Groups
– Vector Spaces
– Inner Product Spaces
– Orthogonal / Unitary Operators
• Representation Theory

3. Algebra Review
Numbers (Reals)
Real numbers, ℝ, are the set of numbers that we
express in decimal notation, possibly with infinite,
non-repeating, precision.

4. Algebra Review
Numbers (Reals)
Example: π=3.141592653589793238462643383279502884197…
Completeness: If a sequence of real numbers gets
progressively “tighter” then it must converge to a
real number.
Size: The size of a real number a∈ℝ is the square
root of its square norm:
2
aa =

5. Algebra Review
Numbers (Complexes)
Complex numbers, , are the set of numbers that weℂ
express as a+ib, where a,b∈ andℝ i= .
Example: eiθ
=cosθ+isinθ
1−

6. Algebra Review
Numbers (Complexes)
Let p(x)=xn
+an-1xn-1
+…+a1x1
+a0 be a polynomial with
ai∈ .ℂ
Algebraic Closure:
p(x) must have a root, x0 in :ℂ
p(x0)=0.

7. Algebra Review
Numbers (Complexes)
Conjugate: The conjugate of a complex number a+ib
is:
Size: The size of a real number a+ib∈ℂ is the square
root of its square norm:
ibaiba −=+
22
)()( baibaibaiba +=++=+

8. Algebra Review
Groups
A group G is a set with a composition rule + that
takes two elements of the set and returns another
element, satisfying:
– Asscociativity: (a+b)+c=a+(b+c) for all a,b,c∈G.
– Identity: There exists an identity element 0∈G such
that 0+a=a+0=a for all a∈G.
– Inverse: For every a∈G there exists an element -a∈G
such that a+(-a)=0.
If the group satisfies a+b=b+a for all a,b∈G, then
the group is called commutative or abelian.

9. Algebra Review
Groups
Examples:
– The integers, under addition, are a commutative group.
– The positive real numbers, under multiplication, are a
commutative group.
– The set of complex numbers without 0, under
multiplication, are a commutative group.
– Real/complex invertible matrices, under multiplication
are a non-commutative group.
– The rotation matrices, under multiplication, are a non-
commutative group. (Except in 2D when they are
commutative)

10. Algebra Review
(Real) Vector Spaces
A real vector space is a set of objects that can be
added together and scaled by real numbers.
Formally:
A real vector space V is a commutative group with a scaling operator:
(a,v)→av,
a∈ ,ℝ v∈V, such that:
1. 1v=v for all v∈V.
2. a(v+w)=av+aw for all a∈ ,ℝ v,w∈V.
3. (a+b)v=av+bv for all a,b∈ ,ℝ v∈V.
4. (ab)v=a(bv) for all a,b∈ ,ℝ v∈V.

11. Algebra Review
(Real) Vector Spaces
Examples:
• The set of n-dimensional arrays with real coefficients is a
vector space.
• The set of mxn matrices with real entries is a vector
space.
• The sets of real-valued functions defined in 1D, 2D, 3D,
… are all vector spaces.
• The sets of real-valued functions defined on the circle,
disk, sphere, ball,… are all vector spaces.
• Etc.

12. Algebra Review
(Complex) Vector Spaces
A complex vector space is a set of objects that can be
added together and scaled by complex numbers.
Formally:
A complex vector space V is a commutative group with a scaling operator:
(a,v)→av,
a∈ ,ℂ v∈V, such that:
1. 1v=v for all v∈V.
2. a(v+w)=av+aw for all a∈ ,ℂ v,w∈V.
3. (a+b)v=av+bv for all a,b∈ ,ℂ v∈V.
4. (ab)v=a(bv) for all a,b∈ ,ℂ v∈V.

13. Algebra Review
(Complex) Vector Spaces
Examples:
• The set of n-dimensional arrays with complex coefficients
is a vector space.
• The set of mxn matrices with complex entries is a vector
space.
• The sets of complex-valued functions defined in 1D, 2D,
3D,… are all vector spaces.
• The sets of complex-valued functions defined on the
circle, disk, sphere, ball,… are all vector spaces.
• Etc.

14. Algebra Review
(Real) Inner Product Spaces
A real inner product space is a real vector space V
with a mapping 〈V,V〉→ℝ that takes a pair of vectors
and returns a real number, satisfying:
〈u,v+w〉= 〈u,v〉+ 〈u,w〉 for all u,v,w∈V.
〈αu,v〉=α〈u,v〉 for all u,v∈V and all α∈ℝ.
〈u,v〉= 〈v,u〉 for all u,v∈V.
〈v,v〉≥0 for all v∈V, and 〈v,v〉=0 if and only if v=0.

15. Algebra Review
(Real) Inner Product Spaces
Examples:
– The space of n-dimensional arrays with real
coefficients is an inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
〈v,w〉=v1w1+…+vnwn
– If M is a symmetric matrix (M=Mt
) whose eigen-
values are all positive, then the space of n-
dimensional arrays with real coefficients is an inner
product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
〈v,w〉M=vMwt

16. Algebra Review
(Real) Inner Product Spaces
Examples:
– The space of mxn matrices with real coefficients is an
inner product space.
If M and N are two mxn matrices then:
〈M,N〉=Trace(Mt
N)

17. Algebra Review
(Real) Inner Product Spaces
Examples:
– The spaces of real-valued functions defined in 1D,
2D, 3D,… are real inner product space.
If f and g are two functions in 1D, then:
– The spaces of real-valued functions defined on the
circle, disk, sphere, ball,… are real inner product
spaces.
If f and g are two functions defined on the circle, then:
∫
∞
∞−
= dxxgxfgf )()(,
∫=
π
θθθ
2
0
)()(, dgfgf

18. Algebra Review
(Complex) Inner Product Spaces
A complex inner product space is a complex vector
space V with a mapping 〈V,V〉→ℂ that takes a pair of
vectors and returns a complex number, satisfying:
〈u,v+w〉= 〈u,v〉+ 〈u,w〉 for all u,v,w∈V.
〈αu,v〉=α〈u,v〉 for all u,v∈V and all α∈ℝ.
– for all u,v∈V.
〈v,v〉≥0 for all v∈V, and 〈v,v〉=0 if and only if v=0.
uv,vu, =

19. Algebra Review
(Complex) Inner Product Spaces
Examples:
– The space of n-dimensional arrays with complex
coefficients is an inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
– If M is a conjugate symmetric matrix ( ) whose
eigen-values are all positive, then the space of n-
dimensional arrays with complex coefficients is an
inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
〈v,w〉M=vMwt
nn11 wv...wvwv, ++=
t
MM =

20. Algebra Review
(Complex) Inner Product Spaces
Examples:
– The space of mxn matrices with real coefficients is an
inner product space.
If M and N are two mxn matrices then:
( )NMNM, t
Trace=

21. Algebra Review
(Complex) Inner Product Spaces
Examples:
– The spaces of complex-valued functions defined in
1D, 2D, 3D,… are real inner product space.
If f and g are two functions in 1D, then:
– The spaces of real-valued functions defined on the
circle, disk, sphere, ball,… are real inner product
spaces.
If f and g are two functions defined on the circle, then:
∫
∞
∞−
= dxxgxfgf )()(,
∫=
π
θθθ
2
0
)()(, dgfgf

22. Algebra Review
Inner Product Spaces
If V1,V2⊂V, then V is the direct sum of subspaces V1,
V2, written V=V1⊕V2, if:
– Every vector v∈V can be written uniquely as:
for some vectors v1∈V1 and v2∈V2.
21 vvv +=

23. Algebra Review
Inner Product Spaces
Example:
If V is the vector space of 4-dimensional arrays, then
V is the direct sum of the vector spaces V1,V2⊂V
where:
– V1=(x1,x2,0,0)
– V2=(0,0,x3,x4)

24. Algebra Review
Orthogonal / Unitary Operators
If V is a real / complex inner product space, then a
linear map A:V→V is orthogonal / unitary if it
preserves the inner product:
〈v,w〉= 〈Av,Aw〉
for all v,w∈V.

25. Algebra Review
Orthogonal / Unitary Operators
Examples:
– If V is the space of real, two-dimensional, vectors and
A is any rotation or reflection, then A is orthogonal.
A=
v2
v1
A(v2)
A(v1)

26. Algebra Review
Orthogonal / Unitary Operators
Examples:
– If V is the space of real, three-dimensional, vectors
and A is any rotation or reflection, then A is
orthogonal.
A=

27. Algebra Review
Orthogonal / Unitary Operators
Examples:
– If V is the space of functions defined in 1D and A is
any translation, then A is orthogonal.
A=

28. Algebra Review
Orthogonal / Unitary Operators
Examples:
– If V is the space of functions defined on a circle and A
is any rotation or reflection, then A is orthogonal.
A=

29. Algebra Review
Orthogonal / Unitary Operators
Examples:
– If V is the space of functions defined on a sphere and
A is any rotation or reflection, then A is orthogonal.
A=

30. Outline:
• Algebra Review
• Representation Theory
– Orthogonal / Unitary Representations
– Irreducible Representations
– Why Do We Care?

31. Representation Theory
Orthogonal / Unitary Representation
An orthogonal / unitary representation of a group G
onto an inner product space V is a map Φ that sends
every element of G to an orthogonal / unitary
transformation, subject to the conditions:
1. Φ(0)v=v, for all v∈V, where 0 is the identity element.
2. Φ(gh)v=Φ(g) Φ(h)v

32. Representation Theory
Orthogonal / Unitary Representation
Examples:
– If G is any group and V is any vector space, then:
is an orthogonal / unitary representation.
– If G is the group of rotations and reflections and V is
any vector space, then:
is an orthogonal / unitary representation.
vvg =Φ )(
vgvg )det()( =Φ

33. Representation Theory
Orthogonal / Unitary Representation
Examples:
– If G is the group of nxn orthogonal / unitary
matrices, and V is the space of n-dimensional arrays,
then:
is an orthogonal / unitary representation.
( )vgvg =Φ )(

34. Representation Theory
Orthogonal / Unitary Representation
Examples:
– If G is the group of 2x2 rotation matrices, and V is
the vector space of 4-dimensional real / complex
arrays, then:
is an orthogonal / unitary representation.
( ) ( )),(),,(,,,)( 43214321 xxgxxgxxxxg =Φ

35. Representation Theory
Irreducible Representations
A representation Φ, of a group G onto a vector space
V is irreducible if cannot be broken up into smaller
representation spaces.
That is, if there exist W⊂V such that:
Φ(G)W⊂W
Then either W=V or W=∅.

36. Representation Theory
Irreducible Representations
If W⊂V is a sub-representation of G, and W⊥
is the
space of vectors perpendicular to W:
〈v,w〉=0
for all v∈W⊥
and w∈W, then V=W⊕W⊥
and W⊥
is also
a sub-representation of V.
For any g∈G, v∈W⊥
, and w∈W, we have:
So if a representation is reducible, it can be broken
up into the direct sum of two sub-representations.
( ) ( ) ( ) ( ) ( ) wvgwggvgwgv ,,,0 11
ΦΦΦΦ=Φ= −−

37. Representation Theory
Irreducible Representations
Examples:
– If G is any group and V is any vector space with
dimension larger than one, then:
is not an irreducible representation.
vvg =Φ )(

38. Representation Theory
Irreducible Representations
Examples:
– If G is the group of 2x2 rotation matrices, and V is
the vector space of 4-dimensional real / complex
arrays, then:
is not an irreducible representation since it maps the
space W=(x1,x2,0,0) back into itself.
( ) ( )),(),,(,,,)( 43214321 xxgxxgxxxxg =Φ

39. Representation Theory
Why do we care?

40. Representation Theory
Why we care
In shape matching we have to deal with the fact that
rotations do not change the shape of a model.
=

41. Representation Theory
Exhaustive Search
If vM is a spherical function representing model M
and vn is a spherical function representing model N,
we want to find the minimum over all rotations T of
the equation:
( )
( )NMNM
NMNM
vTvvv
vTvvvT
,2
),,(
22
2
−+=
−=D

42. Representation Theory
Exhaustive Search
If V is the space of spherical functions then we can
consider the representation of the group of rotations
on this space.
By decomposing V into a direct sum of its
irreducible representations, we get a better
framework for finding the best rotation.

43. Representation Theory
Exhaustive Search (Brute Force)
Suppose that {v1,…,vn} is some orthogonal basis for
V, then we can express the shape descriptors in terms
of this basis:
vM=a1v1+…+anvn
vN=b1v1+…+bnvn

44. Representation Theory
Exhaustive Search (Brute Force)
Then the dot-product of M and N at a rotation T is
equal to:
( )
( )
( )∑
∑∑
∑∑
=
==
==
=
=








=
n
ji
jiji
n
j
jj
n
i
ii
n
j
jj
n
i
iiNM
vTvba
vTbva
vbTvavTv
1,
11
11
,
,
,,

45. ( ) ( )∑=
=
n
ji
jijiNM vTvbavTv
1,
,,
Representation Theory
Exhaustive Search (Brute Force)
So that the nxn cross-multiplications are needed:
T(vn)
vM
v1
v2
vn
=
+
+
+
T(v1)
=
+
+
+
T(v2)
T(vN)
…
…

46. Representation Theory
Exhaustive Search (w/ Rep. Theory)
Now suppose that we can decompose V into a
collection of one-dimensional representations.
That is, there exists an orthogonal basis {w1,…,wn} of
functions such that T(wi)∈wiℂ for all rotations T and
hence:
〈wi,T(wj)〉=0 for all i≠j.

47. Representation Theory
Exhaustive Search (w/ Rep. Theory)
Then we can express the shape descriptors in terms
of this basis:
vM=α1w1+…+αnwn
vN=β1w1+…+βnwn

48. Representation Theory
Exhaustive Search (w/ Rep. Theory)
And the dot-product of M and N at a rotation T is
equal to:
( )
( )
( )
( )∑
∑
∑∑
∑∑
=
=
==
==
=
=
=








=
n
i
iiii
n
ji
jiji
n
j
jj
n
i
ii
n
j
jj
n
i
iiNM
wTw
wTw
wTw
wTwvTv
1
1,
11
11
,
,
,
,,
βα
βα
βα
βα

49. ( ) ( )∑=
=
n
i
iiiiNM wTwvTv
1
,, βα
Representation Theory
Exhaustive Search (w/ Rep. Theory)
So that only n multiplications are needed:
T(wn)
vM
w1
w2
wn
=
+
+
+
T(w1)
=
+
+
+
T(w2)
T(vN)
…
…