An introduction to Digital Image Processing as a continuation of a classic Digital Signal Processing course delivered at the University of Plymouth (2011)

1. Digital Image Processing
Athanasios Anastasiou
Signal Processing & Multimedia
Communications Research Group
UoP

2. Learning Objectives
• Get The Bigger Picture Of Digital Image
Processing And Its Applications
• Understand And Be Able To Carry Out Basic
Operations
– +,-,/,*, Filtering, Correlation, Image Transforms
– MATLAB (and other software)
• Extend What You Already Know From DSP
To Higher Dimensions

3. Topics
What Can You Do
With Digital Image Why Digital Image
Processing? Processing?
How Does A 2D
Fourier Spectrum
Look Like? What Is An Image?
Is There Aliasing In
What‟s This Thing Digital Image
Who Is Lenna Processing?
Called “Spatial
Soderberg?
Are There 3D Frequency”?
„Images‟? What Are The
Relationships
Transforms Used In Between DSP and
Digital Image DIP?
Are There 4D Processing?
Images?…How Do How Do You Filter
They Look Like? An Image?
What Is The Fourier
Transform For
What Is The Fourier
Images?
Transform For
Volumetric Data?

4. Why Digital Image Processing?
• Convenience, Practicality
• Capture
– Obtain The Representation Of A Scene
• Process
– Pre Process
– Extract Information
• Make Sense Of The Image / Scene
– Further Processing
• Classify, Modify, Combine,…
• Visualise
– Not As Simple As You May Think
– Transform Image
• Transmit / Store
– Data Format
– Compression

5. Why Digital Image Processing?
• Emergent Applications
– Machine Vision
– Augmented Reality
– (More) Surveillance
– Gaming
• …To The Limits Of Your Imagination And
Beyond (!)

6. Breadth & Depth Of D.I.P
Demonstrators
• Simultaneous Localisation And Mapping
– S.L.A.M.
• Merging And Making Sense Of Images On (A Ridiculously Large) Scale
– Microsoft PhotoSynth
– Giga & Peta Pixel Panoramic Photography
• Gigapixel Panoramas
• Petapixel Panorama
– Scene Completion Using Millions Of Photographs
– Scale Invariant Feature Transform
• (S.I.F.T & S.U.R.F)
• Augmented Reality
– Bluring The Divide Between The Real And The Virtual
– Tracking Facial Features
– Lip Sync A Virtual Character
– Recognise If Someone Is Bored By Their Facial Expression
• Make A Game To Respond To That
• Making Sense Of Images
– MS Kinect
– Eye Pet

7. Capturing Digital Images
R
R Focusing R E Control, Acquisition
R O Transducer
Apparatus & Digitisation
•Mechanical •Photoelectric •Exposure!
•Optical •Valve
•Electromechanical •CCD D
•Electronic •Photodiode
•Photoresistor
•PiezoElectric
•Inductive
•Capacitive
•Other Digital Image
•Medium / Format
•Film
•Paper
•Digital
•R: Some Form Of Radiation
•O: Object
•E: Electric Charge
•D: Digital Signals

8. The Concept Of Exposure
• Controls How Much Radiation Is Captured
H E t
• E: Radiation Flux
– Depends On The Focusing Mechanism & Transducer
• Spectral Response! / Aperture!
– Units:
• Energy / Surface (Watts / m2)
• t: Time
– Unit: Seconds
• Remarks
– Exposure Creates Contrast (!!!)
– Exposure Is A Product (!)
• When E Goes Down, t Must Go Up To Maintain The Same H

9. Photography
Visible Light
Flash!!!
R
CHEESE!
R Focusing R E Control, Acquisition
O Transducer
Apparatus & Digitisation
Light Is Reflected Lens System CCD
Or Emitted By An D
Object
Digital Image
Image Credit: http://www.jiscdigitalmedia.ac.uk/images/slr02.jpg
JPG

10. Radar / Sonar
Radio Frequencies
R
R Focusing R E Control, Acquisition
O Transducer
Apparatus & Digitisation
Sound Waves Are Electronic / Inductive
Reflected By An Electromechanical
D
Object
Digital Image
HDF, DEM, GRB & Others
Image Credits: Wikipedia, Microwave Journal, Defense Industry Daily, Met Office

11. Radiography
(X-Ray, CT)
What‟s Up
Doc?
R O
R Focusing R Transducer
E Control, Acquisition
Apparatus & Digitisation
X-Rays Absorbed Mechanical (*) X-Ray Sensitive Coating
By Object Secondary Radiation D
CCD
Digital Image
*: CT Stands For Computed Tomography
Image Credits: apex.it, aapm.org, Wikipedia, Medscape.com
DICOM

12. Nuclear Imaging
(SPECT, PET)
SPECT / γ - Camera Positron Emission Tomography
R R Focusing R E Control, Acquisition
RO Transducer
R Apparatus & Digitisation
Gamma Rays Mechanical Scintillator
Emitted By The Object Photomultiplier
(How Did It Get In There?) D
“You Will Feel A
Tiny Prickle Now…”
Digital Image
Image Credits: Utah.edu, msha.com, PET ISLLC DICOM

13. Ultrasonography
(Ultrasound)
UltraSound
R
R Focusing R E Control, Acquisition
O Transducer
Apparatus & Digitisation
Sound Waves Are Mechanical / Piezoelectric Crystal
Reflected By An Electronic /
D
Object Electromechanical
Digital Image
DICOM
Image Credits: Wrightwoodmedical.com, zhweichao.com, UoC MIG, examiner.com|

14. Magnetic Resonance Imaging
RF Pulses
R
R Focusing R E Control, Acquisition
O Transducer
Apparatus & Digitisation
Electronic Inductive
D
Paramagnetic Atom
Precession To Initial Digital Image
Spin
Image Credits:
sarctrials.com, pedimaging.com, UoL, Wikimedia.co DICOM
m

15. What Is A Digital Image?
• How Does Radiation Varies Across Space?
– Samples Across Space
• A Two Dimensional Signal
– Higher Dimensions Are Possible (3D, 4D)
• How Does A Quantity Varies Across Volume?
• How Does A Quantity Varies Across Volume & Time?
• A Two Dimensional Projection Of A 3D Scene (!)
• An Array
– X(i,j)
0,1,1,0,0,0
0,1,1,0,0,0
0,1,1,0,0,0
0,1,1,1,1,0
0,1,1,1,1,0

16. Space!!!
• Spatial Frequency
– Large Objects
• Low Frequencies
• Form
– Fundamental Shape
– Small Objects
• High Frequencies
• Details!
– Edges!!!
• Spatial Resolution
– What Is The Smallest Object I Can Observe?
– Is It Uniform???

17. Characteristics Of Digital Images
• Samples
– Picture Elements
• Pixels
• Resolution
– Pixel
• 512x512 pixels
• Dimensions
– Spatial
• 50m * 50m
• 5mm * 5mm
• Dynamic Range
– Color Depth
• “Number Of Colors”
– Contrast

18. DSP VS DIP
• x(t) • x(i,j)
– Amplitude – Intensity
• • “Brightness”
– Frequency – Spatial Frequency
– Sampling Frequency – Resolution
• • Nyquist Theorem?
– Phase – Orientation!!!

19. Aliasing
• A Byproduct Of Sampling
• Images Sample Space
Constrain The Spatial Bandwidth To Fs/2

20. Interlude
Aliasing In Video I(I,j,t)
• How Many RPM? What Direction?
– Wheel
– Rotor
– Prop
• How Would These Questions Translate In
DSP Terms?

21. Basic Operations
• Add, Subtract (“Mix” Two Images)
• Multiply, Divide (Modulation)
• What About Negative Or Over The Range
Values???

22. “Out Of Range” Values
The Use Of Windows
• What Is A „Window‟?
– What Is An Image Histogram?
• Why Do We Need One?
• How Does It Look Like?
• How Is It Used?
• Demo

23. Convolution & Correlation
Nh
y ( m) h(i ) x(m i )
i 0
N hX N hY
y(m, n) h(i, j ) x(m i, n j)
i 0 j 0
*: Discrete Formulas

24. 2D Filters & Image Filtering
0.4
0.35
0.3
0.25
0.2
Amplitude
0.15
0.1
0.05
0
-0.05
0 10 20 30 40 50 60 70
Sample / Time
Image Filters (2D) Are An Extension Of Signal Filters (1D)
Think In Terms Of Spatial Frequency

25. 2D Filters & Image Filtering
1.2
0.12
1
0.1
0.08 0.8
0.06 0.6
0.04 0.4
70
0.02
0.2 70
0 60
0
60
-0.02
50 -0.2
70 50
70
60 40
60 40
50
30 50
30
40
40
30 20 20
30
LP 20
10
0
10
HP 20
10
0
10
0 0
0.6
0.6
0.5 0.5
0.4 0.4
0.3 0.3
0.2
0.2
0.1
0.1 70
70
0
0
60
-0.1 60
-0.1
50 -0.2 50
70
70
60 40 40
60
50
30 50
30
40
40
BP 30
20
10
10
20
BR 30
20
10
10
20
0 0 0
0

26. 2D Filters & Image Filtering
How Does A Filtered Image Look Like?

27. 2D Filters & Image Filtering
Meet Lenna
Lena Soderberg : Model
Playmate 1972: Picture Includes Only The Interesting Parts
Photography :Dwight Hooker

28. 2D Filters & Image Filtering
1.2
0.12
1
0.1
0.08 0.8
0.06 0.6
0.04 0.4
70
0.02
0.2 70
0 60
0
60
-0.02
50 -0.2
70 50
70
60 40
60 40
50
30 50
30
40
40
30 20 20
30
LP 20
10
0
10
HP 20
10
0
10
0 0
0.6
0.6
0.5 0.5
0.4 0.4
0.3 0.3
0.2
0.2
0.1
0.1 70
70
0
0
60
-0.1 60
-0.1
50 -0.2 50
70
70
60 40 40
60
50
30 50
30
40
40
BP 30
20
10
10
20
BR 30
20
10
10
20
0 0 0
0

29. 2D Filters & Image Filtering
How Does It Look Like?
50 50
100 100
150 150
200 200
250 250
300 300
350 350
400 400
450 450
500 500
550 550
50 100 150 200 250 300 350 400 450 500 550 50 100 150 200 250 300 350 400 450 500 550
50 50
100 100
150 150
200 200
250 250
300 300
What Is The DSP Name 350 350
For That Frame Around 400 400
The Filtered Images? 450 450
500 500
550 550
50 100 150 200 250 300 350 400 450 500 550 50 100 150 200 250 300 350 400 450 500 550

30. Applications Of Filtering In
Digital Signal Processing
• Art
– Image Effects
• Key Preprocessing Stage
– „Noise‟ Reduction
– Image Pyramids
• Feature Extraction
– Where Are The Edges Of An Object?

31. Convolution & Correlation
Nh
y ( m) h(i ) x(m i )
i 0
A Simple Metric Of Similarity
Why Does It Work?
X (m), Y (m) with m 0 N
N 1
1 cov X , Y X X Y Y
X X m
m 0
N 1
2 1 2 1: Alike
(X ) X m X
N m 0 cov X , Y
cor X , Y 0: No Rel
X 2
X X Y
-1: Rev Rel

32. Applications Of Correlation In
Digital Image Processing
“Dude! Where’s My Airplane?”
Correlation (And Satellite Imaging)Tells You!
N hX N hY
y(m, n) h(i, j ) x(m i, n j)
i 0 j 0
x : An Image
h : Commonly Referred To As A Template (What To Look For)
y(m,n) : How “Similar” Is The Patch Centred Around m,n With h??
Similar As In: “Locally The Samples Go Over And Under
The Mean Value (More Or Less) Frequently”

33. Correlation In Digital Image
Processing
Dude, Where’s My Airplane?
Find In

34. Correlation In Digital Image
Processing
Sir, We Have Three Potential Matches
100
200
300
400
500
600
700
800
900
1000
200 400 600 800 1000 1200 1400 1600
The Maxima Of y(m) Are The Centre Locations Of Positive Matches.
Why Does It Find More Than One Targets?

35. Correlation In Digital Image
Processing
A More Down To Earth Example…
(ROITracker Demo)

36. Transforms In
Digital Image Processing
• K(i,j) = F(x(i,j) * kernel(i,j))
– K(i,j) = F(Rows) + F(Columns)
• Fourier Transform
– Decomposition Into Spatial Frequencies (!!!)
• Some Spatial Frequency
– Shifting The Origin (Optionally)
• Wavelet Transform
– Decomposition Into Elementary „Tiles‟
• The 2D Spectrum
• The 2D Filter

37. The Discrete Fourier Transform
N 1 2 ikn
1D DFT  N
X k xn e
n 0
N X 1 NY 1 kn lm
2 i
N X NY
2D DFT  X k, l x n, m e
n 0 m 0
l
Making Sense f k ,l k 2 l 2 , d k ,l tan 1
Of The 2D k
Spectrum  2 2
A k,l X r k,l X i k,l
1 X i (k , l )
P(k , l ) tan
X r k,l
Note: A logarithmic transformation is usually applied to f or A because of the
relative magnitude distribution

38. The Image Spectrum
(A Very) Representative Example
50
100
150
200
250
300
350
400
450
500
50 100 150 200 250 300 350 400 450 500
What Is The
Fundamental Function
In This Spectrum?  12
10
140
120
8
6
100
4
2 80
0
60
140
120
100 40
80
60 20
40
20
0
0

39. The Image Spectrum
(Another) Representative Example
50
50
100
100
150 150
200 200
250 250
300 300
350 350
400 400
450 450
500
500
50 100 150 200 250 300 350 400 450 500
50 100 150 200 250 300 350 400 450 500
What Is The 0.07
Fundamental Function 0.06
0.05
In This Spectrum?  0.04
0.03
0.02
0.01
140
0
120
140
120 100
100 80
80
60
60
40
40
20
20
0 0

40. The Image Spectrum
Some More Examples
50
100
150
200
250
300
350
400
450
500
50 100 150 200 250 300 350 400 450 500
100
200
300
400
500
600
700
100 200 300 400 500 600 700

41. The Image Spectrum
A Realistic Example
50
100
150
200
250
300
350
400
50 100 150 200 250 300 350 400

42. References