Using high speed camera to catch raindrops and do analysis on them to predict rainrate

1. An Image-based Disdrometer
Verification and Raindrop Analysis
影像式雨滴譜儀系統驗證與雨滴分析
指導教授: 鐘太郎 教授
學生: 102061539 黃政翰

2. Outline
 Introduction
 Raindrop Analysis Theories
 Structure of the Proposed Disdrometer and Software Algorithm
 Result of Experiments
 Conclusion

3. Introduction

4. Precipitation Observation
 Weather forecasting must tell the information of precipitation
 Behavior of rainfall phenomena are due to local and sudden precipitation
 Predicting rainfall intensity
 Advance preparation can prevent potential disasters from happening

5. Observing Systems
 Primitive method
 Collect drops by a box with dye paper
 Wasting time and low efficiency
 Radar and Satellite sensors
 Collect data in large region
 Not accurate enough in analyzing rather small region
 Disdrometers
 Analyze raindrop particles

6. Why Disdrometer?
 Increase the accuracy of the raining condition in a small region
 Increase the accuracy of the radar measurement
 Raindrop feature analysis can be used in
 Air traffic control
 Scientific examination
 Weather observation system

7. Types of Disdrometers
 Joss-Waldvogel Disdrometer
 Acoustic Disdrometer
 Optical Particle Size Velocity Disdrometer
 2-D Video Disdrometer
 Image-based Disdrometer

8. Joss-Waldvogel Disdrometer
 Discriminate drop size by receiving the impact kinetic energy
 It cannot determine the drop shape
 Low sensitivity in drizzling and heavy rain

9. Acoustic Disdrometer
 Drops hitting on the sensor induces sonic wave
 The piezoelectric sensor can measure the rainfall intensity
 It does not provide raindrop distribution data
 Wind influences the measurement easily

10. Optical Particle Size Velocity Disdrometer
 A row of laser beam points to a sensor
 Measure drop size by calculating the duration of light extinction
 2 dimension is possible; speed measurement is available
 Drop mismatch makes errors

11. 2-D Video Disdrometer
 Using 2 line-scan cameras to measure size and shape
 Velocity can be calculated by traveling time and distance between two frames
 Slanted particle falling path makes image distortion and lead to errors

12. Image-based Disdrometer
 Use CCD camera to capture raindrops
 Double exposures in each frame
 Too many drops in one image influences matching accuracy a lot

13. Disdrometer Comparison
Disdrometer Type Measuring Mechanics Advantages Disadvantages
Joss-Waldvogel Falling impact
Good performace in
small size variation
Poor at measuring
drops that are too
small or too large
Acoustic Inducing sonic wave
Good at monitoring
large size drops
Wind influence easily
OTT Parsivel
Make drops falling through
a laser beam
With good accuracy in
measuring drop size
Drop mismatch and
near drops makes
error
2D Image
Use 2 line-scan camera to
measure size and shape
Size, shape and
velocity measurement
are available
Slanted falling drops
are distorted
Image-based
use CCD camera to
capture images and find
parameters by image
processing
Low cost of recording
and flexible system
setting
Camera frame rate and
resolution influence the
result

14. Raindrop Analysis Theories

15. Drop Size Distribution (DSD)
 Marshall and Palmer (1948) [11]
 Announced that DSD can be described by an exponential distribution
D
D eNN 
 0
4
0 08.0 
 cmN
121.0
41 
 cmR

16. Drop Size Distribution (DSD)
 The relationship comes up with errors in small drops
 Ulbrich (1983) [12]-> Gamma function
 are the parameters
D
D eDNN 
 
0
,, 0N

17. Drop Size Distribution (DSD)
 The relationship comes up with errors in small drops
 Feingold and Levin (1986) [13]-> Lognormal function
 are the parametersTg ND ,,

18. DSD is related to
 DSD has some relations with rain rate and rain types
 Kozu and Nakamura (1991) [14]
 DSD is related to reflectivity factor measured by radar
 Doviak and Zrnic (1984) [15]
 DSD can calibrate and increase the accuracy of the radar
 Joss and Waldvogel (1969) [16]

19. Drop Velocity
 Gunn and Kinzer (1949) [17]
 Experiments of raindrop terminal velocities through stagnant air
 Battan (1964) [18]
 Experiments in thunderstorm
 Foote and DuToit (1969) [19], Beard (1976) [20]
 Experiments in different air density

20. Drop Velocity
Different velocity curve under different conditions Different velocity curve under different air density
Atlas et al. [5]
Mitchell [21]
Beard [20]

21. Structure of the Proposed Disdrometer and
Software Algorithm

22. System Structure
 Optical Unit
 Light source (Part A)
 Image Acquisition Unit
 Lens (Part B)
 CCD camera (Part C)
 Data Processing Unit
 Processing Algorithm (Part D)
System Structure Diagram

23. System Structure
System Structure Photograph

24. Optical Unit
 Viswell HBL-100
 Uniform blue LED light source
 Bring up light intensity
 Enhance image contrast

25. Optical Unit
 Relative position between light source and camera
 It depends on the lens used in the system
 Using a telecentric lens
 We can only put the light source facing to the camera

26. Optical Unit
Light source on the side Light source facing to the camera

27. Optical Unit
 Proper adjustment of light intensity
Light Intensity 50 52.5 55 57.5
Contrast under
0 degree light
0.241 0.232 0.053 0

28. Image Acquisition Unit: Camera
 CCD Camera: Pylon Basler Aca640-90gm
 Monochrome, adjustable gain and exposure time
 High frame rate (90 fps in stable)
 659 pixels * 494 pixels
 SDK is provided

29. Image Acquisition Unit: Lens
 Lens: OPTO Engineering TC13064
 Telecentric lens
 Minimizes blur effect
 Long depth of field

30. Image Acquisition Unit: Lens
 Compare with the old one: Computar M1214-MP2
TC13064 M1214-MP2
FOV 6.5cm*4.8cm 5cm*3.7cm
DoF about 15cm about 3~5cm
Focus Fixed Manual adjustable
Iris Fixed Manual adjustable
Out of focus
Blurriness
Slight Severe

31. Image Acquisition Unit: Lens
Image taken by M1214-MP2 Image taken by TC13064

32. Data Processing Unit
 Software platform: Visual Studio 2012, using Visual C++
 Combined with: OpenCV 2.4.9, Pylon 4 SDK, Matlab2010

33. Data Processing Unit:
Camera Parameter Setting
 Done before taking every set of images
 Critical parameters
 Image size
 Exposure time
 Gain
 Recording duration

34. Data Processing Unit:
Camera Parameter Setting
Set the duration time of recording
or the number of images taken
Set the exposure time
Set the gain
Set the image size
Start taking images
Start the program

35. Data Processing Unit:
Drop Extraction and Analysis

36. Drop Extraction and Analysis:
Make a Background
 Background calculation: average all the frames
 is the ith frame, is the number of taken images
k
if
Bg
k
i


][
][if k
k
average

37. Drop Extraction and Analysis:
Make a Background
Original image Image without background

38. Drop Extraction and Analysis:
Image Binarization
 Median Filter 5*5 -> reduce noise
 Choose proper threshold method
 Depend on the image we get
 Max Entropy, Iterative, Otsu, Region Growing, Level Set had been tried
 100us exposure time image: Max Entropy Thresholding
 2000us exposure time image: Iterative Thresholding

39. Drop Extraction and Analysis:
Max Entropy Thresholding
 The threshold determines by maximizing the entropy of foreground and
background
 is the gray-level probability density function for the image
)
)(
)(
log(
)(
)(255
1 Tq
ip
Tq
ip
H
fTi f
f  
)
)(
)(
log(
)(
)(
0 Tq
ip
Tq
ip
H
b
T
i b
b  
)(ip

40. Drop Extraction and Analysis:
Max Entropy Thresholding
 and are the probabilities that a given pixel belongs to foreground or
background when the threshold is


255
1
)()(
Ti
f ipTq


T
i
b ipTq
0
)()(
)(Tqf )(Tqb
T
)max( bf HHT 

41. Drop Extraction and Analysis:
Iterative Thresholding
 An initial threshold is chosen, typically the average intensity of the image
 Mean gray value of foreground and background are calculated
 is the gray-level probability density function for the image
T


255
1
)(
Ti
f iip


T
i
b iip
0
)(
2
bf
T
 

)(ip

42. Drop Extraction and Analysis:
Binary images after thresholding
100us binary image 2000us binary image

43. Data Processing Unit:
Drop Extraction and Analysis

44. Drop Extraction and Analysis:
Contour Finding
 Binary images have high contrast
 Easy to determine edges

45. Drop Extraction and Analysis:
Ellipse Fitting

46. Drop Extraction and Analysis:
Minimal Bounding Rectangle

47. Drop Extraction and Analysis:
Unsuitable Objects Elimination
 Out of bound elimination
 Any contour touches the border are eliminated
 100us images
 Axis ratio 0.4~1.2 -> treated as raindrops
 2000us images
 Eliminate if the width is larger than height

48. Drop Extraction and Analysis:
Size Calculation
1.01.0  PA
2
1
)(2

A
Dm 
mm1.0
mm1.0
P

49. Drop Extraction and Analysis:
Velocity Calculation
2
2
w
hd 
tdv / h
w
2
w
2
w

50. Drop Extraction and Analysis:
DSD calculation
 According to Liu et al. (2013)
 : DSD
 : number of drops of each bin
 : bin interval (1mm)
 : Sampling volume of the drop-falling space
)(DN )( 13 
mmm
)(DNum
dD
)(15.0 3
mWHV 
m
m
m
dDV
DNum
DN


Pr
)(
)(
framemymx
osuremyosuremx
tvv
tvhHtvwW



)()(
Pr expexp

51. Drop Extraction and Analysis:
Rain Rate Calculation
 : velocity of the measured diameter



0
3
)(
6
mmmm dDvDDNR

mv

52. Result of Experiments

53. Marble Experiments
 Throwing marbles from 1mm to 5mm separately
 100us exposure time, 300 gain, maximum light
 3 seconds duration, 270 frames, as one set of images

54. Marble Experiments
Measured diameter vs Calculated Diameter Measured diameter vs Measured Area

55. Marble Experiments
Theoretical Value Average Std Min Max Error
diameter(100us) (mm) 0.25 0.3110 0.0902 0.0707 0.4998 24.3815
area(100us) (mm
2
) 0.0491 0.1317 0.0511 0.055 0.215 168.1602
Theoretical Value Average Std Min Max Error
diameter(100us) (mm) 1 1.0219 0.2217 0.5 1.4948 2.1872
area(100us) (mm
2
) 0.785 0.7369 0.4036 0.09 1.9 6.1687
Theoretical Value Average Std Min Max Error
diameter(100us) (mm) 2 2.0560 0.2808 1.5 2.4749 2.8018
area(100us) (mm
2
) 3.142 3.2074 1.195 0.535 5.305 2.0950
Theoretical Value Average Std Min Max Error
diameter(100us) (mm) 3 3.0887 0.2514 2.5 3.4883 2.9583
area(100us) (mm
2
) 7.069 7.4855 1.5203 1.62 10.735 5.8988
Theoretical Value Average Std Min Max Error
diameter(100us) (mm) 4 4.0374 0.2346 3.5 4.4721 0.9339
area(100us) (mm
2
) 12.566 12.4727 2.2965 4.225 17.085 0.7456
Theoretical Value Average Std Min Max Error
diameter(100us) (mm) 5 4.9206 0.3012 4.5 5.4447 1.5886
area(100us) (mm
2
) 19.635 18.1703 3.5536 5.985 24.27 7.4594

56. Marble Experiments
 Overlapping leads to the presence of outliers
 Larger marbles have higher error
 Axis ratio recognition gives larger range to be distinguished in large size objects
 Small marbles error
 Some noises are remained after thresholding

57. Water Sprinkling Experiments
 Spread water by sprinkler
 100us exposure time, 300 gain, maximum light
 2000us exposure time, 300 gain, half light
 5 seconds duration, 450 frames, as one set of images

58. Water Sprinkling Experiments
Measured diameter vs Calculated Diameter Measured diameter vs Measured Area

59. Water Sprinkling Experiments
Axis ratio distribution

60. Water Sprinkling Experiments
Diameter vs Speed Canting angle histogram

61. Water Sprinkling Experiments
Theoretical Value Average Std Min Max Error
diameter(2000us) 0.25 0.324 0.096 0.045 0.500 29.7393
speed(2000us) 0.7847 0.643 0.530 0.069 2.705 18.0382
diameter(100us) 0.25 0.456 0.045 0.300 0.499 82.2501
area(100us) 0.0491 0.136 0.045 0.005 0.235 177.8743
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 1 0.9795 0.2773 0.5 1.4997 2.0474
speed(2000us) (m/s) 3.9972 2.1910 0.8306 0.05 8.6185 45.1867
diameter(100us) (mm) 1 0.7777 0.1927 0.5 1.4977 22.2321
area(100us) (mm2
) 0.7854 0.4596 0.2566 0.02 1.795 41.4878
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 2 1.8524 0.2785 1.5 2.4989 7.3819
speed(2000us) (m/s) 6.5477 3.4311 1.1124 0.1 10.454 47.5979
diameter(100us) (mm) 2 1.7033 0.2171 1.5 2.4660 14.8353
area(100us) (mm2
) 3.1416 1.6046 0.5596 0.475 3.685 48.9244
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 3 2.8973 0.2750 2.5 3.4985 3.4220
speed(2000us) (m/s) 7.9474 4.3873 1.4764 0.75 10.8093 44.7961
diameter(100us) (mm) 3 2.8418 0.0189 2.82843 2.8552 5.2725
area(100us) (mm
2
) 7.0686 2.97 0.9334 2.31 3.63 57.9831
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 4 3.8909 0.2694 3.5 4.4933 2.7287
speed(2000us) (m/s) 8.7156 5.1360 1.9515 0.75 12.8701 41.0717
diameter(100us) (mm) 4 N/A N/A N/A N/A N/A
area(100us) (mm2
) 12.5664 N/A N/A N/A N/A N/A
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 5 4.9799 0.2777 4.5 5.4811 0.4011
speed(2000us) (m/s) 9.1372 5.9424 2.1716 1.4 11.4378 34.9649
diameter(100us) (mm) 5 N/A N/A N/A N/A N/A
area(100us) (mm2
) 19.6350 N/A N/A N/A N/A N/A
Average Std Min Max
canting angle -35.6758 26.6744 -180 0

62. Water Sprinkling Experiments
 Larger error of speed difference in larger drop size
 Overlapping issue
 Sprinkled water are not in terminate velocity
 Few drops are grabbed in this size interval

63. Raining Experiments
 Real raining condition at 17:00, 27 Aug 2015 at Hsinchu, Taiwan
 5 seconds duration, 450 frames, as one set of images, 30 seconds in total
 2700 images taken in 100us and 2000us respectively

64. Raining Experiments
Histogram of size distribution Axis ratio distribution

65. Raining Experiments
Measured diameter vs Calculated Diameter Measured diameter vs Measured Area

66. Raining Experiments
Histogram of size distribution Histogram of canting angle distribution

67. Raining Experiments
Diameter vs Speed

68. Raining Experiments
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 0.25 0.374864 0.108827 0.14 0.497947 49.9457
speed(2000us) (m/s) 0.7847 1.15072 0.598629 0.28 3.2 46.6446
diameter(100us) (mm) 0.25 0.441055 0.065593 0.31305 0.494975 76.4218
area(100us) (mm2
) 0.0491 0.137188 0.060991 0.025 0.215 179.4043
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 1 0.9394 0.2447 0.5 1.4999 6.0578
speed(2000us) (m/s) 3.9972 2.6945 0.8629 1.05 5.3062 32.5902
diameter(100us) (mm) 1 0.8658 0.2196 0.5 1.4863 13.4154
area(100us) (mm2
) 0.7854 0.6174 0.3419 0.055 1.87 21.3847
Theoretical Value Average Std Min Max Error
diameter(2000us) (mm) 2 1.7479 0.1486 1.5402 2.0803 12.6055
speed(2000us) (m/s) 6.5477 5.3268 0.5195 4.0784 6.0691 18.6465
diameter(100us) (mm) 2 1.9637 0.1418 1.5 2.2030 1.8129
area(100us) (mm
2
) 3.1416 3.0464 0.3600 1.695 3.695 3.0294
Average Std Min Max
canting angle -10.276 29.3070 -180 -0.273

69. Raining Experiments
 Small raindrops dominant
 Small canting angle -> almost no wind
 Speed are lower than theoretical value
 Image processing leads to the error
 Raining condition difference

70. Raining Experiments:
Image Processing Error
 If there is one-pixel error in width
 An 1mm raindrop is in 10% error
 Theoretical velocity is in 9% error

71. Raining Experiments:
Image Processing Error

72. Raining Experiments
 According to the statistical data of Central Weather Bureau
 Rain rate = 0.5 mm/h
 Wind speed = 0.3 m/s
 The calculated data
 Rain rate = 0.5721 mm/h, Error = 14.42%
 Wind speed = 0.2095 m/s, Error = 30.01%

73. Conclusion

74. Conclusion
 We have built an image-based disdrometer:
 Low cost and Easy-assembling
 Results are in the tendency of the empirical formula
 Keep good performance in windy situation
 Three kinds of experiments were done to verify the system
 The structure and processing procedures are feasible
 Thresholding calibration is needed
 Calculated rain rate is in the error around 15%

75. Future Work
 Still need further calibration in every set of images to increase measurement
accuracy
 Increasing FOV or frame rate to increase capture probability
 Improve contrast in field experiment
 Overlapping issue -> Set 2 CCD camera to make 3D images