Recent advancement in sensor technology allows very high spatial resolution along with multiple spectral bands. There are many studies, which highlight that Object Based Image Analysis(OBIA) is more accurate than pixel-based classification for high resolution(< 2m) imagery. Image segmentation is a crucial step for OBIA and it is a very formidable task to estimate optimal parameters for segmentation as it does not have any unique solution. In this paper, we have studied different segmentation algorithms (both mono-scale and multi-scale) for different terrain categories and showed how the segmented output depends on upon various parameters. Later, we have introduced a novel method to estimate optimal segmentation parameters. The main objectives of this study are to highlight the effectiveness of presently available segmentation techniques on very high-resolution satellite data and to automate segmentation process. Pre-estimation of segmentation parameter is more practical and efficient in OBIA. Assessment of segmentation algorithms and estimation of segmentation parameters are examined based on the very high-resolution multi-spectral WorldView-3(0.3m, PAN sharpened) data.

1. Comparison of Segmentation Algorithms and
Estimation of Optimal Segmentation
Parameters for Very High Resolution
Satellite Imagery
P. R. Sarkar1 P. Srinivas2 J .Srinivasulu3 D. Sita4
1Department of Avionics
Indian Institute of Space Science and Technology
2Head,GRD
3Scientist- ‘F’
4Scientist - ’E’
Advanced Data Processing Research Institute

2. Outline
New paradigm in remote sensing, GEOBIA
Image Segmentation, the first and critical step
Segmentation algorithms
Objective of this Study
Preparatory work
Study area & Quality assessment technique
Methodologies
Selection of optimal segmentation parameters
Estimation of segmentation parameters
Existing methods
Proposed approach
Results
Evaluation of segmentation algorithms
Evaluation of Automatic Estimation of Segmentation Parameter Technique
Conclusions

3. Why GEOBIA?
Geographic Object Based Image Analysis
1. The latest breed of very high resolution (VHR) commercial satellite imagery
(<2.0m) imagery, which has dramatically increased new possibilities in
remote sensing and cartographic applications.
2. Pixel based analysis neglects the spatial information of very high resolution
images.
3. Several study demonstrated that geographic object based classifier is a
significantly better approach than the classical pixel classifier.
4. A dramatic increase in availability of affordable powerful
computing systems.
5. Need of the multi-scale approaches in the monitoring, modelling and
management of the environment. GEOBIA is the best choice for this.

4. Image Segmentation
Segmentation is a process of completely partitioning an image into
non-overlapping regions based on homogeneity parameters or on
differentiation of heterogeneity parameters.
Mathematically, it can be defined as follows: if F is the set of all pixels
and P( ) is a homogeneity defined on group of connected pixels, then
segmentation is a partitioning of the set F into a set of connected
regions (S1, S2, ..., Sn) such that:

5. Why Segmentation?
1. In GEOBIA, we treat a remotely sensed image as union of meaningful
image objects. To extract each image objects, we need to use
segmentation techniques.
2. As discussed in slide: 3, considering an image as objects gives better
classification accuracy and also opportunity to use spatial information.

6. Segmentation Algorithms
1. Mean-shift Segmentation
2. Multi-resolution Segmentation
3. k-means Segmentation
4. Watershed Segmentation

7. Mean-shift segmentation
Mean-shift algorithm was introduced by Fukunaga and Hostetler in
1975. It is a versatile, non-parametric density gradient estimation for
mode finding or clustering procedure.
Idea: For each data point, mean shift defines a window around it and
computes the mean of the data point. Then it shifts the center of the
window to the mean and repeats the algorithm till it converges. After
each iteration, we can consider that the window shifts to a more
denser region of the dataset.

8. Mean-shift segmentation
At the high level, we can specify Mean Shift as follows:
1. Fix a window around each data point.
2. Compute the mean of data within the window.
3. Shift the window to the mean and repeat till convergence

9. Mean-shift segmentation

10. Mean-shift segmentation

12. Multi-resolution segmentation
1. It is a bottom-up region merging technique, which starts from one-pixel
object.
2. Throughout this clustering process, the underlying optimisation
procedure minimizes the weighted heterogeneity nh in the image objects
where,
n is the number of the pixels of image segment
h is the definition of heterogeneity measure.
3. The increment of heterogeneity (f) is calculated before the
merging of two adjacent object happens.
4. No further fusion takes place when the increment in
heterogeneity exceeds a threshold or scale.

13. Multi-resolution segmentation

14. k-means segmentation
This algorithm has been the most popular unsupervised learning
algorithm which is widely used for image segmentation.
The k-means clustering algorithm is as follows:
1. Initialize the cluster centroids µ1, µ2, ..., µk randomly.
2. Repeat till convergence

15. Watershed segmentation
1. The watershed transform is a region based segmentation approach.
2. Imagine the landscape is being immersed in a lake, with holes
pierced in topical minima. The process of filling water is stopped
when water reaches the highest peak in the landscape. This process
finally gives a portioned region or basins separated by dams also
called watershed lines.
3. Flow diagram of this algorithm is given in the next slide.

16. Watershed segmentation

17. Objective of this Study

18. Objective of this study
1. Study of various segmentation algorithms and deciding which
algorithm works better than the existing techniques for different
h-resolution terrain surfaces.
2. Comparing the quality of the segmentation algorithms for very high
spatial multi-spectral remotely sensed satellite imagery.
3. Automatic estimation of optimal parameter for segmentation
algorithms

19. Study Area & Quality Assessment Technique

20. Study area
The study area analysed for this research downloaded from Digitalglobe website.
Figure: Main
study area with
16384 × 16384
resolution

21. Regions of study
Rural area
Urban area Forest area
Residential area

22. Quality assessment technique
1. Visual assessment is taken into account for quality checking of
segmentation techniques.
2. In this paper, we decided upon visual assessment on the human eye
is considered as strong and experienced source for evaluation of
segmentation techniques

23. Selection of the Optimal Segmentation Parameters

24. Segmentation algorithms and parameters in ORFEO
Mean-shift environment Watershed environment

25. Segmentation algorithm and parameters in eCogniton
Parameters in eCognition

26. Segmentation algorithms and parameters in Matlab
1. Mean-shift algorithm has only one bandwidth parameter.
2. k-means has only class parameter.
3. Watershed algorithm does not have any parameter.

27. Parameter selection for mean-shift segmentation
We used spatial profile to select optimal parameters
Spatial profile in horizontal axis

28. Spatial profile in vertical axis

29. 1. From these spatial profiles we can notice peaks in all bands which
indicates transition between two different classes. The range
between the pixel positions at left side maximum value and right
side maximum value, is taken as range radius.
2. Spatial radius is taken by a trial-and-error method.
3. The other three parameters are taken as constants for all
the images under consideration.
4. The optimal parameters are chosen based on visual assessments.
The windowing method is described in the next slide.

30. Parameter selection at different window size

31. Optimal output of mean-shift
Parameter selection at 761 X 577 Parameter selection at 780 x 577

32. Optimal output of multi-resolution
WinSize =
128 X 128
WinSize =
352 X 352
WinSize =
468 X 468
WinSize =
212 X 212

33. Parameter selection for k-means and watershed segmentation
k-means:
1. The only parameter here is class. Selection of proper class is not difficult
when the test image is small, but it produce slots of ambiguity when it
comes for big remote images.
2. One approach to learn the class is to use Elbows method.
Watershed segmentation:
The two parameters for watershed segmentation are chosen based on
visual inspection.

34. Estimation of the Optimal Segmentation Parameters

35. Existing methods
Post-estimation of segmentation parameters:
1. Kim et. al has used the concept of local variance to determine optimal
segmentation parameters of alliance-level forest classification.
2. Dragut et. al used local variance in image object level and extended it into multi-
scale analysis based on single layer and later in multiple layers.
3. Zhao et. al has used the rate of change of local variance which is similar to the ESP
tool, to estimate the optimal parameter.
4. Emary et. al used a similarity measure between two image objects in set of
successive scales.
5. Karl and Maurer et. Al have used semi-variogram based spatial dependency
prediction.
Pre-estimation of segmentation parameters:
Ming et. al applies spatial statistics to select optimal parameters for mean-
shift segmentation algorithm.

36. Proposed Approach

37. Input features extraction
Calculation of spatial statistics before feeding in BPNN

38. Training of the BPNN
Training BPNN for estimating parameters

40. Results

41. Evaluation of Segmentation Algorithms

42. Rural area Multi-resolution in eCognition Mean-shift in ORFEO
Mean-shift in Matlab K-means in Matlab Watershed in Matlab
Multi-resolution
parameters
Mean-shift
parameters

43. Residential area Multi-resolution in eCognition Mean-shift in ORFEO
Mean-shift in Matlab K-means in Matlab Watershed in Matlab
Multi-resolution
parameters
Mean-shift
parameters

44. Urban area Multi-resolution in eCognition Mean-shift in ORFEO
Mean-shift in Matlab K-means in Matlab Watershed in Matlab
Multi-resolution
parameters
Mean-shift
parameters

45. Forest area Multi-resolution in eCognition Mean-shift in ORFEO
Mean-shift in Matlab K-means in Matlab Watershed in Matlab
Multi-resolution
parameters
Mean-shift
parameters

46. Evaluation of Automatic Estimation of Segmentation
Parameter Technique

47. Mean Square Error plot for the Trained Network

48. Average local variance v/s band for four regions of study

49. Test Images
Rural area Residential area
Urban area Forest area

50. Estimation of Segmentation Parameters of Multi-resolution
Rural area Residential area
Urban area Forest area

51. Estimation of Segmentation Parameters of Multi-resolution
Rural area Residential area
Urban area Forest area

52. Conclusions

53. Assessment of Segmentation Algorithm
• we see that the watershed algorithm is poor.
• k-means gives a good segmentation solution though it is affected by
salt and paper noise due to within-field variability.
• Multi-resolution segmentation is now becoming very famous due to
its robust nature and fast processing using multi-threading. From the
outcomes, we see that it gives very good segmentation in object level.
• Mean-shift segmentation is also a multi-scale segmentation like multi-
resolution, but it take a huge amount of time to process in both
ORFEO and Matlab.

54. Comparison between multi-resolution and mean-shift
segmentation

55. Automatic Estimation of the Optimal Segmentation Parameters
Advantages
• The proposed method is a pre-estimation method while other studies are
related to post-estimation. Only one approach is there which does pre-
estimation, but it only limits its scale selection to certain segmentation
algorithm.
• This proposed approach is not limited to any particular segmentation
algorithm. It can estimate any number of parameters for any segmentation
algorithm.
• Different remote sensing analysis vendors provide different implementation
of segmentation algorithms. This approach tries to map between any VHR
landcover and optimal segmentation parameter.
• Geo-statistics of every band is being considered which makes it more robust
to any multi-spectral data.

56. Limitations
• Training the BPNN network is a difficult task. Discussion about BPNN
network is beyond of the scope of this project.
• Calculation of local variance in multiple scale takes a long time. Due
to the advancement of high performance computing this limitation
can be overcome.
• The most important disadvantage is to make a training set for
learning the network. Initial human effort is needed but once the
network is learnt, the automation process is done

57. Thank you!
Any questions?