1. Manifold Alignment for MultitemporalHyperspectral Image Classification H. Lexie Yang1, Dr. Melba M. Crawford2 School of Civil Engineering, Purdue University and Laboratory for Applications of Remote Sensing Email: {hhyang1, mcrawford2}@purdue.edu July 29, 2011 IEEE International Geoscience and Remote Sensing Symposium

2. Outline Introduction Research Motivation Effective exploitation of information for multitemporal classification in nonstationary environments Goal: Learn “representative” data manifold Proposed Approach Manifold alignment via given features Manifold alignment via correspondences Manifold alignment with spectral and spatial information Experimental Results Summary and Future Directions

3. Introduction N>>30 3 2 1 2001 2003 2004 2005 2006 2002 2001 N narrow spectral bands June July May May May May June Challenges for classification of hyperspectral data temporally nonstationary spectra high dimensionality

4. Research Motivation Nonstationarities in sequence of images Spectra of same class may evolve or drift over time Potential approaches Semi-supervised methods Adaptive schemes Exploit similar data geometries Explore data manifolds Good initial conditions required

5. Manifold Learning for Hyperspectral Data Characterize data geometry with manifold learning To capture nonlinear structures To recover intrinsic space (preserve spectral neighbors) To reduce data dimensionality Classification performed in low dimensional space Original space Manifold space 3rd dim Spectral bands n Spatial dimension 6 5 4 3 2 1 Spatial dimension 1st dim 2nd dim

8. Approach: Extract and optimally align local geometry to minimize overall differencesLocality Spectral space at T2 Spectral space at T1

9. Proposed Approach: Conceptual Idea (Ham, 2005)

11. Manifold Alignment: Introduction and are 2 multitemporalhyperspectral images Predict labels of using labeled Explore local geometries using graph Laplacian and some form of prior information Define Graph Laplacian Twopotential forms of prior information: given features and pairwise correspondences [Ham et al. 2005]

12. Manifold Alignment via Given Features Minimize Joint Manifold Given Features

13. Manifold Alignment via Pairwise Correspondences Minimize Correspondences between and Joint Manifold

14. MA with spectral and spatial information Combine spatial locations with spectral signatures To improve local geometries (spectral) quality Idea: Increase similarity measure when two samples are close together Weight matrix for graph Laplacian: where spatial location of each pixel is represented as

16. May, June pair: Adjacent geographical area

17. June, July pair: Targeted the same areaMay June July

18. Experimental Results: Framework L L L I1, I2 I1 I2 Graph Laplacian Prior information Joint manifold Given features Classification with KNN Correspondences Develop Data Manifold of Pooled Data

19. Manifold Learning for Feature Extraction Global methods consider geodesic distance Isometric feature mapping (ISOMAP) Local methods consider pairwise Euclidian distance Locally Linear Embedding (LLE): (Saul and Roweis, 2000) Local Tangent Space Alignment (LTSA): (Zhang and Zha, 2004) LaplacianEigenmaps (LE): (Belkin and Niyogi, 2004) (Tenenbaum, 2000)

20. MA with Given Features Baseline: Joint manifold developed by pooled data 79.21 77.29 77.88 76.31 (May, June pair)

22. Summary and Future Directions Multitemporal spectral changes result in failure to provide a faithful data manifold Manifold alignment framework demonstrates potential for nonstationary environment by utilizing similar local geometries and prior information Spatial proximity contributes to stabilization of local geometries for manifold alignment approaches Future directions Investigate alternative spatial and spectral integration strategy Address issue of longer sequences of images

23. Thank you. Questions?

24. References J. Ham, D. D. Lee, and L. K. Saul, “Semisupervised alignment of manifolds,” in International Workshop on Artificial Intelligence and Statistics, August 2005.

25. Backup Slides

26. Local Manifold Learning for Feature Extraction (s,f) Local geometry preserved via various strategies for embedding Popular local manifold learning methods Locally Linear Embedding (LLE): (Saul and Roweis, 2000) Local Tangent Space Alignment (LTSA): (Zhang and Zha, 2004) LaplacianEigenmaps (LE): (Belkin and Niyogi, 2004) Pairwise distance between neighbors computed using Gaussian kernel function - O(pN2) method Embedding computed to minimize the total distance between neighbors

27. LE: Impact of Parameter Values Parameter values for local embedding s obtained via grid search k, p obtained empirically BOT Class 3, 6 BOT Classes 1-9

30. Alignment Results: Critical Class Critical class: Woodlands

32. MA Results – Classification Accuracy Classified via Given Features (Spectral + spatial) Classified via Correspondences (Spectral + spatial) Labeled Class (Subset Data) Classified via Given Features (Spectral) Classified via Correspondences (Spectral) May, June pair