Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

SpaceNet: Accelerating Machine Learning for Foundational Mapping Challenges

SpaceNet is a nonprofit LLC designed to accelerate machine learning against geospatial problems, such as mapping road network routes after a natural disaster using exclusively remote sensing data. Over the last two and half years, SpaceNet has released over 6500 sq km of high-resolution satellite imagery, with ~800,000 building footprint labels and 8000 sq km of road network labels. In addition to open sourcing a large, curated data set, SpaceNet has developed and administered four data science challenges to solve the problem of extracting building footprint and road networks from satellite imagery at scale. We will discuss the challenges of deploying these machine learning algorithms in operational timelines, and how AWS products be used to accelerate delivery of timely information derived from satellite imagery after a natural disaster. We will also highlight upcoming analytic challenges.

  • Sé el primero en comentar

  • Sé el primero en recomendar esto

SpaceNet: Accelerating Machine Learning for Foundational Mapping Challenges

  1. 1. P U B L I C S E C T O R S U M M I T Washington DC
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T SpaceNet: Accelerating Machine Learning for Foundational Mapping Challenges Ryan Lewis SVP IQT CosmiQ Works S e s s i o n I D Joe Flasher Open Geospatial Data Lead AWS Adam Van Etten Research Director IQT CosmiQ Works Todd Bacastow Senior Director Maxar
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Agenda SpaceNet Introduction Previous Challenge Results Upcoming Challenges Information Channels
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 5 © SpaceNet LLC 2019. (1) Machine learning algorithms & (2) increased overhead data collection will fundamentally disrupt geospatial analytics Convergence of Two Tech Trends 5
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 6 © SpaceNet LLC 2019. Solutions Are Required Source: DigitalGlobe/Maxar, CNN, and Humanitarian Open Street Map (HOT). Required to Map Puerto Rico After Hurricane Maria 70+ Days to Completely Map 5,300+ Volunteer Mappers 950,000 Building Labels 30,000 Kms of Road Labels 6
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 3 Market Challenges 7 Lack of Curated, Labeled Data Sets for Geospatial Applications Open Source, AI Models Designed for Different Problems Open Software Tools for Geospatial Analysis Are Limited
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 8 SpaceNet’s Mission SpaceNet is a nonprofit LLC focused on: 1. Data Developing Open Source Data Sets 2. Algorithms Fostering Applied Research for AI Software 3. Evaluation Benchmarking Performance for Applications
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T SpaceNet: 4 Pillars 9 Labeled Data Sets Competitions Algorithms Software Tools • Images of 6 Cities • 800,000+ Building Footprints • 10,000 km2 Road Labels • 4 Competitions on TopCoder • $200,000 in Total Prizes • 1,000+ Submissions Worldwide • 18 Algorithms o 13 Building Detection o 5 Road Detection & Routing • Ease Use of Imagery • Simplify Evaluation • Speed Up Model Deployment © SpaceNet LLC 2019.
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T SpaceNet: Opening the Floodgates for GEOINT R&D 2016 SpaceNet 1: Building Footprint Extraction Cars Overhead With Context (COWC) IARPA Multi-View Stereo 3D Mapping 2017 SpaceNet 2: Multi-City Building Footprints IARPA Functional Map of the World USSOCOM Urban 3D Challenge 2018 SpaceNet 3: Road Network Extraction SpaceNet 4: Off-Nadir Building Footprints CrowdAI Mapping Challenge DIUx xView Object Detection Challenge 2019 Microsoft U.S. & Canadian Building Footprints Upcoming: SpaceNet 5: Roads with travel time 10
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Competitions to Date 11 SpaceNet 1 11/2016 – 1/2017 SpaceNet 2 6/2017 – 8/2018 SpaceNet 3 11/2017 – 2/2018 SpaceNet 4 10/2018 – 1/2019 Building Footprint Detection Rio De Janeiro Building Footprint Detection Las Vegas, Paris, Khartoum, & Shanghai Road Extraction & Routing Las Vegas, Paris, Khartoum, & Shanghai Building Footprint Detection (Off- Nadir) Atlanta © SpaceNet LLC 2019.
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Automated Overhead Imagery Analysis is Improving 12
  13. 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 13 Public Data Sets: 1 Open Source Software: 1 Open Source Software: 2 Competition Submissions 347M Total Repository Hits 18 Algorithms 22 CosmiQ Repositories 1,000+ Across All 4 Challenges Public Data Sets: 2 268TB Total Downloads Serving an Unmet Need International Participation 79 Countries with Downloads
  14. 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Impact: Public Data Sets 14 Top Country Hits (2018) 1. USA 2. Canada 3. China 4. India 5. UK © SpaceNet LLC 2019.
  15. 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 15 Fei Fei Li’s, founder of ImageNet, presentation at CVPR 2017 Community Acknowledgement © SpaceNet LLC 2019.
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T SpaceNet 4 Overview 17 Imagery Data Set 27 Collects Over Atlanta 7O to 54O Off-Nadir 655 km2 Covered 0.5 m Resolution Labels 126,747 Building Footprints From 20 m2 to >2,000 m2 Urban, Industrial, & Suburban ~3,000 km Road Network Labels Algorithms 5 Open Sourced Solutions 15 Computer Vision Models w/ Solution Explanations > 250 Competition Submissions
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Why Off-Nadir Imagery 18 Urgent Collections are Often Off-Nadir (below) State-of-the-art algorithms were untested on off-nadir imagery Daiichi Power Plant | Fukushima, Japan Look Angle: About 45º Imagery Courtesy of DigitalGlobe
  19. 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Barriers to Off-Nadir Imagery Analysis 19 Variable Shadows Occluded Structures Footprint Displacement Resolution Degradation
  20. 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Scoring: Building Footprint Extraction 20 1. Find predicted buildings with Intersection over Union (IoU) > 0.5 Truth Pred. IoU = 0.75 Success IoU = 0.15 Failure 2. Aggregate successes/failures across all collections in three look angle bands: Ground truth A. Nadir: 0-25 B. Off-nadir: 26-40 C. Very off-nadir: > 40
  21. 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Algorithmic Challenges: Nadir Angle 21 40% drop in score for all algorithms from 7º to 54º
  22. 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T 22 Algorithmic Challenges?: Shadows Image from the South (facing north) Image from the North (facing south)
  23. 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Top buildings: Not occluded Bottom buildings: Occluded by trees Algorithmic Challenges?: Occluded Buildings
  24. 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Algorithmic Challenges: Building Size
  25. 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Footprint Quality Threshold Matters 25 Truth Pred. IoU = 0.75 IoU = 0.15
  26. 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Research Publication from SpaceNet 4 26 Link: https://arxiv.org/abs/1903.12239
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T
  28. 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T What’s Next: Returning to Road Networks Scoring based on pixel masks does not always incentivize the desired outcomes
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T • Segmentation efforts have demonstrated some success in identifying road pixels from overhead imagery but do not always incentivize the desired outcome • Evaluation metrics are pixel-based: (1) completeness, correctness, quality, and (2) relaxed F1 (correct value within 3 pixels Wang et al 2016 (Qaulity = 0.86) http://www.mdpi.com/2220-9964/5/7/114 Zhang et al 2017 (relaxed F1 = 0.92) https://arxiv.org/pdf/1711.10684.pdf Mhih and Hinton 2010 (relaxed F1 = 0.90) http://www.cs.toronto.edu/~fritz/absps/road_detection.p df Limitations of Current Segmentation Techniques
  30. 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Scoring Routing Information: APLS •Average Path Length Similarity (APLS) was developed for SN3 •Both Logical and Physical Topology Are Important for Road Detection •Sum the Difference in Paths between Ground Truth & Proposals •Betweenness Centrality is Fundamental to the APLS Metric
  31. 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Ground Truth 1. Parse road labels into 400m sections concurrent with SpaceNet imagery 2. Create ground truth masks by drawing a 2m buffer around road centerlines in road labels 3. Augment the training dataset by a factor of 3 via HSV rescaling and rotations [increases performance by 8% (Vegas) to 13% (Khartoum)] 4. Train a deep learning segmentation model (PSPNet, U-Net) using SpaceNet imagery & road masks 5. Perform post-processing to eradicate short segments and close small gaps Predictions: Segmentation Model
  32. 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T A. Extract skeleton from proposal mask B. Build a proposal graph from the skeleton C. Simplify and smooth proposal graph A B C Predictions: Mask to Graph
  33. 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Entrant Country Avg Score Las Vegas Paris Shanghai Khartoum albu Russia 0.6663 0.7977 0.6040 0.6543 0.6093 cannab Russia 0.6661 0.7804 0.6446 0.6398 0.5996 pfr France 0.6660 0.8009 0.6008 0.6646 0.5975 selim_sef Germany 0.6567 0.7884 0.5991 0.6472 0.5922 fabastani USA 0.6284 0.7710 0.5474 0.6326 0.5628 ipraznik Germany 0.6215 0.7578 0.5668 0.6078 0.5537 tcghanareddy India 0.6182 0.7591 0.5710 0.6014 0.5415 hasan.asyari Norway 0.6097 0.7407 0.5557 0.5952 0.5472 aveysov Russia 0.5943 0.7426 0.5805 0.5751 0.4789 SpaceNet 3 Results Winning entrant, albu, submitted a generalized model across all four cities
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T AOI_2_Vegas_img1011 – APLS = 0.512 AOI_2_Vegas_Img1045 – APLS = 0.988 Dave Lindenbaum, GTC 2018 Las Vegas Results
  35. 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I TDave Lindenbaum, GTC 2018 Khartoum Results AOI_5_Khartoum_img404 - APLS = 0.385 AOI_5_Khartoum_img398 – APLS = 0.897 35
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Ground Truth Raw Proposal Mask Extract Entire Khartoum Road Network •Combined BASISS & Albu’s Implementation (w/ Extra Post-Processing from CosmiQ Works) o Image Size (Pixels) 55,420 x 161,258 (9 terapixels) o Image Size (Km) 16 x 48 o File Size (GB) 89 o Nodes 195,938 o Edges 258,655 Total Processing Time = 6.3 Hours (Single GPU/CPU)
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T SpaceNet 5: Expanding Upon Routing 37 Challenge Participants Will Be Asked to Infer: … From a Single Satellite Image Road Networks Routing Information Travel Times The SpaceNet 5 Challenge is Scheduled to Launch in September 2019
  38. 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T APLS = 0.81 Test Image: RGB-PanSharpen_AOI_2_Vegas_img727.tif Multiclass Baseline 38 •Trained resnet34 + unet segmentation model o Use 7– channel training masks RGB Image Labels Projection
  39. 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T SpaceNet 6: Preliminary Planning 39 Model Deployability / Generalizability New Applications Beyond Foundational Mapping Sensor & Data Fusion
  40. 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T
  41. 41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Information Channels 41 CosmiQ’s Repo: https://github.com/CosmiQ SpaceNet’s Repo: https://github.com/SpaceNetChalle nge SpaceNet Competition Hosting Site https://www.topcoder.com/space net SpaceNet Data on AWS https://registry.opendata.aws/spac enet/ CosmiQ’s Blog: The DownLinQ https://medium.com/the- downlinq CosmiQ’s Twitter: @CosmiQWorks Title: Training_Data Found on: Apple Podcasts, Spotfiy, Stitcher, & SoundCloud
  42. 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T Ryan Lewis rlewis@iqt.org
  43. 43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C TO R S U M M I T

    Sé el primero en comentar

    Inicia sesión para ver los comentarios

SpaceNet is a nonprofit LLC designed to accelerate machine learning against geospatial problems, such as mapping road network routes after a natural disaster using exclusively remote sensing data. Over the last two and half years, SpaceNet has released over 6500 sq km of high-resolution satellite imagery, with ~800,000 building footprint labels and 8000 sq km of road network labels. In addition to open sourcing a large, curated data set, SpaceNet has developed and administered four data science challenges to solve the problem of extracting building footprint and road networks from satellite imagery at scale. We will discuss the challenges of deploying these machine learning algorithms in operational timelines, and how AWS products be used to accelerate delivery of timely information derived from satellite imagery after a natural disaster. We will also highlight upcoming analytic challenges.

Vistas

Total de vistas

422

En Slideshare

0

De embebidos

0

Número de embebidos

0

Acciones

Descargas

0

Compartidos

0

Comentarios

0

Me gusta

0

×