Making Earth observation data available by using Amazon S3 is accelerating scientific discovery and enabling the creation of new products. Attend and learn how the scale and performance of Amazon S3 lets earth scientists, researchers, startups, and GIS professionals gather and analyse planetary-scale data without worrying about limitations of bandwidth, storage, memory, or processing power. Co-presented with support of the Australian Geoscience Data Cube collaboration, DigitalGlobe’s Geospatial Big Data Platform and the developer of the popular ObservedEarth mobile app. Speakers: Craig Lawton, Public Sector Solutions Architect, Amazon Web Services Lachlan Hurst, Observed Earth Matt Paget, Senior Experimental Scientist, CSIRO Dan Getman, Digital Globe

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Craig Lawton
Public Sector Solutions Architect
Amazon Web Services
Level 200
Earth on AWS
Next-Generation Open Data Platforms
Dan Getman
Geospatial Big Data Solutions
DigitalGlobe
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Matt Paget
Senior Experimental Scientist
CSIRO
Lachlan Hurst
Senior Software Engineer
Observed Earth

2. What to Expect from the Session
• Open data on AWS and the advantages of using Amazon S3
• ObservedEarth Developer
• CSIRO & Australian Geoscience Data Cube
• DigitalGlobe & PSMA

3. Why Does AWS Care About Open Data?
Sharing data on AWS makes it accessible to a large and
growing community of researchers, entrepreneurs,
and enterprises who use the AWS cloud.

4. “…data must be organised, well-documented,
consistently formatted, and error free. Cleaning
the data is often the most taxing part of data
science, and is frequently 80% of the work.”
- Data Driven by DJ Patil and Hilary Mason
Undifferentiated Heavy Lifting

5. Traditional Data Acquisition
Data Acquisition in the Cloud
AWS IoT
AWS Snowball

6. When data is shared in the cloud,
anyone can analyse any volume of
data without needing to download or
store it themselves.

7. Elevation
Models
Aerial
Imagery
Climate
Models
Satellite
Imagery
High-resolution
Radar
aws.amazon.com/earth

8. AWS Cloud Credits for Research
provide promotional AWS cloud credits
for anyone to conduct research using
Earth Observation data.
aws.amazon.com/earth/research-credits

9. Landsat
Landsat 8 satellite Raster data

10. Landsat on AWS
Amazon
EC2
s3://landsat-pds
.tarUSGS
.tiff
Lambda

11. Serverless browser interface to
Landsat on AWS
• GDAL – http://www.gdal.org/
• Rasterio –
https://mapbox.github.io/rasterio/
• sat-utils suite –
https://github.com/sat-utils
landsatonaws.com

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Lachlan Hurst
Senior Software Engineer, Observed Earth
Level 200
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ObservedEarth
Earth Observation Data on your iPhone and iPad

13. First Came the Data…
Landsat 8
• Multispectral
• 30m resolution
• Global coverage
• Updated every 16 days
• Open data
Landsat on AWS
• 1 million scenes (Feb 2017)
• 800MB per scene
• Updated daily
• Data freely available on S3
Image credit: NASA, GSFC,
Landsat

14. … then the Idea
ObservedEarth aims to make Landsat data
accessible to everyone.
• Smartphone
• App with access to data
• Build in domain expertise
Little app for a lot of data

15. User Experience

16. Filters
Different combinations of Landsat bands
highlight specific attributes

17. Kati Thanda/Lake Eyre – South Australia
Largest lake in
Australia
• 9500 km2
• 15m below sea level
• Avg 1.5m deep
• Land speed records

18. Nygan – Regional NSW
Any guesses?

19. How Does it Work?
GPUs are great for processing raster data
• Landsat data downloaded from S3 read into textures
• ObservedEarth filters implemented as fragment shaders

20. Reception
Users
• 200000 scenes viewed
• 400GB data downloaded
Media
ObservedEarth: A Talk with its Creator - Q&A with NASA
#Landsaturated - Medium post by Joe Morrison
“. . .a brilliant little tool for
students or anyone interested
in the earth and remote
sensing. Thank you”
“Awesome!! Thank you for your
time. I'm a student, currently
working on landslide hazard ...I
have no words. It is beautiful!”
“Great to play with”
App Store reviews

21. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Matt Paget
Data and Systems for Earth Observation Sciences
CSIRO
Level 200
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Earth Observation and CSIRO analytics
Next-Generation Open Data Platforms

22. http://map.geo-rapp.org

24. Our Next Challenge
People expect access to EO data
and products
New EO satellite missions
Will require fundamental changes
to how EO data are analysed
Courtesy of Geoscience Australia

25. What are (EO) Data Cubes?
• Data Cube = Spatially aligned pixels
ready for analysis
• Proven concept by Geoscience
Australia (GA), the Australian Science
Agency (CSIRO) and a super-
computer facility (NCI)
• Shift in Paradigm ... Pixels vs Files
• Analysis Ready Data (ARD) ...
Reduce processing burden on users
• Supports integration of multiple
datasets.
• Multi-platform … PC, HPC, Cloud TIME
Open Source Software
https://github.com/opendatacube

26. Data Cubes and AWS
AWS for exploratory analysis
and market access
• Fast, scalable and accessible
• Address multi-platform criteria
New data structures for EO
• Aligned with S3 principles
• Analytics to take advantage of
I/O concurrency, built on new
low-level APIs
• Execution engine exploiting
the cloud

27. Earth Analytics Industry Hub
Public good component
using free satellite data
• A global network of
interoperable data cubes
Commercial component
• Business processes and engagement to sustain
science and data pipelines
“App Store” of analytics routines
• EO and geoscience researchers delivering to
minerals, agriculture and environment domains
Multi-platform
• Data cube applications work seamlessly across
Cloud, HPC and your desktop

28. Thank you
Matt Paget matt.paget@csiro.au
Rob Woodcock robert.woodcock@csiro.au
Australian Geoscience Data Cube and the
Open Data Cube Community
• Geoscience Australia
• CSIRO
• National Computational Infrastructure
• Committee on Earth Observing Satellites
• NASA and USGS

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dan Getman
Geospatial Big Data Solutions, DigitalGlobe
Level 200
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Getting Data Out of Jail
DigitalGlobe’s Geospatial Big Data Platform

30. DigitalGlobe is…
The market leader in remote sensing.
IKONOS®
.82 meter
resolution
9 m CE90
QuickBird®
.65 meter
resolution
23 m CE90
WorldView-4®
.30 meter
resolution
<3.0 m CE90
WorldView-3®
.30 meter
resolution
<3.0 m CE90
WorldView-2®
.46 meter
resolution
<3.5 m CE90
GeoEye-1®
.46 meter
resolution
<3.5 m CE90
WorldView-1®
.50 meter
resolution
<4 m CE90
CURRENTLY IMAGING IN ORBIT

31. 3,500,000 km2
13,200,000,000,000 pixels

32. three days

33. two weeks

34. three months

35. 16 yr
digital
inventory

37. Information
Road
Not sure…
Building
Intersection
Tree
Car
Boat
Hotel

38. 1.
Three Trends Working in our Favour

39. 2. hidden layer 1 hidden layer 2 hidden layer 3
Three Trends Working in our Favour

40. 3.
Three Trends Working in our Favour

41. Open up Geo to world of innovators working on Deep Learning
Solve a new class of problems
to impact business, government, and people

42. Strategic Approach: Leverage the Ecosystem
Crowdsourcing

43. Storage Compute
GBDX Ecosystem
GBD Partner
Tools

44. SWIMMING POOLS
DIGITAL TERRAIN MODEL
ROOF MATERIALS
BUILDING POLYGONS
ROOF COMPLEXITY
TREE HEIGHT
SOLAR PANELS
ROOF HEIGHT
LULC

45. million square km7.6
24
13
million people
million structures

46. Scale humans until machines get smarter:
Use humans to create training data
Crowdsourcing

47. Training and testing set to support
benchmarking and iteration

48. We have common frameworks to accelerate prototyping
We need a common data set to evaluate performance

49. Rio Olympics – Origins of SpaceNet
• Governments around the world were
preparing for the upcoming Olympics in
Rio De Janeiro
• Thorough understanding of the
security and safety of the athletes is
front of mind
• Knowledge of high risk areas, traffic
patterns, line-of-sight and overall
structure distribution is critical to ensure
a secure games

51. - 50cm WV-2 image w/8 spectral bands
- Each image 200m2
- Covering Rio de Janeiro, Brazil
- 7000 Geotiff images
Available for your computing pleasure on

52. 1) Satellite imagery analytics will be way more common in your industry
2) Computer vision tasks are going to change the way we see the world
3) We are going to learn a lot
What You Can Expect

53. What You Can Do
ObservedEarth app observedearth.com
Australian Geoscience Data Cube: www.datacube.org.au
DigitalGlobe: aws.amazon.com/public-data-sets/spacenet/
developer.digitalglobe.com
Earth on AWS : aws.amazon.com/earth
AWS Open Data & Partners:
aws.amazon.com/government-education/open-data

54. Thank you!