Big Open Data Transformation Through Public Data Sets - AWS Washington D.C. Symposium 2014
•
2 recomendaciones•1,553 vistas
In this conversation, AWS and government thought leaders will discuss ways to encourage public private partnership to solve societal problems thru open data. Ariel Gold, Program Manager, AWS and Tsengdar Lee of NASA will shares insight on NASA NEX.
Recomendados
Recomendados
Más contenido relacionado
Destacado
Destacado (20)
Similar a Big Open Data Transformation Through Public Data Sets - AWS Washington D.C. Symposium 2014
Similar a Big Open Data Transformation Through Public Data Sets - AWS Washington D.C. Symposium 2014 (20)
Más de Amazon Web Services
Más de Amazon Web Services (20)
Último
Último (20)
Big Open Data Transformation Through Public Data Sets - AWS Washington D.C. Symposium 2014
- 1. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Big Open Data: Transformation through Public Data Sets Ariel Gold arigold@amazon.com
- 2. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Agenda • Open Data Overview (Ariel Gold, AWS) • NASA NEX (Dr. Tsengdar Lee, NASA) • Data Distribution Model (Ariel Gold, AWS) • Q&A
- 3. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014
- 4. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 What is Open Data? Data released to the public in ways that make it easy to discover, access, and use.
- 5. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Why are governments making their data open? Five main goals seen across different policies and programs.
- 6. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Reduced backlog in manual submissions while improving accuracy and reducing costs Enabled ingestion of new open data sources to improve mission delivery Shifted from quarterly reports to developer-focused APIs, raw data, and documentation
- 7. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Release Definition & Policy Launch Open Data Catalog Do Things w/ Open Data Release New (Big) Open Data Repeat w/ Increased Focus Impact Gather Customer Feedback Open Data Adoption Curve
- 8. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 How do you drive impact with (big) (open) data? Build communities and tools around sustainable open data sets.
- 9. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 OpenNEX as an Open Data Analytical Service Tsengdar Lee, Ph.D. tsengdar.j.lee@nasa.gov
- 10. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 NASA EARTH EXCHANGE (NEX) Virtual collaborative-for-global-change science COLLABORATION Web & System COMPUTING NASA HECC Pleiades 180k+ cores Endeavour 1.5k+ cores, 6TB D-wave Quantum 512 Qbit DATA REPOSITORY 800 TB, 126 PB network links
- 11. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 NEX Provides a Complete Work Environment “Science as a Service” COLLABORATION Over 400 Members CENTRALIZED DATA REPOSITORY Over 1300 TB of Data COMPUTING Scalable Diverse Secure/Reliable KNOWLEDGE Workflows Machine Images Re-useable software
- 12. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 • 33 CMIP5 models available • All 4 RCPs (2.6, 4.5, 6.0, 8.5) and historical runs • 30 arc-second (800m) spatial resolution, monthly time-step, 1950-2099 • Max/min temperature and precipitation • Statistical downscaling (bias-corrected spatial disaggregation; Maurer et al., 2007) Thrasher et al., 2013, Eos Creating NEX-DCP30 Downscaled Climate Projections at 30 Arc-Second Resolution for the National Climate Assessment
- 13. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 • Sam Goward, PI • Detecting 30m resolution forest disturbance annually • National runs complete • Data distribution being worked North American Forest Dynamics (NAFD) 13
- 14. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 April 2010 October 2010 • David Roy, PI • Completed processing 140,000 scenes for first year of global Landsat (monthly and annual) • Public release later in June Web-Enabled Landsat Data (WELD) 14
- 15. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 California Drought Monitoring at 250m Scale Changes in High Latitude Ecosystems - Greening
- 16. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 A tale of two droughts/Amazon 2005 & 2010 TRMM MODIS Workflows Improving Research Productivity Faster (24 months vs. 6 months), consistent (same analytical methods, quality flags) and reproducibleSamantha et al., GRL, 2011 Xu et al., GRL, 2011
- 17. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 NEX and AWS Sandbox on Public Cloud - Prototyping - Development NAS HECC - Large-Scale processing - NCA Example COLLABORATION (more public members) COMPUTING HECC + AWS1 DATA REPOSITORY (AWS OpenData, with “Market driven” Cache) AWS
- 18. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 2014 International Space Apps Challenge 46 countries participated 735 virtual participants 69 virtual team projects Space Apps kicked off Thursday night in Doha, Qatar at midnight and ended in Seattle, WA at 6 p.m. Pacific—76 hours of around-the-world hacking on NASA data.
- 19. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 2014 International Space Apps: Most Popular Challenges Earth Watch: Where on Earth (44 projects), Climate & Neighborhood (22), Cool It (21) Robotics: Exomars Rover is My Robot (42) Asteroids: Asteroid Prospector (38) Space Tech: Space Wearables (25), Alert-Alert (24) Space Flight: Growing Food for A Martian Table (22), SpaceT (20)
- 20. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 2014 International Space Apps: OpenNEX Data Challenges Climate and My Neighborhood: 22 projects Climate Alert: People’s Choice Local Award Winner Climatehood: Best Use of Data Local Award Winner Ways to Die: Most Inspiring Local Award Winner NASA-NEX-Climate-Changes: People’s Choice Local Award Winner
- 21. 21 OpenNEX Workshops and Challenges 7/1 – 11/15/2014 • Virtual Workshop teaches how to use OpenNEX AMIs and Public Data sets on AWS. • Collaborate within arms length - Users work with vm templates including prepackaged tools, they instantiate, extend and own accordingly. - Core data is read only public data sets on S3 • Bring citizen scientists together around data, services and knowledge • Extract more value out of assets, i.e. data and knowledge, via the group • http://nex.nasa.gov/OpenNEX
- 22. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Thank You AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Thank You Tsengdar Lee, Ph.D. tsengdar.j.lee@nasa.gov
- 23. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 How do you drive impact with (big) (open) data? Build communities and tools around sustainable open data sets.
- 24. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 National Agriculture Imagery Program
- 25. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Before • Pay $5k • Wait weeks to get data on disks • Technical and GIS expertise to make data usable After • No or minimal cost • Get access immediately • No technical or GIS expertise to start using Significantly increase openness of a 48TB aerial imagery data set at a low and sustainable cost ($1,600/month) National Agriculture Imagery Program
- 26. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Managing the Cost of Data Distribution • Amazon Simple Storage Service (S3) “requestor pays” • Eliminate variable costs • Multiple configuration options depending on your use case
- 27. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Insert NAIP Demo Slides (or link to live pages)
- 28. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Q & A For more information on AWS and open data, visit: http://aws.amazon.com/government-education/open-data/
- 29. AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014 Thank You Ariel Gold arigold@amazon.com