SlideShare a Scribd company logo

Climate Data Lake: Empowering Citizen Scientists in Acadia National Park

•Download as PPTX, PDF•
•
Srivatsan Ramanujam
Srivatsan Ramanujam

Learn how EMC and Pivotal are teaming up to empower citizen scientists @ Acadia National Park to study climate change and its influence on phenology in the park, by building a Climate Data Lake.

Data & Analytics
1 of 21
1© Copyright 2013 Pivotal. All rights reserved. 1© Copyright 2013 Pivotal. All rights reserved. Climate Data Lake: Empowering Citizen Scientists in Acadia National Park Srivatsan Ramanujam Principal Data Scientist, Pivotal @being_bayesian John Cardente Distinguished Engineer/Lead Data Scientist, EMC @johncardente 19-Feb-2015
2© Copyright 2013 Pivotal. All rights reserved. 2© Copyright 2013 Pivotal. All rights reserved. Climate Change is happening Source: NASA
3© Copyright 2013 Pivotal. All rights reserved. Source: National Geographic At the Ice Caps… From Chasing Ice by James Balog
4© Copyright 2013 Pivotal. All rights reserved. And in the Oceans…
5© Copyright 2013 Pivotal. All rights reserved. 5© Copyright 2013 Pivotal. All rights reserved. Weather is Nature’s clock
6© Copyright 2013 Pivotal. All rights reserved. 6© Copyright 2013 Pivotal. All rights reserved. And the clock is shifting Source: EPA
7© Copyright 2013 Pivotal. All rights reserved. 7© Copyright 2013 Pivotal. All rights reserved. English Oak Winter Moth Pied Flycatcher Both et al. 2006 Nature EARLIER SAME TIME EACH YEAR Disrupting species interactions
8© Copyright 2013 Pivotal. All rights reserved. 8© Copyright 2013 Pivotal. All rights reserved. …the study of the timing of recurring plant and animal life- cycle stages, or phenophases, and their relationship to environmental conditions. Phenology Source: USA National Phenology Network
9© Copyright 2013 Pivotal. All rights reserved. 9© Copyright 2013 Pivotal. All rights reserved. Acadia National Park Source: Wikipedia A key phenology observation point
10© Copyright 2013 Pivotal. All rights reserved. 10© Copyright 2013 Pivotal. All rights reserved. Citizen Science, the “Hadoop” of Phenology
11© Copyright 2013 Pivotal. All rights reserved. Challenges impeding scientific progress  A multitude of disparate data sources  Different approaches to data conditioning  Different analysis platforms and scripts Inconsistent data treatment distracts the scientific debate from the findings!
12© Copyright 2013 Pivotal. All rights reserved. Challenges to encouraging citizen science  Different infrastructures for data collection  Unable to see how contributions help science  Inability to analyze the data and collaborate with scientists Missing a feedback loop to encourage and engage citizen scientists
13© Copyright 2013 Pivotal. All rights reserved. EMC Federation teams up with Earthwatch
14© Copyright 2013 Pivotal. All rights reserved. Objectives • Authoritative, curated data repository • Shared analysis platform and software libraries • Web portals to – Show citizen scientists how contributions benefit science – Present data visualization and exploration tools – Provide access to data sets for volunteer analysis – Offer common data collection APIs and interfaces – Host forums to interact with leading scientists
15© Copyright 2013 Pivotal. All rights reserved. Climate Data Lake PL/X Hadoop++ Data Data Lake Foundation Explore CloudFoundry Collect Isilon ECS
16© Copyright 2013 Pivotal. All rights reserved. The kick-off expedition at Acadia
17© Copyright 2013 Pivotal. All rights reserved. Post kick-off activities • Established partnerships with data providers • Stood up initial Data Lake platform • Active collaboration across all participants on data conditioning, analysis, feature selection, and visualization • Launching an initial web presence in March
18© Copyright 2013 Pivotal. All rights reserved. What’s next • Pursuing the “full vision” using Federation Technologies • Talking with other potential corporate sponsors • Exploring ways to expand support to other climate and academic projects
19© Copyright 2013 Pivotal. All rights reserved. 19© Copyright 2013 Pivotal. All rights reserved.
20© Copyright 2013 Pivotal. All rights reserved. Learn More • White House Press Release: Harnessing Climate Data to Boost Ecosystem & Water Resilience • Pivotal Blog: Pivotal Data Science Looks at Climate Change @ Acadia NP • EMC Reflections Blog: Big Data vs. Climate Change BlogsEMC TV video
21© Copyright 2013 Pivotal. All rights reserved. The Team @alyssacaddle1 @stevendozen @johncardente @lilewite @being_bayesian @maryanngarwood @emctv @TheMatthewNolan @johnnysuire @sriauggie31 @earthwatch_org @schoodicinst @hwschoodic + many more (not on twitter ) !

Recommended

Slide deck-cybernetic pv-tree-ver4
Slide deck-cybernetic pv-tree-ver4Slide deck-cybernetic pv-tree-ver4
Slide deck-cybernetic pv-tree-ver4Sarah Ramdhany
 
Slide deck-cybernetic pv-tree-ver5
Slide deck-cybernetic pv-tree-ver5Slide deck-cybernetic pv-tree-ver5
Slide deck-cybernetic pv-tree-ver5Sarah Ramdhany
 
Investigating Different Perspectives
Investigating Different PerspectivesInvestigating Different Perspectives
Investigating Different PerspectivesKiciaCigail
 
Energy+Conservation2
Energy+Conservation2Energy+Conservation2
Energy+Conservation2raysparks
 
Presentation1
Presentation1Presentation1
Presentation1k17himart
 
Global Geothermal Development, CanGEA Conference, March 2013
Global Geothermal Development, CanGEA Conference, March 2013Global Geothermal Development, CanGEA Conference, March 2013
Global Geothermal Development, CanGEA Conference, March 2013ThinkGeoEnergy
 
Rural Learning Center Campus Vision update 6.25.2008
Rural Learning Center Campus Vision update 6.25.2008Rural Learning Center Campus Vision update 6.25.2008
Rural Learning Center Campus Vision update 6.25.2008Joe Bartmann
 
Ag Data Commons for AgBioData
Ag Data Commons for AgBioDataAg Data Commons for AgBioData
Ag Data Commons for AgBioDataCyndy Parr
 

Recommended

Slide deck-cybernetic pv-tree-ver4
Slide deck-cybernetic pv-tree-ver4Slide deck-cybernetic pv-tree-ver4
Slide deck-cybernetic pv-tree-ver4Sarah Ramdhany
 
Slide deck-cybernetic pv-tree-ver5
Slide deck-cybernetic pv-tree-ver5Slide deck-cybernetic pv-tree-ver5
Slide deck-cybernetic pv-tree-ver5Sarah Ramdhany
 
Investigating Different Perspectives
Investigating Different PerspectivesInvestigating Different Perspectives
Investigating Different PerspectivesKiciaCigail
 
Energy+Conservation2
Energy+Conservation2Energy+Conservation2
Energy+Conservation2raysparks
 
Presentation1
Presentation1Presentation1
Presentation1k17himart
 
Global Geothermal Development, CanGEA Conference, March 2013
Global Geothermal Development, CanGEA Conference, March 2013Global Geothermal Development, CanGEA Conference, March 2013
Global Geothermal Development, CanGEA Conference, March 2013ThinkGeoEnergy
 
Rural Learning Center Campus Vision update 6.25.2008
Rural Learning Center Campus Vision update 6.25.2008Rural Learning Center Campus Vision update 6.25.2008
Rural Learning Center Campus Vision update 6.25.2008Joe Bartmann
 
Ag Data Commons for AgBioData
Ag Data Commons for AgBioDataAg Data Commons for AgBioData
Ag Data Commons for AgBioDataCyndy Parr
 
Geothermal Energy - Global Development Status
Geothermal Energy - Global Development StatusGeothermal Energy - Global Development Status
Geothermal Energy - Global Development StatusThinkGeoEnergy
 
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016ThinkGeoEnergy
 
Future Energy Summit 2017 - Environmental Law & Policy Center
Future Energy Summit 2017 - Environmental Law & Policy CenterFuture Energy Summit 2017 - Environmental Law & Policy Center
Future Energy Summit 2017 - Environmental Law & Policy CenterEconomic Alliance of Kankakee County
 
Nha Renewables Mann
Nha Renewables MannNha Renewables Mann
Nha Renewables MannDebbi Smith
 
Energy Research and Innovation
Energy Research and InnovationEnergy Research and Innovation
Energy Research and InnovationCUSP | Univ of Guelph
 
Presentation
Presentation Presentation
Presentation Alaa Khaled
 
9. Media Exposure- DNA News
9. Media Exposure- DNA News9. Media Exposure- DNA News
9. Media Exposure- DNA NewsJaymin Gajjar
 
Andy Fox DART
Andy Fox DARTAndy Fox DART
Andy Fox DARTDavid LeBauer
 
ICHEC - Observation systems, technologies and big data
ICHEC - Observation systems, technologies and big dataICHEC - Observation systems, technologies and big data
ICHEC - Observation systems, technologies and big dataEnvironmental Protection Agency, Ireland
 
Introdution to Landsat and Google Earth Engine
Introdution to Landsat and Google Earth EngineIntrodution to Landsat and Google Earth Engine
Introdution to Landsat and Google Earth EngineVeerachai Tanpipat
 
Improving access to geospatial Big Data in the hydrology domain
Improving access to geospatial Big Data in the hydrology domainImproving access to geospatial Big Data in the hydrology domain
Improving access to geospatial Big Data in the hydrology domainClaudia Vitolo
 
Climate Data Sharing for Urban Resilience - OGC Testbed 11
Climate Data Sharing for Urban Resilience - OGC Testbed 11Climate Data Sharing for Urban Resilience - OGC Testbed 11
Climate Data Sharing for Urban Resilience - OGC Testbed 11George Percivall
 
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...nabo_ghea
 
Scoping meeting presentation 4 29-13
Scoping meeting presentation 4 29-13Scoping meeting presentation 4 29-13
Scoping meeting presentation 4 29-13San Diego County Water Authority
 
Facing data sharing in a heterogeneous research community: lights and shadows...
Facing data sharing in a heterogeneous research community: lights and shadows...Facing data sharing in a heterogeneous research community: lights and shadows...
Facing data sharing in a heterogeneous research community: lights and shadows...Research Data Alliance
 
2013 keynote com.geo_reed v2
2013 keynote com.geo_reed v22013 keynote com.geo_reed v2
2013 keynote com.geo_reed v2Carl Reed
 
Nisqually Funding Mechanisms Workshop Part 2
Nisqually Funding Mechanisms Workshop Part 2Nisqually Funding Mechanisms Workshop Part 2
Nisqually Funding Mechanisms Workshop Part 2Nisqually River Council
 
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...GEO Analytics Canada
 
Esri and the Scientific Community
Esri and the Scientific CommunityEsri and the Scientific Community
Esri and the Scientific CommunityDawn Wright
 
The changing landscape of science
The changing landscape of scienceThe changing landscape of science
The changing landscape of scienceChelle Gentemann
 
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark Summit
 
Herring Noaa Spring08
Herring Noaa Spring08Herring Noaa Spring08
Herring Noaa Spring08Internet2 K20 Initiative
 

More Related Content

What's hot

Geothermal Energy - Global Development Status
Geothermal Energy - Global Development StatusGeothermal Energy - Global Development Status
Geothermal Energy - Global Development StatusThinkGeoEnergy
 
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016ThinkGeoEnergy
 
Nha Renewables Mann
Nha Renewables MannNha Renewables Mann
Nha Renewables MannDebbi Smith
 
9. Media Exposure- DNA News
9. Media Exposure- DNA News9. Media Exposure- DNA News
9. Media Exposure- DNA NewsJaymin Gajjar
 

What's hot (7)

Geothermal Energy - Global Development Status
Geothermal Energy - Global Development StatusGeothermal Energy - Global Development Status
Geothermal Energy - Global Development Status
 
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
Keynote presentation at GEORG Geothermal Workshop, Nov. 24, 2016
 
Future Energy Summit 2017 - Environmental Law & Policy Center
Future Energy Summit 2017 - Environmental Law & Policy CenterFuture Energy Summit 2017 - Environmental Law & Policy Center
Future Energy Summit 2017 - Environmental Law & Policy Center
 
Nha Renewables Mann
Nha Renewables MannNha Renewables Mann
Nha Renewables Mann
 
Energy Research and Innovation
Energy Research and InnovationEnergy Research and Innovation
Energy Research and Innovation
 
Presentation
Presentation Presentation
Presentation
 
9. Media Exposure- DNA News
9. Media Exposure- DNA News9. Media Exposure- DNA News
9. Media Exposure- DNA News
 

Similar to Climate Data Lake: Empowering Citizen Scientists in Acadia National Park

Introdution to Landsat and Google Earth Engine
Introdution to Landsat and Google Earth EngineIntrodution to Landsat and Google Earth Engine
Introdution to Landsat and Google Earth EngineVeerachai Tanpipat
 
Improving access to geospatial Big Data in the hydrology domain
Improving access to geospatial Big Data in the hydrology domainImproving access to geospatial Big Data in the hydrology domain
Improving access to geospatial Big Data in the hydrology domainClaudia Vitolo
 
Climate Data Sharing for Urban Resilience - OGC Testbed 11
Climate Data Sharing for Urban Resilience - OGC Testbed 11Climate Data Sharing for Urban Resilience - OGC Testbed 11
Climate Data Sharing for Urban Resilience - OGC Testbed 11George Percivall
 
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...nabo_ghea
 
Facing data sharing in a heterogeneous research community: lights and shadows...
Facing data sharing in a heterogeneous research community: lights and shadows...Facing data sharing in a heterogeneous research community: lights and shadows...
Facing data sharing in a heterogeneous research community: lights and shadows...Research Data Alliance
 
2013 keynote com.geo_reed v2
2013 keynote com.geo_reed v22013 keynote com.geo_reed v2
2013 keynote com.geo_reed v2Carl Reed
 
Nisqually Funding Mechanisms Workshop Part 2
Nisqually Funding Mechanisms Workshop Part 2Nisqually Funding Mechanisms Workshop Part 2
Nisqually Funding Mechanisms Workshop Part 2Nisqually River Council
 
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...GEO Analytics Canada
 
Esri and the Scientific Community
Esri and the Scientific CommunityEsri and the Scientific Community
Esri and the Scientific CommunityDawn Wright
 
The changing landscape of science
The changing landscape of scienceThe changing landscape of science
The changing landscape of scienceChelle Gentemann
 
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark Summit
 
IEDA Overview & Updates, March 2014
IEDA Overview & Updates, March 2014IEDA Overview & Updates, March 2014
IEDA Overview & Updates, March 2014iedadata
 
LPS19 ExtremeEarth Project
LPS19 ExtremeEarth ProjectLPS19 ExtremeEarth Project
LPS19 ExtremeEarth ProjectExtremeEarth
 
Ease Leads to Exposure , Exposure Leads to Adoption
Ease Leads to Exposure , Exposure Leads to AdoptionEase Leads to Exposure , Exposure Leads to Adoption
Ease Leads to Exposure , Exposure Leads to AdoptionDawn Wright
 
'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...
'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...
'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...Justin Gleeson
 
Experiences as a producer, consumer and observer of open data
Experiences as a producer, consumer and observer of open dataExperiences as a producer, consumer and observer of open data
Experiences as a producer, consumer and observer of open dataProgCity
 

Similar to Climate Data Lake: Empowering Citizen Scientists in Acadia National Park (20)

Andy Fox DART
Andy Fox DARTAndy Fox DART
Andy Fox DART
 
ICHEC - Observation systems, technologies and big data
ICHEC - Observation systems, technologies and big dataICHEC - Observation systems, technologies and big data
ICHEC - Observation systems, technologies and big data
 
Introdution to Landsat and Google Earth Engine
Introdution to Landsat and Google Earth EngineIntrodution to Landsat and Google Earth Engine
Introdution to Landsat and Google Earth Engine
 
Improving access to geospatial Big Data in the hydrology domain
Improving access to geospatial Big Data in the hydrology domainImproving access to geospatial Big Data in the hydrology domain
Improving access to geospatial Big Data in the hydrology domain
 
Climate Data Sharing for Urban Resilience - OGC Testbed 11
Climate Data Sharing for Urban Resilience - OGC Testbed 11Climate Data Sharing for Urban Resilience - OGC Testbed 11
Climate Data Sharing for Urban Resilience - OGC Testbed 11
 
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
Managing Social Science Data from the Arctic with ELOKA, ACADIS, NSIDC, and (...
 
Scoping meeting presentation 4 29-13
Scoping meeting presentation 4 29-13Scoping meeting presentation 4 29-13
Scoping meeting presentation 4 29-13
 
Facing data sharing in a heterogeneous research community: lights and shadows...
Facing data sharing in a heterogeneous research community: lights and shadows...Facing data sharing in a heterogeneous research community: lights and shadows...
Facing data sharing in a heterogeneous research community: lights and shadows...
 
2013 keynote com.geo_reed v2
2013 keynote com.geo_reed v22013 keynote com.geo_reed v2
2013 keynote com.geo_reed v2
 
Nisqually Funding Mechanisms Workshop Part 2
Nisqually Funding Mechanisms Workshop Part 2Nisqually Funding Mechanisms Workshop Part 2
Nisqually Funding Mechanisms Workshop Part 2
 
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
Advances in satellite EO data analytics for aquaculture - Aquaculture Canada ...
 
Esri and the Scientific Community
Esri and the Scientific CommunityEsri and the Scientific Community
Esri and the Scientific Community
 
The changing landscape of science
The changing landscape of scienceThe changing landscape of science
The changing landscape of science
 
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
 
Herring Noaa Spring08
Herring Noaa Spring08Herring Noaa Spring08
Herring Noaa Spring08
 
IEDA Overview & Updates, March 2014
IEDA Overview & Updates, March 2014IEDA Overview & Updates, March 2014
IEDA Overview & Updates, March 2014
 
LPS19 ExtremeEarth Project
LPS19 ExtremeEarth ProjectLPS19 ExtremeEarth Project
LPS19 ExtremeEarth Project
 
Ease Leads to Exposure , Exposure Leads to Adoption
Ease Leads to Exposure , Exposure Leads to AdoptionEase Leads to Exposure , Exposure Leads to Adoption
Ease Leads to Exposure , Exposure Leads to Adoption
 
'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...
'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...
'Planning Reform on the island of Ireland: From Policy to Practice', Cooperat...
 
Experiences as a producer, consumer and observer of open data
Experiences as a producer, consumer and observer of open dataExperiences as a producer, consumer and observer of open data
Experiences as a producer, consumer and observer of open data
 

More from Srivatsan Ramanujam

Machine Learning Driven Sales and Marketing for Everyone
Machine Learning Driven Sales and Marketing for EveryoneMachine Learning Driven Sales and Marketing for Everyone
Machine Learning Driven Sales and Marketing for EveryoneSrivatsan Ramanujam
 
Data Science for predictive maintenance in connected vehicles
Data Science for predictive maintenance in connected vehiclesData Science for predictive maintenance in connected vehicles
Data Science for predictive maintenance in connected vehiclesSrivatsan Ramanujam
 
Data Driven Action : A Primer on Data Science
Data Driven Action : A Primer on Data ScienceData Driven Action : A Primer on Data Science
Data Driven Action : A Primer on Data ScienceSrivatsan Ramanujam
 
Analyzing Power of Tweets in Predicting Commodity Futures
Analyzing Power of Tweets in Predicting Commodity FuturesAnalyzing Power of Tweets in Predicting Commodity Futures
Analyzing Power of Tweets in Predicting Commodity FuturesSrivatsan Ramanujam
 
Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal
Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal
Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal Srivatsan Ramanujam
 
A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...
A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...
A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...Srivatsan Ramanujam
 
PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...
PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...
PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...Srivatsan Ramanujam
 
Python Powered Data Science at Pivotal (PyData 2013)
Python Powered Data Science at Pivotal (PyData 2013)Python Powered Data Science at Pivotal (PyData 2013)
Python Powered Data Science at Pivotal (PyData 2013)Srivatsan Ramanujam
 

More from Srivatsan Ramanujam (9)

Machine Learning Driven Sales and Marketing for Everyone
Machine Learning Driven Sales and Marketing for EveryoneMachine Learning Driven Sales and Marketing for Everyone
Machine Learning Driven Sales and Marketing for Everyone
 
Data Science for predictive maintenance in connected vehicles
Data Science for predictive maintenance in connected vehiclesData Science for predictive maintenance in connected vehicles
Data Science for predictive maintenance in connected vehicles
 
All thingspython@pivotal
All thingspython@pivotalAll thingspython@pivotal
All thingspython@pivotal
 
Data Driven Action : A Primer on Data Science
Data Driven Action : A Primer on Data ScienceData Driven Action : A Primer on Data Science
Data Driven Action : A Primer on Data Science
 
Analyzing Power of Tweets in Predicting Commodity Futures
Analyzing Power of Tweets in Predicting Commodity FuturesAnalyzing Power of Tweets in Predicting Commodity Futures
Analyzing Power of Tweets in Predicting Commodity Futures
 
Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal
Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal
Pivotal Data Labs - Technology and Tools in our Data Scientist's Arsenal
 
A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...
A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...
A Pipeline for Distributed Topic and Sentiment Analysis of Tweets on Pivotal ...
 
PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...
PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...
PyMADlib - A Python wrapper for MADlib : in-database, parallel, machine learn...
 
Python Powered Data Science at Pivotal (PyData 2013)
Python Powered Data Science at Pivotal (PyData 2013)Python Powered Data Science at Pivotal (PyData 2013)
Python Powered Data Science at Pivotal (PyData 2013)
 

Recently uploaded

Log Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxLog Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxJohnnyPlasten
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxolyaivanovalion
 
Smarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxSmarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxolyaivanovalion
 
Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxolyaivanovalion
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Researchmichael115558
 
100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptxAnupama Kate
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...SUHANI PANDEY
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfLars Albertsson
 
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceDelhi Call girls
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAroojKhan71
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysismanisha194592
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...amitlee9823
 
Mature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptxMature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptxolyaivanovalion
 
BabyOno dropshipping via API with DroFx.pptx
BabyOno dropshipping via API with DroFx.pptxBabyOno dropshipping via API with DroFx.pptx
BabyOno dropshipping via API with DroFx.pptxolyaivanovalion
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxolyaivanovalion
 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfMarinCaroMartnezBerg
 
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdfMarket Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdfRachmat Ramadhan H
 

Recently uploaded (20)

Log Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxLog Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptx
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 
Smarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptxSmarteg dropshipping via API with DroFx.pptx
Smarteg dropshipping via API with DroFx.pptx
 
Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFx
 
Discover Why Less is More in B2B Research
Discover Why Less is More in B2B ResearchDiscover Why Less is More in B2B Research
Discover Why Less is More in B2B Research
 
100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx
 
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
VIP Model Call Girls Hinjewadi ( Pune ) Call ON 8005736733 Starting From 5K t...
 
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get CytotecAbortion pills in Doha Qatar (+966572737505 ! Get Cytotec
Abortion pills in Doha Qatar (+966572737505 ! Get Cytotec
 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
 
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort ServiceBDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
Sampling (random) method and Non random.ppt
Sampling (random) method and Non random.pptSampling (random) method and Non random.ppt
Sampling (random) method and Non random.ppt
 
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
Junnasandra Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore...
 
Mature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptxMature dropshipping via API with DroFx.pptx
Mature dropshipping via API with DroFx.pptx
 
BabyOno dropshipping via API with DroFx.pptx
BabyOno dropshipping via API with DroFx.pptxBabyOno dropshipping via API with DroFx.pptx
BabyOno dropshipping via API with DroFx.pptx
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptx
 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdf
 
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdfMarket Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
 

Climate Data Lake: Empowering Citizen Scientists in Acadia National Park

  • 1. 1© Copyright 2013 Pivotal. All rights reserved. 1© Copyright 2013 Pivotal. All rights reserved. Climate Data Lake: Empowering Citizen Scientists in Acadia National Park Srivatsan Ramanujam Principal Data Scientist, Pivotal @being_bayesian John Cardente Distinguished Engineer/Lead Data Scientist, EMC @johncardente 19-Feb-2015
  • 2. 2© Copyright 2013 Pivotal. All rights reserved. 2© Copyright 2013 Pivotal. All rights reserved. Climate Change is happening Source: NASA
  • 3. 3© Copyright 2013 Pivotal. All rights reserved. Source: National Geographic At the Ice Caps… From Chasing Ice by James Balog
  • 4. 4© Copyright 2013 Pivotal. All rights reserved. And in the Oceans…
  • 5. 5© Copyright 2013 Pivotal. All rights reserved. 5© Copyright 2013 Pivotal. All rights reserved. Weather is Nature’s clock
  • 6. 6© Copyright 2013 Pivotal. All rights reserved. 6© Copyright 2013 Pivotal. All rights reserved. And the clock is shifting Source: EPA
  • 7. 7© Copyright 2013 Pivotal. All rights reserved. 7© Copyright 2013 Pivotal. All rights reserved. English Oak Winter Moth Pied Flycatcher Both et al. 2006 Nature EARLIER SAME TIME EACH YEAR Disrupting species interactions
  • 8. 8© Copyright 2013 Pivotal. All rights reserved. 8© Copyright 2013 Pivotal. All rights reserved. …the study of the timing of recurring plant and animal life- cycle stages, or phenophases, and their relationship to environmental conditions. Phenology Source: USA National Phenology Network
  • 9. 9© Copyright 2013 Pivotal. All rights reserved. 9© Copyright 2013 Pivotal. All rights reserved. Acadia National Park Source: Wikipedia A key phenology observation point
  • 10. 10© Copyright 2013 Pivotal. All rights reserved. 10© Copyright 2013 Pivotal. All rights reserved. Citizen Science, the “Hadoop” of Phenology
  • 11. 11© Copyright 2013 Pivotal. All rights reserved. Challenges impeding scientific progress  A multitude of disparate data sources  Different approaches to data conditioning  Different analysis platforms and scripts Inconsistent data treatment distracts the scientific debate from the findings!
  • 12. 12© Copyright 2013 Pivotal. All rights reserved. Challenges to encouraging citizen science  Different infrastructures for data collection  Unable to see how contributions help science  Inability to analyze the data and collaborate with scientists Missing a feedback loop to encourage and engage citizen scientists
  • 13. 13© Copyright 2013 Pivotal. All rights reserved. EMC Federation teams up with Earthwatch
  • 14. 14© Copyright 2013 Pivotal. All rights reserved. Objectives • Authoritative, curated data repository • Shared analysis platform and software libraries • Web portals to – Show citizen scientists how contributions benefit science – Present data visualization and exploration tools – Provide access to data sets for volunteer analysis – Offer common data collection APIs and interfaces – Host forums to interact with leading scientists
  • 15. 15© Copyright 2013 Pivotal. All rights reserved. Climate Data Lake PL/X Hadoop++ Data Data Lake Foundation Explore CloudFoundry Collect Isilon ECS
  • 16. 16© Copyright 2013 Pivotal. All rights reserved. The kick-off expedition at Acadia
  • 17. 17© Copyright 2013 Pivotal. All rights reserved. Post kick-off activities • Established partnerships with data providers • Stood up initial Data Lake platform • Active collaboration across all participants on data conditioning, analysis, feature selection, and visualization • Launching an initial web presence in March
  • 18. 18© Copyright 2013 Pivotal. All rights reserved. What’s next • Pursuing the “full vision” using Federation Technologies • Talking with other potential corporate sponsors • Exploring ways to expand support to other climate and academic projects
  • 19. 19© Copyright 2013 Pivotal. All rights reserved. 19© Copyright 2013 Pivotal. All rights reserved.
  • 20. 20© Copyright 2013 Pivotal. All rights reserved. Learn More • White House Press Release: Harnessing Climate Data to Boost Ecosystem & Water Resilience • Pivotal Blog: Pivotal Data Science Looks at Climate Change @ Acadia NP • EMC Reflections Blog: Big Data vs. Climate Change BlogsEMC TV video
  • 21. 21© Copyright 2013 Pivotal. All rights reserved. The Team @alyssacaddle1 @stevendozen @johncardente @lilewite @being_bayesian @maryanngarwood @emctv @TheMatthewNolan @johnnysuire @sriauggie31 @earthwatch_org @schoodicinst @hwschoodic + many more (not on twitter ) !

Editor's Notes

  1. John
  2. John
  3. John
  4. John
  5. Vatsan
  6. Vatsan
  7. Vatsan
  8. Vatsan
  9. John
  10. John
  11. Vatsan
  12. John
  13. John, Vatsan
  14. Plug in blog links
  15. John
  16. John
  17. John
  18. John