Talk at GeoPython conference about mapping data in Jupyter notebooks with PixieDust

1. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Mapping Data in Jupyter
Notebooks with PixieDust
Margriet Groenendijk & Raj Singh
GeoPython - 10 May 2017 - Basel, Switzerland

2. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
from mpl_toolkits.basemap import Basemap
from matplotlib.offsetbox import AnnotationBbox, OffsetImage
from matplotlib._png import read_png
from itertools import izip
matplotlib.style.use('bmh')
fig, axes = plt.subplots(nrows=1, ncols=2,
figsize=(10, 12))
# background maps
m1 = Basemap(projection='mill',resolution=None,
llcrnrlon=-7.5, llcrnrlat=49.84,urcrnrlon=2.5,
urcrnrlat=59,ax=axes[0])
m1.drawlsmask(land_color='dimgrey',
ocean_color='dodgerBlue',lakes=True)
# temperature map
for [temp,city] in izip(temps,cities):
lat = city[1]
lon = city[2]
if temp>8:
col='indigo'
...
elif temp>4:
col='turquoise'
x1, y1 = m2(lon,lat)
bbox_props = dict(boxstyle="round,pad=0.3",
fc=col, ec=col, lw=2)
axes[1].text(x1, y1, temp, ha="center", va="center",
size=11,bbox=bbox_props)
plt.tight_layout()
https://www.flickr.com/photos/mjtmail/2170840714

3. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
PixieDust with Mapbox

4. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Watson Data Platform
Weather Machine Learning Watson APIsTwitter Geo

5. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

6. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

7. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
IBM Data Science Experience + Project
Jupyter

8. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

9. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr https://medium.com/ibm-watson-data-lab

10. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Jupyter + Pixiedust =
1. PackageManager
2. Visualizations
3. Cloud Integration
4. Scala Bridge
5. Extensibility
6. Embedded Apps

11. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Package Manager
Uses the GraphFrame Python APIs
Install GraphFrames Spark Package
Install Spark packages or plain jars

12. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Visualizations One simple API: display()
Call the Options dialog
Panning/Zooming
options
Performance statistics

13. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Cloud Integration
Easily export your data to csv, json, html, etc. locally on your laptop
or into a cloud-based service like Cloudant or Object Storage

14. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Scala Bridge
Execute Scala code directly from your python Notebook
Define a Python variable
Use the Python var in Scala
Define a Scala variable
Use the Scala var in Python

15. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Extensibility
Easily extend PixieDust to create your own visualizations using
HTML/CSS/JavaScript
Customized
visualisation for
GraphFrame
Graphs

16. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Embed Apps in Notebooks
PixieApps encapsulate analytics into lightweight HTML UIs

17. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
PixieDust demo
https://apsportal.ibm.com/analytics/notebooks/
e76cad7a-30f7-493f-9ea0-7e14be178053/view?
access_token=acc140772a62646df3bd2987ca4a8463c0f8169a
02c9fe6bafc0311fadca85f2

18. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

19. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
How it all works
• Spark DataFrame -> GeoJSON
• /display/chart/renderers/mapbox/mapBoxMapDisplay.py
• Get bin cutoff points for quantiles
• /display/chart/renderers/mapbox/mapBoxMapDisplay.py
• Create choropleth styling JSON
• /display/chart/renderers/mapbox/mapBoxMapDisplay.py
• GeoJSON data and styling JSON => Jinja2 template
• /display/chart/renderers/mapbox/templates/mapView.html
• Render template inside an <iframe> inside the cell
• /display/chart/renderers/mapbox/templates/iframesrcdoc.html
• Call Mapbox basemapping service for streets underlay
• /display/chart/renderers/mapbox/templates/mapView.html

20. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

21. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Where next with PixieDust?
• Polygon support?
• Grid/raster support?
• More cartographic options?
• Animated temporal visualization?
• arcgis?
• Bokeh maps?
• Folium?
• … any visualisation you want!

22. Register now!
seti.org/ML4SETI
from The SETI Institute
Hackathon
Galvanize, San Francisco
June 10-11, 2017
Code Challenge
June/July, 2017

23. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

24. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
References
• IBM Data Science Experience - https://datascience.ibm.com
• Pixiedust - https://github.com/ibm-cds-labs/pixiedust
• Project Jupyter - http://jupyter.org/
• Slides - https://www.slideshare.net/MargrietGroenendijk/presentations
• Blog - https://medium.com/ibm-watson-data-lab
• Me - mgroenen@uk.ibm.com - @MargrietGr