Using Google AppEngine and Fusion Tables with ArcGIS Server

1. Using Google AppEngine and Fusion Tables with ArcGIS Server Brian Flood Arc2Earth

2. Interoperability with ArcGIS Server REST API Using ArcGIS Server for Cloud geoprocessing

3. Google AppEngine Google Fusion Tables ArcGIS Server – Geometry Service Arc2Cloud

4. Google Fusion Tables Basically, a Cloud database Storage/Query A NoSQLDB that acts like a SQL one Visualization Maps, Maps API support (really fast) Charts, timelines, etc Merge Tables – Collaborative Analysis HTTP API Spatial and Attribute Query Read/Write Others SimpleGeo ArcGIS FeatureLayer AppEngineDatastore Fusion Tables

5. Fusion Tables - Storage SQL Syntax not quite but close enough SELECT, GROUP BY, Filters BBOX and Radius Search need to filter in userland for polylines/polygons “contains ignoring case” – poor man’s full text search Read/Write Inserts, Bulk Inserts, Updates, Deletes rowidas primary key, a little wonky BigTable/Megastore backend Huge Scale Hilbert curve for spatial index Spatial and attribute portions of query execute in parallel, results merged (“all data near NYC with four stars”)

6. Fusion Tables - Visualization Maps Dynamic Map Tiles – really fast Google Maps API support Could be used from any client api that supports tiles Renderer Styles Info Window templates Simple Heatmaps Timelines Charts Table View

7. Fusion Tables – Why? Instant Scalability Zero Configuration Zero Maintenance Premium Access from Google Google Infrastructure

8. GeoServices REST Specification Published by ESRI Open Source Spec, not implementation Make your map services look like ArcGIS Server Interoperable ArcGIS.com ArcGIS Explorer AGS Mobile Apps

9. Can we make Fusion Tables look like an ArcGIS Server?

10. Arc2Cloud Additional geospatial functionality for Fusion Tables Google AppEngine Datasources Maps FeatureCache Native API Fusion Tables Tilesets Geospatial REST Spec OAuth Services Task Queue WFS & WFS/T Analysis Multi Tenant Config Sites

11. Arc2Cloud Each feature has an endpoint and multiple formats. Feature attachments (photos etc) are stored in the AppEngineblobstore instance id fusion table arc2cloud.com/brian8/data/datasources/Counties/101?f=html Counties ?f=json ?f=gjson ?f=kml ?f=gml ?f=wkt ?f=qr /brian9 /NJState /census /a2eProjects /whateveryouwant Fusion Tables States .../brian8/ArcGIS/rest/services/Census/FeatureServer/0/101 /attachments /addAttachment ../query ../addFeatures etc ESRI REST

12. Arc2Cloud Admin

13. Demo – Import Map

14. Google AppEngine PaaS – Platform as a Service Zero Configuration Auto Scaling Utility Based Billing Only pay for what you use Python or Java Lots of secondary languages/frameworks can be compiled to Java Task Queue – Parallel Execution

15. Google AppEngine 400 requests/second

16. How can we leverage AppEngine’s Task Queue with ArcGIS Server’s geoprocessing?

17. Task Queue Geoprocessing Experimental! Why? Basic full layer geoprocessing without desktop software Merging Fusion Tables based on location is powerful Fun project MapReduce-like processing Work on many feature collections in parallel Common overlay/proximity operations between FT Tables Buffer, Merge, Join, Clip, Snap etc Divide and conquer Use ArcGIS Server to perform spatial operations Write results back to FT in parallel

18. Prepare yourself for programmer artwork

20. Ranges

22. Merge

23. Union DS: States Range: 10-19

25. Ranges

28. Edit Feed

29. WebHooksEdit Request

31. Merge

32. UnionArcGIS Server DS: States Range: 10-19

33. Phase 4 – Store Results Task Queue (parallel execution) Running in same Task as Phase2 Write results back to newly create Fusion Table Concurrent writes to Fusion Tables can be slower (seconds) Google Premium access will help with this DS: Counties Range: 0-9 DS: Counties Range: 10-19 Fusion Tables

34. Demo SpatialJoin between two Fusion Tables

35. Pros/Cons Distributed dataset analysis is more attractive then snapshots sent to GP server Analysis results are immediately available as a Fusion Table Tasks are idempotent, small errors will be re-queued Lots of bandwidth consumed Oversampling of data in grid strategy (although FeatureCache access helps) Tried and true algorithms needs to be tweeked/changed <- showstopper?

36. Questions? google.com/fusiontables Arc2Earth.com Arc2Cloud.com @bFlood on twitter videos of demos: screenr.com/user/bFlood