The document discusses how the City of Chicago has implemented a NoSQL database to store and analyze large amounts of municipal data. Some key points discussed include how the database works well for geospatial data, can grow elastically, uses a loose schema, and allows the city to bring all data into one place, make it available in real time, and find relationships within the data. Challenges mentioned include issues with certain geospatial queries and safely scaling the database.

1. Municipal
Government
meets NoSQL
Ryan Briones
IT Director - Software Architecture and Design
Department of Innovation and Technology
City of Chicago
Tuesday, November 13, 12

2. This is my boss
Brett Goldstein
Chief Data Officer
Commissioner, DoIT (CIO)
Tuesday, November 13, 12

3. This is my boss
Brett Goldstein
Chief Data Officer
Commissioner, DoIT (CIO)
Data Junkie
Tuesday, November 13, 12

4. Crazy Ideas
My Job
Reality
Tuesday, November 13, 12

5. Tuesday, November 13, 12

6. 2010
Tuesday, November 13, 12

7. 2011
Tuesday, November 13, 12

8. 2011
Tuesday, November 13, 12

9. 2011 How do we tell the story
about a place?
Tuesday, November 13, 12

10. How do we tell the story
about a place?
911 CPS CTA CFD 311
Tuesday, November 13, 12

11. 2012
Tuesday, November 13, 12

12. 2012 How can we use realtime data
to make better decisions?
Tuesday, November 13, 12

13. Bring all the data all in one place
Make it available in realtime
Find relationships in the data
Tuesday, November 13, 12

14. Tuesday, November 13, 12

15. Spatial / Temporal
Tuesday, November 13, 12

16. Spatial / Temporal
Tuesday, November 13, 12

17. Tuesday, November 13, 12

18. Works well with geospatial data
Grow elastically
Loose schema
Tuesday, November 13, 12

19. Tuesday, November 13, 12

20. Works well with geospatial data
Grow elastically
Loose schema
Tuesday, November 13, 12

21. Works well with geospatial data
Grow elastically
Loose schema
Tuesday, November 13, 12

22. {
pos: [41.884267623865384,
-87.62977480888367]
}
/* ... */
db.windygrid.ensureIndex({pos: “2d”})
/* ... */
db.windygrid.find({ $within: { $box:
[ [41.88227, -87.63208]
[41.88499, -87.62838] ] }})
Tuesday, November 13, 12

23. Works well with geospatial data
Grow elastically
Loose schema
Tuesday, November 13, 12

24. 69GB data Router
200K objs
Primary Shard 1 Primary Shard 2 Primary Shard 3
C
Secondary Shard 1 Secondary Shard 2 Secondary Shard 3
C
Secondary Shard 1 Secondary Shard 2 Secondary Shard 3
C
Tuesday, November 13, 12

25. Works well with geospatial data
Grow elastically
Loose schema
Tuesday, November 13, 12

26. {
type: “311”,
shardkey: “2a08c28b3ca3bb65bd...”
where: { location: [-180,180], ... },
when: { shardtime: 0, ... },
what: { ... },
who: { ... }
}
Tuesday, November 13, 12

27. Bring all the data all in one place
Make it available in realtime
Find relationships in the data
Tuesday, November 13, 12

28. Bring all the data all in one place
Make it available in realtime
Find relationships in the data
Tuesday, November 13, 12

29. Tuesday, November 13, 12

30. Tuesday, November 13, 12

31. Bring all the data all in one place
Make it available in realtime
Find relationships in the data
Tuesday, November 13, 12

32. Tuesday, November 13, 12

33. Challenges
$within/$box
safe scaling
Tuesday, November 13, 12

34. The Future
Predictive Analytics
Natural-language Processing
Machine Learning
Distributed Computing
Visualizations
Tuesday, November 13, 12

35. github.com/Chicago
github.com/ryanbriones
@ChicagoCDO
@ryanbriones
Ryan Briones
IT Director - Software
Architecture and Design
Department of Innovation
and Technology
City of Chicago
Tuesday, November 13, 12