This document discusses pre-aggregating data with counters in MongoDB to enable dashboard-style reports. It involves defining metrics and aggregations upfront, and using the $inc operator to increment values in aggregation collections during write operations. This allows generating real-time aggregated data without requiring map-reduce or the aggregation framework, providing better performance for dashboard queries.
3. Framework
• Know your metrics/counter
• Prepared reports
• Calculate during write
• Fast queries
• Always up to date data
• Record time-series collections
4. Rationale
• Documents are updated in-place*
• $inc update operator
• Working set is small
• Aggregations are much smaller*
7. Roads not traveled
• Map/Reduce
• Reprocess raw data
• Now possible to do partial reduce
• Aggregation Framework (aggregate in 2.2)
• Also reprocess data on operation (initial release)
• Optimizations to come
• More costly during reads
8. Not Appropriate For
• Ad-hoc aggregations (unknown metrics)
• One-off reports
• Possibly complex calculations
9. Processing
• Event received
• Split into many updates w/$inc
• Aggregate
• Input Field(s)
• Time periods (hourly, daily, monthly)
• Defined Metrics
17. The Whys
• Writing more data up front, helps with reads
• Multiple data points per document
• Documents hold many timed points
• Good for graphs by time, or types
• Nested for improved performance