Intro to Crossfilter for the Madison Javascript meetup

1. Crossfilter
Fast Multidimensional Filtering for Coordinated Views

2. Data management and visualization consultant
!
Project to develop a general purpose
collaborative data management, transformation,
and visualization platform
(GPCDMTVP for short).
!
You can find me at
http://esjewett.com or @esjewett”

3. We’ve got problems

4. More
specifically
We’ve got data

5. and filters

6. and aggregations

7. and speed

8. Demo
Moritz Stefaner’s “Elastic Lists” experiment

9. Demo review
•
We control the data
•
Many different simultaneous filters
•
Aggregation: count
•
Data in the browser … in Flash

10. What would a generalpurpose approach to this
problem look like?
Let’s just call it … say … Crossfilter

11. Javascript
(am I at the right meetup?)

12. Data is encapsulated
!
var
!!
!!
!!
!!
data = [ { date: ‘2014-01-01’,!
! ! ! ! ! value: 10,!
! ! ! ! ! color: ‘orange’ },!
! !
! { … },!
! ! ! !
… ];!
!
var transactions = crossfilter(data);!

13. and dimensional
!
var dateDim = transactions.dimension( !
function (d) {!
return “” + d.date;!!
}!
);!

14. Filter on dimensions
!
dateDim.filter(“2014-01-01”);!
!
!
!
!
!
!
!

15. Filter on dimensions
!
dateDim.filter(“2014-01-01”);!
dateDim.filter([“2013-01-01”,”2014-01-01”]);!
!
!
!
!
!
!

16. Filter on dimensions
!
dateDim.filter(“2014-01-01”);!
dateDim.filter([“2013-01-01”,”2014-01-01”]);!
dateDim.filter( !
function (d) {!
return d === “2013-06-01”;!
}!
);!
!

17. Filter on dimensions
!
dateDim.filter(“2014-01-01”);!
dateDim.filter([“2013-01-01”,”2014-01-01”]);!
dateDim.filter( !
function (d) {!
return d === “2013-06-01”;!
}!
);!
dateDim.filterAll();!

18. Aggregate on dimensions
!
// Count transactions per day!
var dateGroup = dateDim.group();!

19. Aggregate on dimensions
!
// Count transactions per month!
var monthGroup = dateDim.group(!
function (d) {!
return d.substr(1,7);!
}!
);!

20. Aggregate on dimensions
!
// Sum by value over groups of days!
dateGroup.reduceSum(!
function (d) {!
// “d” is the complete record!
return d.value;!
}!
);!

21. “Queries”
!
// Month with most activity under current filter!
monthGroup.top(1);!
// { key: “2013-06”, value: 435 }!
!
// Day with highest value under current filter!
dateGroup.top(1);!
// { key: “2013-12-24”, value: 143700 }!
!
// Get all the months values under current filter!
monthGroup.all();!
// [ { key: “2013-06”, value: 435 },!
//
{ key: “2013-12”, value: 315 },!
//
{ key: “2013-02”, value: 250 }, … ]!

22. So what about speed?

23. Demo

24. Some notes
•
Dimension accessors must return naturally ordered
values. Cast before returning!
•
reduce(add, remove, initial)
•
order(ordering)
•
groupAll()

25. Ethan Jewett!
esjewett.com / coredatra.com
@esjewett
https://github.com/esjewett

26. Links
•
Crossfilter - http://square.github.io/crossfilter/
•
Crossfilter API - https://github.com/square/
crossfilter/wiki/API-Reference
•
Moritz Stefaner’s Elastic Lists - http://
moritz.stefaner.eu/projects/elastic-lists/