Presentation on the Data Cube vocabulary to support Linked Data publication of statistics and measurement data sets. Given at SemTech 2011, San Francisco.

1. Linked data hypercubes Dave Reynolds, Epimorphics Ltd

3. Linked Data - great for describing “things” data Government departments Posts and reporting structure e.g. Ministers and senior civil servants

4. Linked Data - great for describing “things” data model choose entities to model department, post, post-holder ... Ontologies org:, foaf: ... domain specific – gov: URI scheme reference data to link to

5. Linked Data - great for describing “things” data model publish convert to RDF in a triple store entity URIs as linked data SPARQL endpoint Linked data API

6. Linked Data - great for describing “things” data model publish use

7. But what about ... data Government budget analysis local authority spend with suppliers regional demographic trends performance metrics air quality measurements energy consumption

8. Publishing tabular data as linked data ? why? how? does it work?

9. Benefits data slices and values becomes addressable annotate, explain, qualify values provenance for values trace back for derived reports integrate, compare, slice across datasets common terms for dimensions and units common identifiers for values (regions, departments ...) link to non-tabular data put the data in context

10. Data cube vocabulary collaborative development sponsored by data.gov.uk simple, flexible vocabulary mirrors core information models from: SDMX (Statistical Data and Metadata eXchange) DDI (Data Documentation Initiative) extension to SCOVO vocabulary

12. Data cube vocabulary1. Top level DataSet provenance and metadata structure qb:component qb:DataStructureDefinition qb:sliceKey qb:structure qb:SliceKey qb:DataSet qb:slice qb:sliceStructure qb:dataset qb:Slice qb:subSlice qb:observation qb:Observation dimension valuesmeasure value(s) attribute values

13. Data cube vocabulary1. Top level DataSet provenance and metadata structure Observation measured values, at dimensions with attributes direct link to DataSet qb:component qb:DataStructureDefinition qb:sliceKey qb:structure qb:SliceKey qb:DataSet qb:slice qb:sliceStructure qb:dataset qb:Slice qb:subSlice qb:observation qb:Observation dimension valuesmeasure value(s) attribute values

14. Data cube vocabulary1. Top level DataSet provenance and metadata structure Observation measured values, at dimensions with attributes direct link to DataSet Slice optional grouping by fixing dimensions guide to presentation allows for abbreviated data qb:component qb:DataStructureDefinition qb:sliceKey qb:structure qb:SliceKey qb:DataSet qb:slice qb:sliceStructure qb:dataset qb:Slice qb:subSlice qb:observation qb:Observation dimension valuesmeasure value(s) attribute values

15. Data cube vocabulary2. Data Structure Definition explicit definition of cube structure, inline in the data enables validation visualization discovery abbreviation still open world qb:DataSet qb:structure qb:DataStructureDefinition qb:component qb:ComponentSpecification qb:componentRequired qb:componentAttachment qb:order qb:dimension qb:measure qb:attribute

16. Data cube vocabulary3. Coding values numeric or symbolic explicit link to coding scheme allows for hierarchical codes SDMX coding schemes and role markers available qb:ComponentProperty qb:concept qb:DimensionProperty qb:measureType skos:Concept qb:AttributeProperty sdmx:Concept qb:MeasureProperty sdmx:ConceptRole qb:CodedProperty qb:codeList sdmx:FrequencyRolesdmx:CountRolesdmx:EntityRolesdmx:TimeRole sdmx:MeasureTypeRole sdmx:NonObsTimeRole sdmx:IdentityRole sdmx:PrimaryMeasureRole skos:ConceptScheme sdmx:CodeList

17. Example eg:dsd-le a qb:DataStructureDefinition; # The dimensions qb:component [qb:dimension eg:refArea; qb:order 1]; qb:component [qb:dimension eg:refPeriod; qb:order 2]; qb:component [qb:dimension sdmx-dimension:sex; qb:order 3]; # The measure(s) qb:component [qb:measure eg:lifeExpectancy]; # The attributes qb:component [qb:attribute sdmx-attribute:unitMeasure; qb:componentAttachment qb:DataSet;] . eg:dataset-le1 a qb:DataSet; rdfs:label "Life expectancy"@en; rdfs:comment "Life expectancy in Welsh Unitary authorities"@en; qb:structure eg:dsd-le ; sdmx-attribute:unitMeasure <http://dbpedia.org/resource/Year> . eg:o1 a qb:Observation; qb:dataset eg:dataset-le1 ; eg:refArea admingeo:newport_00pr ; eg:refPeriod <http://reference.data.gov.uk/id/year/2004> ; sdmx-dimension:sex sdmx-code:sex-M ; eg:lifeExpectancy 76.7 .

18. Case study: Local government payments data UK local authorities publish data on all spending above £500 linked data version to enable comparison

19. Case study: Local government payments data model cube structure measure amount net of recoverable VAT attributes currency dimensions time payer payee expenditure code item package as an ontology

20. Case study: Local government payments data model publish visualizations LD API API structure mirrors cube dimensional structure

21. Case study: Local government payments data model publish use

23. samples

24. assay

26. Case study: Environmental monitoring data model publish visualizations LD API API structure mirrors cube dimensional structure

27. Case study: Environmental monitoring data model publish use

28. Data Cube : Summary foundational approach to publishing multi-dimensional data as linked data enables addressing – annotate, explain, provenance, context integration – slice, dice and compare across sets puts data in context explicit declarative structure => validation discovery automation - web APIs, visualizations, exploration tools

29. Acknowledgements John Sheridan (The National Archive) for sponsoring the development of data cube Richard Cyganiak, JeniTennison co-developers of the data cube vocabulary JeniTennison, Dan Smith developers of the organogram example Stuart Williams, Ian Dickinson developers of the bathing water use case Photos: lesteph @ flickrdullhunk @ flickr Martin Pettitt @ flickr kikasso @ flickrTax_Rebate @ flikr