JSON-stat in the session "The future of standards in statistics", United Nations, Geneva

JSON-stat
standards in statistics
Workshop on International Collaboration
for Standards-Based Modernisation
United Nations Economic Commission for Europe
Geneva, 7 May 2015
The future of
Xavier Badosa
Statistical Institute of Catalonia
}

standards
as conversation enablers

standards
as conversation enablers
What
Who
Why

standards in statistics
What
Who
Why

Cubic
Model
Describe data
in dimension terms

API
data
as an
infrastructure
NSO as a
platform

A B
Shared environment
E
C
H
A
N
G
E

Cubic
Simple
General
Light
<?xml version='1.0' encoding='UTF-8'?>
<message:GenericData
xmlns:footer="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message/footer"
xmlns:generic="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/data/generic"
xmlns:common="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common"
xmlns:message="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xml="http://www.w3.org/XML/1998/namespace">
<message:Header>
<message:ID>45a9f625f3887526f72e5341577e13e5</message:ID>
<message:Test>false</message:Test>
<message:Prepared>2015-04-20T13:37:23</message:Prepared>
<message:Sender id="ESTAT">
<common:Name xml:lang="en">Eurostat</common:Name>
<message:Timezone>+01:00</message:Timezone>
</message:Sender>
<message:Receiver id="RECEIVER"/>
<message:Structure structureID="ESTAT_DSD_cdh_e_fos_1_0“
dimensionAtObservation="TIME_PERIOD">
<common:Structure>
<Ref agencyID="ESTAT" id="DSD_cdh_e_fos" version="1.0"/>
</common:Structure>
</message:Structure>
<message:DataSetAction>Append</message:DataSetAction>
<message:DataSetID>cdh_e_fos</message:DataSetID>
</message:Header>
<message:DataSet structureRef="ESTAT_DSD_cdh_e_fos_1_0">
<generic:Series>
<generic:SeriesKey>
<generic:Value id="UNIT" value="PC"/>
<generic:Value id="Y_GRAD" value="TOTAL"/>
<generic:Value id="FOS07" value="FOS1"/>
<generic:Value id="GEO" value="BE"/>
<generic:Value id="FREQ" value="A"/>
</generic:SeriesKey>
<generic:Obs>
<generic:ObsDimension value="2009"/>
<generic:ObsValue value="NaN"/>
<generic:Attributes>
<generic:Value id="OBS_STATUS" value="na"/>
</generic:Attributes>
</generic:Obs>
<generic:Obs>
</generic:Obs>
</generic:Series>
<generic:Series>
<generic:SeriesKey>
<generic:Value id="UNIT" value="PC"/>
<generic:Value id="Y_GRAD" value="Y_GE1990"/>
<generic:Value id="FOS07" value="FOS1"/>
<generic:Value id="GEO" value="BE"/>
<generic:Value id="FREQ" value="A"/>
</generic:SeriesKey>
<generic:Obs>
<generic:ObsValue value="43.75"/>
</generic:Obs>
<generic:Obs>
</generic:Obs>
</generic:Series>
</message:DataSet>
</message:GenericData>
SDMX-ML

[
{
"area_name": "Panama",
"measurement": "tonnes",
"value": 1152.87890625,
"year": 2007,
"footnotes": [...]
},
...
]

[
{...},
[
{
"indicator": {
"id": "NY.GDP.MKTP.CD",
"value": "GDP (current US$)"
},
"country": { "id": "BR", "value": "Brazil" },
"value": "1620165226993.77",
"decimal": "0",
"date": "2009"
},
...
]
]

{
"JSONDataResult": [
{
"AgencyName": "MCC",
"Amount": "-6983525",
"BenefitingLocation": "Ghana",
"Category": "Economic Development",
"FiscalYear": "2013",
"Sector": "Agriculture"
},
...
]
}

{
...,
"Results": [{
"series": [{
"seriesID": "LAUCN040010000000005",
"data": [
{
"year": "2013",
"period": "M11",
"periodName": "November",
"value": "16393",
"footnotes": [...]
},
...
]
}]
}]
}

{
"BEAAPI": {
"Request": {...},
"Results": {
"Statistic": "Per capita personal income",
"UnitOfMeasure":"dollars",
"PublicTable":"CA1-3 Personal income summary",
"UTCProductionTime":"2014-05-11T17:02:55.817",
"NoteRef":"2", "Dimensions": [...],
"Data": [
{
"GeoFips":"00000",
"GeoName":"United States",
"Code":"PCPI_CI",
"TimePeriod":"2009",
"CL_UNIT":"USD",
"UNIT_MULT":"0",
"DataValue":"39357"
},
...
], "Notes": [...]
}
}
}

{
"BEAAPI": {
"Request": {...},
"Results": {
"Data": [
{
"GeoFips":"00000",
"Code":"PCPI_CI",
"CL_UNIT":"USD",
"UNIT_MULT":"0",
"DataValue":"39357"
},
...
], "Notes": [...]
}
}
}
General

{
"BEAAPI": {
"Request": {...},
"Results": {
"Data": [
{
"GeoFips":"00000",
"Code":"PCPI_CI",
"CL_UNIT":"USD",
"UNIT_MULT":"0",
"DataValue":"39357"
},
...
], "Notes": [...]
}
}
}
General Light?

{
"BEAAPI": {
"Request": {...},
"Results": {
"Data": [
{
"GeoFips":"00000",
"Code":"PCPI_CI",
"CL_UNIT":"USD",
"UNIT_MULT":"0",
"DataValue":"39357"
},
...
], "Notes": [...]
}
}
}
General Light?Cubic

JSON-stat
1. Keeps metadata and data apart
Cubic
Simple
General
Light

{
"columns": [
{ "code":"region", "text": "Region" },
{ "code":"ageG5", "text":"Age", "comment": ... },
{ "code":"period", "text":"Time", "type":"t" },
{ "code":"x", "text":"Population", "type":"c", "unit":"amount" }
],
"comments": [...],
"data":[
{ "key": ["02","0-7","2003"], "values": [ 100 ] },
{ "key": ["02","0-7","2004"], "values": [ 101 ] },
...
]
}

"dataSets" : [
{
"action" : "Information",
"series" : {
"0:0:0:0" : {
"attributes" : [0, 0, 0],
"observations" : {
"0" : [100.0, null],
"1" : [103.3038, null],
"2" : [105.1249, null],
"3" : [107.7003, null]
}
},
"0:0:0:1" : {
"attributes" : [0, 0, 0],
"observations" : {
...

JSON-stat
2. Avoids indices
Cubic
Simple
General
Light

[
]
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
"value" :

{
"dataset": {
"label": "Value (NOK 1 000) by imports/exports...",
"source": "Statistics Norway",
"updated": "2014-05-13T18:36:18Z",
"dimension": {...},
"value": [ 77287250, 70377617, 73499048, ... ],
...
}
}

function arr2num( arr, size ){
for(var i=0, num=0, mult=1, ndims=size.length; i<ndims; i++){
mult*=(i>0) ? size[ndims-i] : 1;
num+=mult*arr[ndims-i-1];
}
return num;
}
Maths is
what computers
do best
Row-major order

JSON-stat
2. Avoids indices
3. Has a simple ontology
Cubic
Simple
General
Light

dataset
dimension
category
value

JSON-stat
2. Avoids indices
3. Has a simple ontology
Cubic
Simple
General
Light4. Requires very few properties

dimension
category
value
id
size
index or label
(from 27 reserved words)

buildable from existing systems

JSON-stat.org
a simple light standard
for all kinds of data disseminators

http://bl.ocks.org/badosa
Examples

Thank you
Xavier Badosa
@badosa
JSON-stat
@jsonstat
Statistical Institute
of Catalonia
(Idescat)

Credits
“Soma” (blocks’ background)
by Dru! (CC BY–NC)
“Deep in conversation” (bar conversation)
by Ross Pollack (BY–NC–SA)
“Metal movable type”
by Willi Heidelbach (CC BY–SA)
“Portrait” (cubic head)
by Thomas Leth-Olsen (CC BY)
“Sterile” (walking girl)
by Lee Nachtigal (CC BY)
“Railroad”
by Xavier Badosa (CC BY)
“Dartboard”
by Jacob Vance (CC BY–NC)

JSON-stat in the session "The future of standards in statistics", United Nations, Geneva

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (20)

Similar a JSON-stat in the session "The future of standards in statistics", United Nations, Geneva

Similar a JSON-stat in the session "The future of standards in statistics", United Nations, Geneva (20)

Más de Xavier Badosa

Más de Xavier Badosa (10)

Último

Último (20)

JSON-stat in the session "The future of standards in statistics", United Nations, Geneva