2016 SDMX Experts meeting, 17-20 October 2016, Aguascalientes, Mexico. More information at: http://www.oecd.org/std/sdmx-expert-group-meeting-2016.htm

1. Abdulla Gozalov, UNSD
Aguascalientes, 20 October 2016

2.  December 2015: Interagency and Expert
Group for SDG Indicators (IAEG-SDGs)
submits its report to the 47th United Nations
Statistical Commission
 March 2016: UNSC agrees the SDGs indicator
framework
 April 2016: UNSD requests SDGs data from
specialized agencies
 May 2016: SDGs indicator data begin to arrive
 18 July 2016: SDGs report and database due
to be released

3. May 2016: SDGs data
begin to arrive
18 July 2016:
Release database
?

4.  April 2016: No certainty over indicators and
disaggregation to be received even in the short
term; relative clarity only reached in June
 To be completed in less than two months:
◦ Receive and catalogue the data
◦ Carry out data entry
◦ Validate and release the data
 To be developed:
◦ SDGs database
◦ Data entry tools
◦ Data validation tools
◦ Data dissemination

5.  Because of time pressure, traditional data
systems development would not work
 SDMX tools had the promise of making it
possible to configure a database system from
building blocks, with little or no software
development involved

6.  MDG DSD was used as a foundation for the
SDGs database
 Concepts and dimensionality were retained
 Code lists were replaced to match the SDGs
dataset
 The DSD had to be updated multiple times
throughout the exercise as data arrived and
codes had to be added for indicators and
their breakdowns, units of measure, etc.

7.  IStat Loader was used to create the SDGs
database
◦ Automatically creates a database from a DSD
◦ Automatically configures mappings in SDMX
Reference Infrastructure for dissemination
 The internal SDGs DSD was fed to Istat Loader
to create and update the database

8.  Eurostat’s SDMX Converter used for data
entry
 Excel data entry spreadsheets were created
and mappings configured between their cells
and the SDGs DSD

9.  The data entry team copied SDGs data from
loosely formatted incoming Excel files to the
data entry spreadsheets and added codes for
series, dimensions, etc
 SDMX Converter was used to retrieve data
from the Excel data entry spreadsheets and
format it as SDMX messages
 Istat Loader was used to upload the SDMX
messages to the database

10.  Eurostat’s SDMX Reference Infrastructure
(SDMX-RI) was used to retrieve data from the
database
 Istat loader configures SDMX-RI mappings
automatically

11.  SDMX-ML Data retrieved from the database was
then converted to JSON, from which presentation
was built for the SDGs Data and Visualization
Platform
 Dissemination at UNSD’s UnData Platform was
configured by adapting procedures developed for
data exchange with UNESCO, which used a DSD
with the same concepts as MDG
 SDMX dissemination through UnData API will be
available shortly
◦ And will then be used to power the Data and
Visualization Platform

12. SDMX
Converter
IStat
Loader
SDMX
RI

13.  Issues with special characters and extended
Latin characters in the tools used
◦ Sometimes took a lot of effort to investigate and
resolve
 Updating the database from an updated DSD
was tedious and error-prone
 These are teething problems, likely to be
overcome soon

14.  Data processing and content validation had to
be done in Excel, outside of the system
◦ Validation and Transformation Language will
address this issue
 Conversion from SDMX-ML to JSON had to be
developed manually using a non-standard
JSON format
◦ Will be resolved when SDMX-JSON support is added
to SDMX-RI

15.  A high-profile database system was created
in a few weeks and populated with over
300,000 observations
 Technology already exists to set up data
systems from building blocks
 Dramatically lower cost of development,
dramatically higher productivity
 Promise of more to come

16. Thank you!
http://unstats.un.org/sdgs
http://data.un.org/SdmxBrowser
http://data.un.org/Explorer.aspx?d=SDGs