SlideShare una empresa de Scribd logo
1 de 15
The FRB and SDMX:
National data and International
standards
San Cannon
Federal Reserve Board
SDMXConference
9-11 January 2007
SDMX Conference
9-11 January 2007 2
Background
• The Fed is a statistical agency as well as a
central bank and regulatory agency.
• Lots of data and information are available on the
public website.
• Statistical data are varied: monthly industrial
production indexes, daily interest and exchange
rates, quarterly financial flows for various sectors
of the economy, surveys of small businesses
and consumers, etc.
SDMX Conference
9-11 January 2007 3
Serving our users better
To some, it may appear that the statistical agency
role is secondary.
• Data are not always easy to find.
• Downloads are not customizable.
• Example: Trying to extract one industrial
production series requires retrieving two text
files, cutting and pasting, reformatting….
• Complete – yes. User Friendly – no.
SDMX Conference
9-11 January 2007 4
Data Download Program (DDP)
• XML designated as key format but project team
wrestled with implementation details.
• Staff weighed a homegrown DTD setup against
the new SDMX standard.
• SDMX looked to have greater benefits and was
adopted.
• Good decision: additional internal applications
as well as interagency projects using SDMX are
in the works.
SDMX Conference
9-11 January 2007 5
A lot to learn
• SDMX is based on data structure definitions
(‘key families’) and codelists, with every concept
represented by a code with a corresponding
definition.
• We were unfamiliar with this type of data
modeling, so it proved challenging.
• Two of our pilot datasets translated easily to this
new format; others needed more work.
SDMX Conference
9-11 January 2007 6
Data structures differ
Some data structures are readily adapted to the
concept/codelist representation. Series “keys”
have no real mnemonic value.
HBBA Int. Rate, Official, Discount rate/Base rate
HBCA Int. Rate, Official, Intra-day loans
SCBA Indust. Production, Motorvehicles, NSA
SCBB Indust. Production, Motorvehicles, SA
SDMX Conference
9-11 January 2007 7
Hierarchical relationships
We allow data to be modeled hierarchically, and use
mnemonics that reflect this.
RIFSPFF_N.B
R.*:Rate
R.I.*:Rate of interest in money
and capital markets
R.I.F.*:Federal Reserve System
R.I.F.S.*:Short-term or money
market
R.I.F.S.P.*:Private securities
R.I.F.S.P.FF.:Federal funds
_N.:Not seasonally adjusted
.B:Business (Five days, Monday-
Friday)
JQI_I02Y3361T3_N.M:
J.*:Indices except of prices
J.Q.*:Production
J.Q.I.:Industrial
_I.*:NAICS-based industry
classification
02Y:codes from year 2002
3361.:Motor Vehicle Manufacturing
T:thru
3363:Motor Vehicle Parts
Manufacturing
_N.:Not seasonally adjusted
.M:Monthly
SDMX Conference
9-11 January 2007 8
Applying the SDMXmodel
• Data represented by a concrete number of concepts are
much easier to represent with key family dimensions and
attributes:
JQI_I02YMF_N.M → Topic_Industry_SA.Freq
FA156900005.Q Prefix (2 digits), Sector(2),→
instrument type (5), series type (1), frequency
• Hierarchical relationships and varying number of concepts
makes life more difficult:
RIFSPPNA2P2D30_N.B → Topic?_SA.Freq
RIFLGFCY20_XII_N.B T→ opic?_Inflate_SA.Freq
SDMX Conference
9-11 January 2007 9
Decisions we made
• Allow a variable number of data structure
definitions per dataset.
• Use the compact format for internal
exchange and external downloads.
• Stick with SDMX 1.0, for now.
• Use a relational database to store data
and XML information for retrieval.
SDMX Conference
9-11 January 2007 10
Final product: DDP!
• We have a flexible application whose interface
is entirely driven by the data structure
definitions.
• “We store the XML as carefully sliced text in a
relational database and we can build an index
structure that allows us to respond to ad-hoc
queries very efficiently, even for large volumes
of data.”
Data Download Program
SDMX Conference
9-11 January 2007 11
Strengths and weaknesses
• Because interface is entirely data driven, it is
easy to add new data.
• Internal architecture is complex, due to security
and the data workflow:
– SDMX files are generated by data staff and
transmitted to public website staff for processing.
– These files are made available on the website and
“shredded” for database entry and lookup.
• Current structure is not set up for codelist
sharing.
SDMX Conference
9-11 January 2007 12
What do the users say?
• “Really excellent. One of the
best I’ve used on the web.”
• “This Data Download thing is
better than sliced bread.”
• “I downloaded the XML files, but I
cannot run them. All I see is the
xml code.”
SDMX Conference
9-11 January 2007 13
What do the numbers say?
• More than 250,000 unique visits since April –
about 50,000 per month.
• Data Download Program is the 6th most visited
area on the Federal Reserve website.
SDMX Conference
9-11 January 2007 14
Next steps
• Add more data: reserves, exchange rates,
consumer credit.
• Continue working with other central banks and
statistical agencies on common framework.
• Prepare to move to SDMX 2.0 to take advantage
of additional features.
SDMX Conference
9-11 January 2007 15
The last slide…
Questions? Comments?
Thank you for your attention!
San Cannon
scannon@frb.gov
(202) 452-3710

Más contenido relacionado

Similar a FRB Adopts SDMX Standard for Data Download Program

resume_spedding_joe
resume_spedding_joeresume_spedding_joe
resume_spedding_joeJoe Spedding
 
Kim Ross-Smith Resume_February2016
Kim Ross-Smith Resume_February2016Kim Ross-Smith Resume_February2016
Kim Ross-Smith Resume_February2016Kimberly Ross-Smith
 
Introduction to Modern Data Virtualization 2021 (APAC)
Introduction to Modern Data Virtualization 2021 (APAC)Introduction to Modern Data Virtualization 2021 (APAC)
Introduction to Modern Data Virtualization 2021 (APAC)Denodo
 
How to Place Data at the Center of Digital Transformation in BFSI
How to Place Data at the Center of Digital Transformation in BFSIHow to Place Data at the Center of Digital Transformation in BFSI
How to Place Data at the Center of Digital Transformation in BFSIDenodo
 
Analysis and evaluation of riak kv cluster environment using basho bench
Analysis and evaluation of riak kv cluster environment using basho benchAnalysis and evaluation of riak kv cluster environment using basho bench
Analysis and evaluation of riak kv cluster environment using basho benchStevenChike
 
Big Data World Forum
Big Data World ForumBig Data World Forum
Big Data World Forumbigdatawf
 
Fbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesFbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesCindy Irby
 
Nationwide_Ensures_Basel_II_Compliance
Nationwide_Ensures_Basel_II_ComplianceNationwide_Ensures_Basel_II_Compliance
Nationwide_Ensures_Basel_II_ComplianceAndrew Painter
 
Bing Zheng SQL Portfolio
Bing Zheng SQL PortfolioBing Zheng SQL Portfolio
Bing Zheng SQL PortfolioBingTZheng
 
Modern Data Management for Federal Modernization
Modern Data Management for Federal ModernizationModern Data Management for Federal Modernization
Modern Data Management for Federal ModernizationDenodo
 
Managing the financial services data explosion
Managing the financial services data explosionManaging the financial services data explosion
Managing the financial services data explosionLaura Hood
 
Machine Learning for z/OS
Machine Learning for z/OSMachine Learning for z/OS
Machine Learning for z/OSCuneyt Goksu
 
CHAPTER-4_RELATIONAL-DATABASE.pptx
CHAPTER-4_RELATIONAL-DATABASE.pptxCHAPTER-4_RELATIONAL-DATABASE.pptx
CHAPTER-4_RELATIONAL-DATABASE.pptxRiaBago
 
iphix-demo-day.pdf
iphix-demo-day.pdfiphix-demo-day.pdf
iphix-demo-day.pdfEd Dodds
 
DBMS - Relational Model
DBMS - Relational ModelDBMS - Relational Model
DBMS - Relational ModelOvais Imtiaz
 
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...Ashnikbiz
 

Similar a FRB Adopts SDMX Standard for Data Download Program (20)

resume_spedding_joe
resume_spedding_joeresume_spedding_joe
resume_spedding_joe
 
Kim Ross-Smith Resume_February2016
Kim Ross-Smith Resume_February2016Kim Ross-Smith Resume_February2016
Kim Ross-Smith Resume_February2016
 
CV D.Vybornov
CV D.VybornovCV D.Vybornov
CV D.Vybornov
 
Introduction to Modern Data Virtualization 2021 (APAC)
Introduction to Modern Data Virtualization 2021 (APAC)Introduction to Modern Data Virtualization 2021 (APAC)
Introduction to Modern Data Virtualization 2021 (APAC)
 
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data EngineNZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
NZS-4532 - Bringing Historical Data to Life with IBMs SMF Data Engine
 
How to Place Data at the Center of Digital Transformation in BFSI
How to Place Data at the Center of Digital Transformation in BFSIHow to Place Data at the Center of Digital Transformation in BFSI
How to Place Data at the Center of Digital Transformation in BFSI
 
Analysis and evaluation of riak kv cluster environment using basho bench
Analysis and evaluation of riak kv cluster environment using basho benchAnalysis and evaluation of riak kv cluster environment using basho bench
Analysis and evaluation of riak kv cluster environment using basho bench
 
Big Data World Forum
Big Data World ForumBig Data World Forum
Big Data World Forum
 
Fbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_servicesFbdl enabling comprehensive_data_services
Fbdl enabling comprehensive_data_services
 
Resume - Santosh B
Resume - Santosh BResume - Santosh B
Resume - Santosh B
 
Nationwide_Ensures_Basel_II_Compliance
Nationwide_Ensures_Basel_II_ComplianceNationwide_Ensures_Basel_II_Compliance
Nationwide_Ensures_Basel_II_Compliance
 
Bing Zheng SQL Portfolio
Bing Zheng SQL PortfolioBing Zheng SQL Portfolio
Bing Zheng SQL Portfolio
 
Modern Data Management for Federal Modernization
Modern Data Management for Federal ModernizationModern Data Management for Federal Modernization
Modern Data Management for Federal Modernization
 
Managing the financial services data explosion
Managing the financial services data explosionManaging the financial services data explosion
Managing the financial services data explosion
 
Machine Learning for z/OS
Machine Learning for z/OSMachine Learning for z/OS
Machine Learning for z/OS
 
resume_abdul_up
resume_abdul_upresume_abdul_up
resume_abdul_up
 
CHAPTER-4_RELATIONAL-DATABASE.pptx
CHAPTER-4_RELATIONAL-DATABASE.pptxCHAPTER-4_RELATIONAL-DATABASE.pptx
CHAPTER-4_RELATIONAL-DATABASE.pptx
 
iphix-demo-day.pdf
iphix-demo-day.pdfiphix-demo-day.pdf
iphix-demo-day.pdf
 
DBMS - Relational Model
DBMS - Relational ModelDBMS - Relational Model
DBMS - Relational Model
 
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
Polyglot Persistence and Database Deployment by Sandeep Khuperkar CTO and Dir...
 

Último

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 

Último (20)

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 

FRB Adopts SDMX Standard for Data Download Program

  • 1. The FRB and SDMX: National data and International standards San Cannon Federal Reserve Board SDMXConference 9-11 January 2007
  • 2. SDMX Conference 9-11 January 2007 2 Background • The Fed is a statistical agency as well as a central bank and regulatory agency. • Lots of data and information are available on the public website. • Statistical data are varied: monthly industrial production indexes, daily interest and exchange rates, quarterly financial flows for various sectors of the economy, surveys of small businesses and consumers, etc.
  • 3. SDMX Conference 9-11 January 2007 3 Serving our users better To some, it may appear that the statistical agency role is secondary. • Data are not always easy to find. • Downloads are not customizable. • Example: Trying to extract one industrial production series requires retrieving two text files, cutting and pasting, reformatting…. • Complete – yes. User Friendly – no.
  • 4. SDMX Conference 9-11 January 2007 4 Data Download Program (DDP) • XML designated as key format but project team wrestled with implementation details. • Staff weighed a homegrown DTD setup against the new SDMX standard. • SDMX looked to have greater benefits and was adopted. • Good decision: additional internal applications as well as interagency projects using SDMX are in the works.
  • 5. SDMX Conference 9-11 January 2007 5 A lot to learn • SDMX is based on data structure definitions (‘key families’) and codelists, with every concept represented by a code with a corresponding definition. • We were unfamiliar with this type of data modeling, so it proved challenging. • Two of our pilot datasets translated easily to this new format; others needed more work.
  • 6. SDMX Conference 9-11 January 2007 6 Data structures differ Some data structures are readily adapted to the concept/codelist representation. Series “keys” have no real mnemonic value. HBBA Int. Rate, Official, Discount rate/Base rate HBCA Int. Rate, Official, Intra-day loans SCBA Indust. Production, Motorvehicles, NSA SCBB Indust. Production, Motorvehicles, SA
  • 7. SDMX Conference 9-11 January 2007 7 Hierarchical relationships We allow data to be modeled hierarchically, and use mnemonics that reflect this. RIFSPFF_N.B R.*:Rate R.I.*:Rate of interest in money and capital markets R.I.F.*:Federal Reserve System R.I.F.S.*:Short-term or money market R.I.F.S.P.*:Private securities R.I.F.S.P.FF.:Federal funds _N.:Not seasonally adjusted .B:Business (Five days, Monday- Friday) JQI_I02Y3361T3_N.M: J.*:Indices except of prices J.Q.*:Production J.Q.I.:Industrial _I.*:NAICS-based industry classification 02Y:codes from year 2002 3361.:Motor Vehicle Manufacturing T:thru 3363:Motor Vehicle Parts Manufacturing _N.:Not seasonally adjusted .M:Monthly
  • 8. SDMX Conference 9-11 January 2007 8 Applying the SDMXmodel • Data represented by a concrete number of concepts are much easier to represent with key family dimensions and attributes: JQI_I02YMF_N.M → Topic_Industry_SA.Freq FA156900005.Q Prefix (2 digits), Sector(2),→ instrument type (5), series type (1), frequency • Hierarchical relationships and varying number of concepts makes life more difficult: RIFSPPNA2P2D30_N.B → Topic?_SA.Freq RIFLGFCY20_XII_N.B T→ opic?_Inflate_SA.Freq
  • 9. SDMX Conference 9-11 January 2007 9 Decisions we made • Allow a variable number of data structure definitions per dataset. • Use the compact format for internal exchange and external downloads. • Stick with SDMX 1.0, for now. • Use a relational database to store data and XML information for retrieval.
  • 10. SDMX Conference 9-11 January 2007 10 Final product: DDP! • We have a flexible application whose interface is entirely driven by the data structure definitions. • “We store the XML as carefully sliced text in a relational database and we can build an index structure that allows us to respond to ad-hoc queries very efficiently, even for large volumes of data.” Data Download Program
  • 11. SDMX Conference 9-11 January 2007 11 Strengths and weaknesses • Because interface is entirely data driven, it is easy to add new data. • Internal architecture is complex, due to security and the data workflow: – SDMX files are generated by data staff and transmitted to public website staff for processing. – These files are made available on the website and “shredded” for database entry and lookup. • Current structure is not set up for codelist sharing.
  • 12. SDMX Conference 9-11 January 2007 12 What do the users say? • “Really excellent. One of the best I’ve used on the web.” • “This Data Download thing is better than sliced bread.” • “I downloaded the XML files, but I cannot run them. All I see is the xml code.”
  • 13. SDMX Conference 9-11 January 2007 13 What do the numbers say? • More than 250,000 unique visits since April – about 50,000 per month. • Data Download Program is the 6th most visited area on the Federal Reserve website.
  • 14. SDMX Conference 9-11 January 2007 14 Next steps • Add more data: reserves, exchange rates, consumer credit. • Continue working with other central banks and statistical agencies on common framework. • Prepare to move to SDMX 2.0 to take advantage of additional features.
  • 15. SDMX Conference 9-11 January 2007 15 The last slide… Questions? Comments? Thank you for your attention! San Cannon scannon@frb.gov (202) 452-3710

Notas del editor

  1. Nice title.