This document is intended for use by data owners within government to learn how Demand-Driven Open Data (DDOD) could benefit their organizations. DDOD is an initiative by the U.S. Department of Health and Human Services (HHS) started in November 2014 as part of its IDEA Lab program. The goal is to leverage the vast data assets throughout HHS’s agencies (CMS, NIH, CDC, FDA, AHRQ and others) to create additional economic and public health value. DDOD provides a systematic, ongoing and transparent mechanism for anybody to tell HHS and its agencies what data would be valuable to them. With this initiative HHS can move from measuring Open Data in terms of number of datasets released to value in terms of use cases enabled. DDOD website: http://ddod.us

1. Demand-Driven Open Data
More info:
Contact:
http://ddod.us
David.Portnoy@HHS.gov, @DPortnoy
Introduction to DDOD for
Data Owners

2. “Demand-Driven Open Data” (DDOD) is
a framework of tools and methods that…
Provide external data users[✽] with a
systematic, ongoing and transparent
way to tell HHS what data they need
...To be managed, measured and executed
in terms of use cases, enabling allocation of
limited resources based on value
What is it?
✽ Such as industry, researchers, nonprofits, media and local governments

3. Why DDOD?
What happens when there is no
feedback loop on the value delivered?

4. The Problem
Without measuring value,
rational individuals deliver datasets that are
1. Easiest to generate
2. Least risky to release
And the growing volume of datasets
makes locating useful ones hard

5. The Opportunity
HHS can create additional economic, health and social value by
changing the way it measures progress on Open Data efforts...
from number
of datasets
released
to value in terms
of use cases
enabled

6. Prior to DDOD, if you wanted to influence the data HHS provides there were primarily
two extremes: participate in one-off events or attempt the regulatory path
...But each had significant limitations
• No systematic feedback to
influence data available
• Limited by short durations and
often unproven business models
Decision process isn’t transparent
No access to restricted use data
• Costly and requires access
• Long lead times
• Uncertain outcomes
• Battle parties with competing
interests
Influence process isn’t fully
transparent
Gap in feedback options
One-off Methods
(Bottom-up attempts)
• Challenges,
• Hackathons,
• Meetups, conferences,
• Crowdsourcing
Regulatory
(Top-down approach)
• Lobbying, public comment,
• FOIA,
• Leverage associations and
consortiums
ExamplesLimitations
Missing potential for creation of
economic & public health value

7. So we need a mechanism that…
Enables systematic, ongoing and transparent signaling of relative
value of data in a way that’s inclusive of all types of participants
That’s “Demand Driven Open Data” (DDOD)
1. Systematic
2. Ongoing
3. Transparent
4. Inclusive

8. • No systematic feedback to
influence data available
• Limited by short durations and
often unproven business models
Decision process isn’t transparent
No access to restricted use data
• Costly and requires access
• Long lead times
• Uncertain outcomes
• Battle parties with competing
interests
Influence process isn’t fully
transparent
Gap in feedback options
Demand-Driven Open Data
Need a mechanism that’s
systematic, ongoing, and transparent
Not limited to arbitrary time frames and
short durations of one-off methods
Mitigate the long lead times, expense
and uncertainty of influencing
legislation
Gain transparency on how your needs
are weighed against competing
interests and costs
Use an approach more compatible with
gaining access to restricted use data
One-off Methods
(Bottom-up attempts)
• Challenges,
• Hackathons,
• Meetups, conferences,
• Crowdsourcing
Regulatory
(Top-down approach)
• Lobbying, public comment,
• FOIA,
• Leverage associations and
consortiums
ExamplesLimitationsDDOD fills the gap and addresses many of the limitations

9. DDOD is positioned to
1. Maximize value, innovation and discovery using existing
data assets
2. Achieve an engaged, active user community
3. Help guide prioritization of open data efforts

10. Moving from...
Build first and then
see if anyone will use it
Directive driven
By understanding the “market” for its data, HHS can better allocate
resources by migrating to a “Lean Startup” methodology
To...
Make sure there are customers
before building
Demand driven

11. DDOD complements and reinforces the existing efforts for
HealthData.gov and Health Data Leads
Health Data Leads DDOD
“Push” by mandate “Pull” by need
Hosting, Indexing, Discovery
Expert Driven Demand Driven

12. Processes for administration of use cases, such as
• Encouraging responsiveness, transparency and documentation
• Ensuring use cases and resulting datasets are indexed in HealthData.gov
Specialized tools for administering use cases
• Workflow engine, communications method, knowledge base
• Data processing, storage, hosting, versioning
Proactive outreach to industry and academia for a thriving
community
DDOD provides 3 core services to Data Owners

13. Each of the participants (Data User, DDOD Admin, and Data Owner) is responsible for
enabling a specific set of milestones
...But all implementation decisions ultimately are made by the Data Owner
The DDOD Admin only facilitates the process when needed

14. DDOD relies on signing up Data Users to advocate for their use cases, participate using
DDOD tools and provide effectiveness feedback. For Users, the process looks like this:
Get started by simply adding
your use case [✽]
We’ll get you going, starting with a
discussion that covers:
● Requirements for your use cases
● Criteria you use for prioritization
As we go about working on your use cases,
you’ll leverage the DDOD tools and
processes for requirements management,
voting and community engagement
You submit verbal and written
evaluations of the DDOD tools and
processes
✽ First search HealthData.gov to see if
the dataset or use case already exists
EvaluateParticipateOnboardAdd

15. Qualitative Quantitative
Prioritization based on self-reported descriptions
from questions provided on a form with each new
use case or feature
While absolute quantitative valuations might be
difficult, it’s possible to use known objective factors
to assess relative value
1. What’s the value to your organization?
2. What’s the value to industry?
3. What’s the value to public health?
4. How consistent is it to the mission of the
agency?
5. How time sensitive is the request? What are
the impacting factors?
1. Cost already spent on a procured study or
survey requested
2. Revenue from cost recovery programs
3. Avoided costs from FOIA requests, manual
periodic releases, etc.
4. Crowdfunding-style cumulative pledges from
multiple parties as a proxy for value
There are both qualitative and quantitative methods for prioritizing use cases

16. Evaluation & feedback
Completed
use cases
ImplementPrioritized
use cases
PrioritizeIncoming
use cases
Prioritization is at the level of program owner
Consider implementation cost, savings from avoided
future requests (such as FOIA), revenue opportunity for
future cost recovery, risk of PII/PHI, risk of
misinterpretation
Including strategic relevance, agency
mission, org priorities, recognition
The decision to implement is not binary. It
involves requirements management for
potentially multiple interested parties
① ② ③
All prioritization and implementation decisions are made by Data Owners.
We found there are typically 3 drivers.

17. Implementation of a use case could fall into one of 3 categories
Time to execute
Cost/Effort
Improve
Promote
Add
Facilitate deployment of
● New datasets
● New APIs
For existing datasets
● Add needed fields
● Improve data quality
● Add / improve metadata
● Add / improve API
If datasets already exist in legacy systems, make
them more available and discoverable
● publicize availability
● index to HealthData.gov and Data.gov
Current
State

18. Decentralized implementation, with
allocated team in each organization
Centralized implementation
team with department-wide buy-in
Centralized DDOD manager
Centralized dev team
Health Data Leads,
Program owners
Dev team at level of program or
data owner
Who executes the Use Case? The circumstances around the execution of each use case
is different...
Typically, implementation by Data Owner’s organization is most efficient, due to the depth of
domain knowledge required.
But depending on the organization’s resources, capabilities and priorities it may be better
executed with the help of a centralized development team.

19. Communications
method
Knowledge
Base
Data Processing
& Storage
Workflow
engine
There are 4 core components to the DDOD tool set..
Their use is guided by DDOD process, policy, and best practices
Transparency
Discussions & decisions
around use cases must
be visible to the public.
Solutions
Use case requirements
and their solutions must
be editable by all parties.
Data
Provide data processing
and storage capabilities
where needed.
Tracking
Track and manage the
status and assignment
of use cases
These tools continue to evolve with an eye towards lowering the learning curve and
improving ease of use for both Data Users and Data Owners. Minimizing the need for
assistance from DDOD Administrators makes it possible for DDOD to scale.

20. Outreach is a core component of DDOD
Reasons that high level of participation from Data Users is needed:
● To help Data Owners with prioritization it by providing sufficient information about
relative value of datasets
● For both Data Users and Data Owners to gain confidence that DDOD is a
sustainable marketplace that enables productive interaction
Methods of proactive and sustained outreach to the user community:
● Targeted online publications
● Participation in conferences and user group meetups
● Collaboration with healthcare-related industry groups, accelerators and incubators
● Collaboration with universities
Scalability for DDOD depends on building:
● Brand awareness through outreach
● Brand equity through the completion of use cases

21. DDOD also serves to enhance content and discoverability for HealthData.gov, as well
as ensuring the relevant system of record is entered in the EDI
Data User
runs search on
HealthData.gov
Data User
creates / updates
use case
DDOD Admin
engages Data Owner
on use case
• keywords, subject
• data dictionary
HealthData.gov
Data
Dictionary
Dataset
Inventory
EDI*
Use
Case
DDOD Admin ensures changes to
EDI get propagated to HD.gov
* Enterprise Data Inventory (EDI), which is a catalog of
HHS “Strategically Relevant Data Assets”
Data
Dictionary
Dataset
Inventory
DDOD Admin enters use
case on HD.gov with links
to specifications
Data Owner adds entry to
EDI, including metadata
DDOD Admin
curates entry &
ensures SLAs
DDOD Admin
creates repository
for use case
Process of adding a new DDOD use case

22. DDOD initiative can be categorized by phases along the Development and
Engagement dimensions
Development
Pilot
1
Engagement
Idea Prototype Market
Small
scale trial
Vision
Pilot
2
AwarenessActionTransformation
Scope of 1-year
DDOD launch
Source: Deloitte analysis of Challenge.gov

23. There are related initiatives that would make DDOD more effective...
One measures the usefulness of existing data,
while the other enables users to discover new data
DDOD ✓ Signaling of
demand
Enables ① systematic and ② ongoing and ③
transparent signaling of relative value of data
for the ④ full range of market participants
Data maturity scorecard,
Data activity scorecard
Usefulness of
supply
Has a feedback loop on the usefulness of
existing and future data
Full metadata inventory Discovery of
supply
Enables users to discover the possible
applications for data, regardless of its privacy
classification or availability