Presentation by the OECD on "Towards a new generation of indicators measuring digital government" at the Workshop on Digital Government Indicators 6 September 2016. More information can be found at: www.oecd.org/gov/digital-government/

1. TOWARDS A NEW GENERATION OF
INDICATORS: MEASURING DIGITAL
GOVERNMENT
Barbara Ubaldi
Senior Project Manager – Digital Government and Open Data
Guillaume Lafortune
Policy Analyst
OECD GOV
Paris, 6 September 2016

2. • What has the OECD done in the field of e-
government/digital government measurement?
• How was the digital government performance survey
structured and what areas did it cover?
• What does the OECD Recommendation on Digital
Government Strategies imply for the development of
OECD Digital Government Indicators?
• What’s next in terms of goals and objectives for
measuring digital government?
Questions from the agenda

3. • Previous OECD work on measuring e-
government
• Data previously collected
• Approach to be adopted in the new data-
collection
CONTENT

4. Previous OECD work on measuring e-government

5. 2010
• E-GovernmentIndicators:Proposal for a New Frameworkand
Methodology[GOV/PGC/EGOV(2010)2]
• ICT expenditures data collection
2011
• Towardsindicators for benchmarkinggovernment ICT
[GOV/PGC/EGOV(2011)3]
• Decision to refine data collection and focus on performance
• ICT expendituresdata collection
2012
•TowardsE-Government PerformanceIndicatorsGOV/PGC/EGOV(2012)8
•Building the basis for new e-government performanceindicators:ICT
spending by central government [GOV/PGC/EGOV(2011)3/REV1]
The genesis of DigGov Performance Indicators
2014 Digital Government Performance Survey

6. Data previously collected

7. Survey constitutes the first step in
gathering key data to assess governments’
performance in using ICTs among other
measures of productivity in the public
sector
2014 Digital Government Performance

8. March-April 2014
• Background research
• Survey draft and testing
April-June 2014
• Data collection
June-October 2014
• Data validation
• Results analysis
Survey timeline and structure
ICT strategy
Digital rights and obligations
Governance
ICT Project management
ICT Business cases
Financial benefits for the central
government
Financial benefits outside the public sector
Central government ICT expenditures
Sub-national governments ICT
expenditures
ICT Procurement
Online service delivery and transaction
costs
Using national online portals
Survey Sections

9. Central Government ICT expenditure template

10. • 2014 OECD Survey on Digital Government Performance quite technical
and data-intensive: how to ensure that validity, reliability and
international comparability are guaranteed?
• Five commonissues related to methodology and data collection require
additional discussion:
a) Build on existingdefinitionsin the survey to provide precise and easy-to-grasp
definitions of concepts in order to ensure a shared understanding
b) Combine objectiveand subjectivedata in order to ensure high relevance, despite
subjective assessments can be often affected by political concerns and cultural
differences
c) Integratead-hoc estimatesprovidedby respondents with consolidatednational
statistics
d) Agree on the most relevant areas for key Digital Government Performance
Indicatorsin order to ask the questions most aligned to national needs in
performancemeasurement
e) Improve the coordination of data collectionacross differentlevelsof government
in order to overcome issues of international comparability across countries, as
well as those related to data availability
2014 OECD Survey on Digital Government
Performance: Methodologicalchallenges

11. Government at a Glance 2015
0
20
40
60
80
100
120
140
AUS CHL CZE DNK EST FIN FRA HUN ISL ITA JPN LUX MEX NLD NZL POL SVN ESP SWE CHE GBR COL LVA
Central government ICT projects with a total project value greater than USD
10 million, 2014
Source:OECD (2014), Survey on Digital Government Performance.

12. Assessing digital government performance
25-50%:
50-75%:
75-100%:
0-25%:
25-50%:
50-75%:
75-100%:
More
18-24
months
24-30
months
30-36
months
Share of total direct financial benefits from ICT
investments that OECD governments report, 2014
Average length of central government ICT projects with
a total project value greater than USD 10 million, (2014)
Source:OECD (2014), Survey on Digital Government Performance.

13. Approach to be adopted in the new
data-collection

14. Measuring Digital Government

15. OECD Recommendation on
Digital Government Strategies
Openness and
Engagement
Governance and
Coordination
Capacities to Support
Implementation
1) Openness, transparency
and inclusiveness
2) Engagement and
participation in a multi-
actor context in policy
making and service delivery
3) Creation of a data-driven
culture
4) Protecting privacy and
ensuring security
5) Leadership and political
commitment
6) Coherent use of digital
technology across policy
areas
7) Effective organizational
and governance frameworks
to coordinate
8) Strengthen international
cooperation with other
governments
9) Development of clear
business cases
10) Reinforced institutional
capacities
11) Procurement of digital
technologies
12) Legal and regulatory
framework
Creating Value Through the Use of ICT

16. • Built on international instruments
• Framework based on the three pillars of the
Recommendation
• Focus: measuring the principles
• Methodology: surveys of government experts
• Behavioural: De facto not De Jure
• Different but complementary with the UN E-
Government Survey, the EC Digital Economy &
Society Index, EU E-Government Benchmark.
Digital Government Indicators

17. An example: measuring OGD
Governance
De jure
• Existence of laws,
directives,
strategies, policies
to promote OGD at
the central/federal
level, international
commitments
(IODC, G20 AC
principles)?
Implementation
De facto
• How far is data
available (scope and
breadth)?
• How far is data easily
accessible?
• How far does the
government support
the re-use in practice?
Impact
• What are the main
measurable impacts
of OGD? (e.g.
business creation,
new forms of service
delivery leading to
savings…)
Short - medium term
Qualitative table (s) and OUR Government Data Index
Long run
OGD Survey +Case Studies
Challenges?

18. 0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Data availability Data accessibility Government support to re-use
NonationalOGDportal
The 2015 OECD OGD pilot Index
The Open-Useful-Re-UsableGovernmentData Index (OURData Index)

19. Where to start?
Openness and
Engagement
Governance and
Coordination
Capacities to Support
Implementation
1) Openness, transparency
and inclusiveness
2) Engagement and
participation in a multi-
actor context in policy
making and service delivery
3) Creation of a data-driven
culture
4) Protecting privacy and
ensuring security
5) Leadership and political
commitment
6) Coherent use of digital
technology across policy
areas
7) Effective organizational
and governance frameworks
to coordinate
8) Strengthen international
cooperation with other
governments
9) Development of clear
business cases
10) Reinforced institutional
capacities
11) Procurement of digital
technologies
12) Legal and regulatory
framework
Monitoring the Recommendation

20. 1. Questions on the approach?
2. Which principles should we prioritize?
3. How can we reconcile the previous data
collection with the new one accompanying
governments in the transition?
4. Wish list vs feasibility of data collection
5. The indicators as part of the Toolkit to
measure the implementation of the
Recommendation
Questions and points for discussion

21. Thank you!