Presentation by Michael Bird, CorporateGovernance and Information, Ministry of Business, Innovation andEmployment, New Zealand, at the 8th Conference on Measuring Regulatory Performance: Realising Impact: The Role of Institutional Frameworks in Regulatory Policy, Breakout Session 3, which took place in Sydney 15-16 June 2016. Further information is available at www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm.

1. Institutional frameworks for ex ante and ex post evaluation:
Insights from the New Zealand Public Sector
Michael Bird
General Manager: Evidence, Monitoring and Governance
Corporate, Governance and Information

2. Overview of the use of evaluation in regulation
• Regulatory Impact Analysis (ex-ante evaluation)
• On-going monitoring, stakeholder engagement and review
• Crown entity (regulator) performance monitoring
• Ex-post evaluation

3. Regulatory Impact Analysis (RIA)
• NZ follows OECD best practice across the board, but challenges remain
• NZ’s RIA requirements try to help by:
• Providing a systematic, but flexible, framework
• Encouraging an early start on analysis, and encouraging consultation
• Acknowledging the importance of identifying key assumptions
• Encouraging good practice around quality assurance
• To what extent is the RIA process leading to meaningful ex post evaluation?

4. On-going monitoring, engagement and review
• Important for understanding how well a regulatory system is performing and for
spotting emerging risks/issues
• Outcomes (ie is the system achieving the expected outcomes?)
• Leading indicators (ie early warning signals)
• Performance measures (ie are the different actors in the system performing well?)
• Difficulties of getting the right information at the right time..…and knowing in
advance which bits of the system we should be looking at!
• MBIE conducts ‘regulatory system assessments’ of how well a system is
performing within a given policy and institutional setting

5. Crown entity monitoring
• Many regulatory functions are carried out by Crown entities or statutory boards,
at arms-length from government
• The Boards of these agencies must advise Ministers about:
• what they aim to achieve
• how their decisions and operating choices have an impact on the objectives
• Ministers also want to know about alignment with legislation and the
Government’s aims, and efficiency of operations
• Assessing effectiveness and efficiency is challenging
• There is increasing use of a ‘Performance Improvement Framework’ to review
Crown entities

6. Ex-post evaluation
• Often seen as something to consider after-the-fact
• Data often not aligned with evaluation objectives
• Need for inefficient, one-off, retrospective data collection
• Deliberate shift practice to consider monitoring and evaluation from the
beginning
• Surfacing evaluation objectives
• Clarifying data needs at the outset
• Recent developments in linked databases offer opportunities for
improving efficiency
• Integrated Data Infrastructure
• Better use of existing administrative data across regulatory functions