1. Evaluation Dr Geoff Curtis Dr Rob Hawtin 29 January 2010 Institutional Innovation Programme

2. Overview Institutional Innovation Programme (IIP) Evaluation questions Headlines from evaluation of Round 1 projects Initial findings Lessons identified Evaluation of Round 2 projects Approach Your contribution Self-evaluation guidance

3. Institutional Innovation Programme Issues with JISC approach to ICT in education and research Often applies to specific problems of department / institution Not the sector Solution often technology- rather than business-led Not necessarily sustainable

4. IIP concept Institutional approach Build on advances from JISC innovation activities Proven practices, technologies, standards and services Often substantial internal business change for institution Supported by significant senior management involvement Fund range of substantial ICT deployments/business change activities Use as exemplars to demonstrate innovation and good practice Extend benefits to sector Benefits realisation activities Problem areas identified professional bodies other initiatives

5. Top-level evaluation questions 1: Is IIP concept useful as general approach for institution-wide problems? 2: Is IIP delivering desired outcomes at institutional level for key areas of institutional ICT concern? Improved community engagement Improved and sustained JISC leadership to the sector Improved capacity, knowledge and skills and diversity of background; Sustainable technological solutions Development of good practice approaches Evidence base for good practice and dissemination Overall sustainability of outcomes 3: Is IIP being managed well?

6. Evaluation Assessment of evaluation questions Lessons identified? Improvements? What else? Evaluating programme not the projects

7. Initial findings –IIP concept Assessment Useful general approach for creating sustainable change More general applicability Lesson identified Need to select/manage projects to meet IIP characteristics Areas for improvement Benefits realisation activities More stringent selection of areas of institutional concern and projects Training on concept and implementation Review approach to planning/funding of benefits realisation activities

8. Initial findings – IIP delivery Assessment Too early for definitive assessment SSBR had only just started Likely to mature post project completion Aspirational nature of outcomes Continue to focus on desired outcomes

9. Initial findings – IIP management The programme is generally well managed Lack of programme management resource and overall scale of Round 2 projects might be a risk to programme Focus role of SSBR

10. Initial findings – summary

11. Evaluation of IIP – phase 2 Approach Phase 1 was by interviews against detailed questions Many more Round 2 projects Evaluation based on Questionnaires Detailed evaluation questions as before Quantitative and qualitative Stakeholders Project teams Follow-up interviews, where necessary Other available information Programme meetings Final and completion reports Self-evaluation

12. Evaluation of Round 2 projects Your contribution Please find time to respond to questionnaires End February Please do your final and completion reports Good things Lessons identified

13. Self-evaluation JISC evaluation handbook Based on your evaluation plans Tips on self-evaluation Outcomes

14. Contact us geoff.curtis@curtiscartwright.co.uk 01483 685023 rob.hawtin@curtiscartwright.co.uk 01483 685024