Project Reviews - Why, what and how

1. Post Project Reviews Gavin Berry April 2010 Gavin@zaragy.com

3. Large Programmes : Lead various programmes across multiple disciplines of up to 250 people.

4. Best practices & Metrics : Developed and introduced an extensive metrics basis to development programmes to evidence the trends and results.

5. Relationship Management :Rescued crashing projects – largely through success in relating to diverse and demanding stakeholders, both internal projects and external vendor.

6. International : Worked across four continents and the various established cultures to deliver both business and technology change successfully.

7. Conversions / Migrations : Extensive experience in migrating tranches of business and converting systems - largely in the financial sector. Both as an internal initiative and as an external vendor.

8. Contracts : Successfully lead and represented major financial organisations in establishing and amending contractual terms.

9. Business Process Re-engineering: Lead the successful establishing of business structures and processes to match strategic goals.

10. Technical skill : Background of development from single address assembler through OOD.

12. Systems Development best practices (SDLC)

13. Business Transformation

14. Client facing

16. What is the case for Post Project Reviews: Research Both Gartner and Forrester report that benefits realisation is not occurring - (75% of programmes deliver no benefit; less than 30% measure benefits – Reiss 2004) 40% of projects fail to achieve their business case within 1 year - The Conference Board survey (2001) 51% [ERP] implementations unsuccessful - Robbins-Gioia survey (2001) average project overrun of 40% pa - McManus, Wood-Harper (2004) 61% of projects fail - The KPMG Canada Survey (1997) 7 out of 10 IT projects fail - The OASIG study (1995) ONLY 16.2% of projects complete on schedule and budget - The CHAOS report (1995) ONLY 29% of projects complete on schedule and budget - The CHAOS report (2004)

17. What is the case for Project Reviews: Research Findings Successful projects require: User / Business involvement Support from the top Executives Clear objectives and requirements that tie into the business case Effective project management Good people Good disciplines and practices

18. What have we done Improved Governance emphasising business & executive involvement – PMI & Prince II More professional project management through improved PM processes such as: Gated process as outlined by Cooper, Edgett and Kleinschmidt

19. Report card – it has improved B u t N O T e n o u g h

20. A key missing piece CHAOS (2004) highlights that measuring & achieving business benefits have lagged other improvements Dilemma on how to Achieve Business Benefits

21. A key Missing piece – Project Reviews Only mitigation is project reviews looking at: Why we are doing the project Review project Benefits against the Business Plan How does the project deliver the benefits Review project scope against the business case rationale Where do the benefits come from Review the business case model

22. Paradigm Shift Traditional View: Project review to check the way the project was run More Useful View: Project review to check we are [still] getting what we want

23. Input to Project Reviews - Strategic If the purpose for the project is business benefit – we MUST have clearly articulated Benefits Case as a key reference A model of costs and revenue, clearly showing each component (Capex, amortisation etc; project resource costs; Sources of Revenue, assumptions and margins etc) Business commitment to adjust business plan Traceability between project scope and business case This is a grid against which the project will be reviewed

24. Input to Project Reviews - Tactical Assumption & Risk Register s Not only must we assess the risks of delivery of the project, but also any risk to the current operation Delivery plan & progress reports Can we be sure that an accurate position is being reflected in the formal reporting. PMO/GPO best practices / methodology Are we sure that the various project artefacts have been produced and stored? This includes various hygiene factors. Can we be sure that the organisations operating capability is protected (capital and people)?

25. When to complete Reviews Every time we change the target: Change in Budget Change in Timeline Change in scope At regular pre-determined milestones Reviews have an inherent cost– like all other costs – reviews must be deployed cost-effectively On completion Measure consistently against your agreed definition [of success] - Dr Walter Fernandez and Mr. Graeme Thomas

26. How to complete reviews Preparation: Agree with major stakeholders what aspects are to be emphasised in the review – Project Board / Steering Committee Develop target groups on whom to focus the review Develop a set of questions (matrix) that will address agreed review topics Structure a collection approach for each group

27. Example Question Matrix

28. How to complete reviews Collection: Collect and produce the analysis of the review over a short period of time (target 2 weeks) Be open with c0ntributors on purpose and any actions Provide for a mechanism to resolve uncertain responses Output: Present to the Governance board first Once key actions have been agreed – present to the contributors

29. Some hints & ideas on the How to Agree policy on Benefit realisation: Full value in following financial year, or Immediate pro-rated benefit in first year Business and their governance process then follow through on the realisation commitment. Deferral and workaround log for any scope aspects that cannot be built and the business workaround to overcome the shortcoming. Doubles as benefit shortfall log. Implementation Questionnaire for completion immediately after system implementation by line users. Measures any implementation disruption

30. Some hints & ideas on the How to Cycle through the following for collection of feedback on similar questions from IT teams: PM Conducts a “Lessons Learnt” session Anonymous questionnaires completed Independent agent conducts “Lessons Learnt” sessions Questionnaire for heads of business areas post implementation Report Back sessions to both team members and governance groups on outcome of “Lessons Learnt”

31. Review Output Confirmation of the extent to which the project deliveries will meet/have met expectations – by Review section e.g.: Benefits Budget / Cost Communication Deliverables Implementation Issues / Other Overall Success Planning & Timelines Quality Resources Risks Scope & Objectives Support Training Documentation of lessons learned in the process – covering: Organisational ability to initiate & govern - ORGANISATION Methodology – how appropriate - PROCESS Human capital – skills, capabilities and morale - PEOPLE

32. Reviews by Case – Organisation view Apple organisation Nature of business: Hugely successful third party administration outfit , predominantly in utilities and financial sector, FTSE100 Business Culture: Directed by the financials, highly regulated environment Project Outline: Contracted with world’s largest Life Assurer (WLLA) to support a financial product that was unknown and unsupported with a view to future business; WLLA pushed Apples to implement capability for distribution window Systems settled down after 6 months – widespread data integrity issues Any remedial activity would be unfunded Spend projected to exceed total revenue from WLLA Governance through main board

33. Reviews by Case – Organisation view Banana organisation Nature of business: Large Life assurer – no longer trading Business Culture: Politically driven, listed – but poor performer, major shareholding by very successful organisation in different industry Project Outline: Replacement Life Administration system for >1000 life products Lack of confidence in own IT favoured off-the–shelf solution Proposal pitched to get support for desired option Approved joint venture with successful US company Governance through committee

34. Reviews by Case – Organisation view Cherry organisation Nature of business: Large Life assurer Business Culture: Results focussed, Listed in financial services group, executive required to have personal holdings Project Outline: Absorption of business from acquisition of larger Life company Governance through accountable individuals

35. Reviews by Case – Organisation view Date organisation Nature of business: Large Life assurer Business Culture: Traditional, Listed – mediocre performer, culturally aligned Project Outline: Continued development of strategic administration platform through regular releases (circa 4 per year) Governance by business executive

36. How Reviews were done

37. Questions

39. Cooper et al 2000 – Making Programme management more effective (Cooper, Edgett & Kleinschmidt, 2000).pdf

40. The Conference Board Survey (2001)

41. Robbins-Gioia survey (2001)

42. The OASIG study (1995)

43. McManus, Wood-Harper (2004)

44. The CHAOS report (1995)

45. The CHAOS report (2004)

46. The KPMG Canada Survey (1997)

47. Dr Walter Fernandez and Mr Graeme Thomas

48. Dorsey 2009 – International journal on Governmental Financial Management - using periodic audits to prevent catastrophic project failure.pdf

49. Chris Sauer, Andrew Gemino, and Blaize Horner Reich (2007) - The impact of size and volatility on IT project performance