Construction Management in Developing Countries, Lecture 10, Project Evaluation and Auditing, types and differences between evaluation and auditing, challenges faced by professional auditors in project auditing in developing countries

1. Project Evaluation and Auditing
(Chapter 10: 6 hours)
Hari K. Shrestha
Nepal Engineering College

2. Project Evaluation
• Project Evaluation is a step-by-step process of collecting,
recording and organizing information about project results,
including short-term outputs (immediate results of activities,
or project deliverables), and immediate and longer-term
project outcomes (changes in behavior, practice or policy
resulting from the project).
• Definition depends on type/objective of organization/project.
• Evaluation involves the systematic collection of information
about the activities, characteristic and outcomes of an activity
or action, in order to determine its worth or merit (Dart et al,
1998). It is a major part of learning, and can provide a wealth
of useful information on the outcomes of a project or action,
and the dynamics of those who undertook the work. Through
the identification of the highlights and lowlights of the project,
evaluation draws conclusions which can inform future decision
making, and assist to define future projects and policies
(Patton, 1997).

3. Project Evaluation Purposes
Project evaluation aims at analyzing research and
development projects, or activities or ideas, for any or
all of the following purposes:
• Getting an overall understanding of the project.
• Making priorities among a set of projects.
• Taking a decision about whether or not to proceed
with a project (at the beginning or after the evaluation
period)
• Monitoring projects, e.g. by following up the
parameters analyzed when the project was selected.
• Terminating projects and evaluating the results
obtained.

4. Needs and Importance of Project Evaluation
Evaluating project results are helpful in providing answers
to key questions like:
• Should the project proceed?
• What progress has been made?
• Were the desired outcomes achieved? Why?
• Are there ways that project activities can be refined to
achieve better outcomes?
• Does project structure needs modification to achieve
project objectives?
• Do the project results justify the project inputs?

5. Why Project Evaluation?
Evaluation of projects are undertaken for a number of reasons:
• to contribute information, for example for the review of a management strategy
• to be accountable to those who are funding the project
• as a debrief to the group on the completion of a project
• to reflect on how you’re developing as a group or organisation, and whether you
need to improve the way things are being done
• to review and adjust the management methods and techniques used in an area
• to develop the skills and understanding of people involved in a project by
enabling them to reflect on, and learn from, their experiences
• to provide information for planning future projects
• to determine the worth of the project
• to justify and promote a management action to the wider community
• to create a historical record of management success over time
• to review management methods and techniques, and how effective they were
• to see if the work is contributing to the management objectives for an area
• to determine whether you have met project objectives.
• It is important to be aware of the reasons for undertaking an evaluation before
commencing, as this will define how the evaluation process should be designed,
and what outcomes are expected as a result.
http://www.planning.wa.gov.au/dop_pub_pdf/5_project_evaluation.pdf

6. Project Types
Successful :
• The project is completed on time and on budget, with all
features and functions originally specified
Challenged :
• The project is completed and operational, but over-budget,
over the time estimate, and with fewer features and
functions than initially specified.
Failed :
• The project is cancelled before completion or never
implemented."

7. Types of Evaluation
Based on Timing
• Ex-ante evaluation
• Mid-term evaluation
• Ex-Post evaluation
Based on position of evaluator
• External evaluation
• Internal evaluation or self-assessment
Based on Purpose
• Formative
• Summative

8. Based on Timing
Evaluation can take place at any time in a project. However, the most appropriate timing
will be governed by the nature of the project, and the reason for carrying out the
evaluation. It will be most effective when it is built in as a fundamental part of your
overall project, and should be considered while the project is being designed.
You may decide that you will:
• carry out an initial baseline exercise against which to compare progress at the end of
the project
• refine the project on an ongoing basis; therefore, evaluation will be part of your regular
project activities (Mid-term Evaluation)
OR
• evaluate the project at agreed milestones, for example on a tri-monthly basis.
Apart from required reporting on the project, the timing and intensity of a project’s
evaluation is up to project participants.
Ex-ante evaluation (Project Appraisal)
• A term that refers to future events, such as future returns or prospects of a company.
• Using ex-ante analysis helps to give an idea of future movements in price or the future
impact of a newly implemented policy
• Conducted before the implementation of a project as part of the planning

9. Based on Timing
Ex-Post evaluation
• Conducted after the project is completed.
• It is based on analysis of past performance.
• Used to assess sustainability of project effects,
impacts.
• Identifies factors of success/failure to inform
future projects.

10. Based on Position of Evaluator
External evaluation
• Initiated and controlled by the supporter as
part of contractual agreement
• Conducted by independent people - who are
not involved in implementation
• Often assisted by project staff
Internal or self assessment
• Internally guided reflective processes
• Initiated and controlled by the group for its
own learning and improvement.

11. Based on Purpose
• Formative and Summative Evaluation
• Formative Evaluation - This evaluation takes
place During the project to guide future
development.
• Summative Evaluation - This evaluation takes
place After completion of a project to
appraise its success.

12. Project Evaluation Criteria (Example)
Link to Strategic Imperatives
Active Sponsor Engagement
Team Actively Engaged
Broad Organizational Awareness of the Project
Project Delivered the Anticipated Results
• Low – The deliverables of the project do not meet the expectations.
• Medium – Project has made substantial progress towards the initial goals , but overall the
deliverables fall short of expectations.
• High – The project delivered the promised results.
Project Completed on Time
• Low – While the project was eventually completed, the overall duration exceeded the
initial schedule by far.
• Medium – Project has been completed within a reasonable amount of time, but the
team has been struggling to complete specific phases.
• High – The team completed the project within the allotted time and the project leader
has managed the project effectively.
Successful Transition of Ownership to Process Owner
Improvement Sustained Over Time
Replication of Results
• Low – The team has not conducted a thorough analysis of whether and how the results of
this project could be replicated.
• Medium – The team has identified opportunities for replicating the results of the original
projects but does not have a plan on how to make this happen.
• High – The team has developed a thorough plan that not only shows how the improvements
could be replicated but also who will be involved.
https://www.isixsigma.com/implementatio
n/project-selection-tracking/10-criteria-
use-evaluating-six-sigma-projects/

13. Example of a project evaluation
http://unicef.org.np/uploads/files/4667126
55337210343-aligning-for-action.pdf

14. Methods of Financial Evaluations
The Five Methods/criteria more often used for
financial evaluation of a project are:
1. Simple Rate of return (SRR)
2. Payback Period (PBP)
3. Benefit Cost Ratio (BCR)
4. Net Present Value (NPV) or Net Present
Worth(NPW)
5. Internal Rate of Return (IRR).
a) Financial IRR
b) Economic IRR

15. Audit
• Audit: systematic, independent and
documented process for obtaining audit
evidence and evaluating it objectively to
determine the extent to which audit criteria
are fulfilled.

16. Types of Audit
• Management Audit
• Internal Audit
• External Audit
• Financial Audit
• Technical Audit
• Environmental Audit
• Social Audit

17. Technical Audit
A systematic, independent process for obtaining evidence and
evaluating this objectively to determine the extent to which needs or
expectations are fulfilled.
Technical audit is the process of maintaining the sustainability in the
quality of any Public development activities through the assessment of
cost effectiveness.
Technical audit is done to monitor the activities, which is done
according to the rules and regulation of Nepal Government, Norms of
development and contract document. The main purpose of audits is to
examine how well the planned or design criteria are met by the project
within the allocated time frame and budget. Technical Audit depends
upon the nature and volume of the public development activities.

18. Social Audit
The process of evaluating a firm's (or project’s) various
operating procedures, code of conduct, and other factors
to determine its effect on a society.
The goal is to identify what, if any, actions of the firm (or
project) have impacted the society in some way. A social
audit may be initiated by a firm that is seeking to improve
its image within the society. If the results are positive,
they may be released to the public.
If a factory (or project) is believed to have a negative
impact, the company may have a social audit conducted
to identify actions that actually benefit the society.

19. Social Audit
A formal review of a company’s (or project’s) endeavors in social
responsibility. Most of the big multinational funded projects requires
social audit.
Parameters of a social audit:
• charitable giving,
• volunteer activity,
• energy use,
• transparency,
• work environment,
• worker pay and benefits,
• gender balance,
• social inclusion
• economic impact to society
Social Audit helps company (or project) to portray positive public
image, which can play a part in project success/failure.

20. Environmental audit
Environmental audit (EA) means various types of evaluations intended to
identify environmental compliance and management system implementation
gaps, along with related corrective actions. In this way they perform an
analogous (similar) function to financial audits.
EA is an emerging area of concern in Nepal and helps to understand
environmental performance of the proponent and stakeholders involved in
implementing the development project. EA provides useful information about
the state of the environment on pre-project and post-construction stage in a
given area. The auditing information also provides a basis to select
environmental protection measures to address environmental problems of
similar nature in similar ecosystem and may contribute to the improvement
of environmental conditions.
The Environment Protection Act, 1996 and its Rules, 1997 urge to conduct
environmental auditing after two years of service provided by the project.
The Ministry of Population and Environment (MoPE) has undertaken
necessary steps to expedite environmental auditing in the
recent days.

21. Regulating and Monitoring Capacity Building for
Environmental Impact Assessment (EIA) of
Hydropower Project in Nepal
A Guide to
Environmental Auditing of
Hydropower Projects
Government of Nepal
Ministry of Environment, Science and Technology
With the assistance of
Royal Norwegian Government and technical assistance of
Norwegian Directorate for Nature Management
Kathmandu, Nepal
September 2006

22. Project Audit
• The project audit is a thorough examination of
the management of a project, its methodology
and procedures, its records, its properties, its
budgets and expenditures and its degree of
completion.
• The primary purpose of a project audit is to
find the reasons for uncomfortable symptom
in the project, and answer questions posed by
the sponsor or senior manager.

23. Depth of the Audit
• Time and money are two of the most common limits on
depth of investigation and level of detail presented in the
audit report.
– General audit - normally most constrained by time and
resources and is usually a brief review of the project touching
lightly on the six parts (phases) of an audit
– Detailed audit - usually conducted when a follow-up to the
general audit is required. When a general audit exposes a
serious problem in the project or in the methods by which the
project was conducted, a detailed audit may be required to
study such problems in detail and recommend solutions.
• Accumulation, storage, and maintenance of auditable data
are important cost elements.
• Two often overlooked issues are the self protective activity
of team members during an audit, and the potential for
project morale to suffer as a result of a negative audit.

24. Timing of the Audit
• The first audits are usually done early in the
project’s life.
• Early audits are often focused on the technical
issues in order to make sure that key technical
problems have been solved.
• Audits done later in the life cycle of a project
are of less immediate value to the project, but
are more valuable to the parent organization.

25. Timing of the Audit
• As the project develops, technical issues are less
likely to be matters of concern.
• Conformity to the schedule and budget become
the primary interests.
• Management issues are major matters of interest
for audits made late in the project’s life
• Post-project audits are often a legal necessity
because the client specified such an audit in the
contract.

26. Benefits of a Well-Done Project Audit
• Identify problems earlier
• Clarify performance/cost/schedule relationships
• Improve project performance
• Identify future opportunities
• Evaluate performance of project team
• Reduce costs
• Inform client of project status/prospects
• Reconfirm feasibility of/commitment to project

27. Major Tasks of a Project Audit
1. Evaluate if the project delivered the expected
benefits to all stakeholders.
• Was the project managed well?
• Was the customer/targeted beneficiary
satisfied?
2. Assess what was done wrong and what
contributed to successes.
3. Identify changes to improve the delivery of
future projects.

28. Types of Project Audits
(time wise)
In-Process Project Audits
• Allow for corrective changes if conditions have
changed and for concentration on project
progress and performance.
Post-Project Audits
• Take a broader and longer-term view of the
project’s role in the organization and
emphasize improving the management

29. The Project Audit Life Cycle
Six basic phases of a Project Audit Life Cycle:
1. Project audit initiation
• Focus and scope of audit; assess methodologies, team members
required
2. Baseline Definition
• Determine the standards against which performance will be measured
3. Establishment of Audit Database
• Gathering/organizing pertinent data; Focus on what is necessary
4. Data Analysis
• The judgment phase: Comparison of actual to standard
5. Audit Report Preparation
• Present findings to PM first: provides a chance to respond, and clarify
factual errors
• Then, prepare final report
6. Audit Termination
• Review of audit process
• Disbanding of team

30. Responsibilities of Project Auditor/Evaluator
• First and foremost the auditor should “tell the
truth”
• The auditor must approach the audit in an
objective and ethical manner
• Must assume responsibility for what is included
and excluded from consideration in the report
• The auditor/evaluator must maintain political and
technical independence during the audit and
treat all materials as confidential.

31. Essentials of an Audit/Evaluation
For an audit/evaluation to be conducted with
skill and precision, and to be generally
accepted by senior management, the client
and the project team, several conditions must
be met:
• The audit team must be properly selected
• All records and files must be accessible
• Free contact with project members must be
preserved

32. Challenges faced by professional project
auditors in developing countries
• Project culture: very weak documentation, treating every
information as confidential unless marked otherwise, considering
audit as unnecessary hassle in project development
• Institutional set up: frequent changes in personnel, budget
disbursement at the last minute, changes in departments
responsible for project completion, non-coordination among
institutions
• Legal: murky legal requirements
• Socio-economic: expectation of “favorable” evaluation, fee and
duration too low for in-depth auditing, suggestions for
improvements interpreted negatively
• Political: indirect pressure to play with words
• Inconsistencies and weaknesses in Standard Operating Procedure
• Audit-phobia: not volunteering information, data, documents

33. Difference between monitoring, evaluation and audit
http://www.aquaknow.net/en/water-toolkit-trainings/monitoring-evaluation-and-audit-water-
development-projects/monitoring-evaluation-and-audit-water-dev/15689

34. Log Frame Approach helps the design of
monitoring, evaluation and audit

35. Additional References
• http://www.bia.ca/systems-audit.htm
• http://wiki.answers.com/Q/What_do_you_ mean_by_project_audit
• http://www.ateec.org/services/evaluation/index.htm
• https://www.nsf-proj-eval-guide.org/html/home.htm
• http://www.projectsmart.co.uk
• http://www.syntropy.co.uk
• https://en.wikipedia.org/w/index.php?title=Evaluation&oldid=34193
0989
• https://en.wikipedia.org/w/index.php?title=Audit&oldid=341668393
• http://www.diffen.com/difference/Audit_vs_Evaluation
• http://www.differencebetween.net/business/difference-between-
audit-and-evaluation/
• http://www.differencebetween.info/difference-between-audit-and-
evaluation
• http://www.aquaknow.net/en/water-toolkit-trainings/monitoring-
evaluation-and-audit-water-development-projects/monitoring-
evaluation-and-audit-water-dev/15689