3. EVALUATION:
The collection and analysis of information to determine program performance,
i.e. “ finding out the value of something”
Evaluation of health program means a systematic and scientific process of determining the
extent to which an action or sets of objectives are achieved.
The process of evaluation involves 3 distinct stages –
vSelecting the attributes for evaluation
vSynthesizing of evidences
vFinal judgement
3
5. HEALTH:
For every evaluation there is a
need of statistics which is
usually called as health statistics
It is an aggregated
data describing and
enumerating
attributes, events,
behavior, services,
resources, outcomes
or cost related to
health, disease and
health services
A state of complete
physical, mental and
social well-being and not
merely an absence of
disease or infirmity
5
6. PROGRAM:
“an organized response to eradicate or eliminate or reduce one or more problems where
the response includes one or more objectives and the expenditure of resources”
Program is a set of procedures to conduct an activity, e.g. control of malaria
Project Appraisal:
Analysis undertaken prior to project implementation to estimate net benefit in relation to costs
PROGRAM PROJECT
Programs will typically span multiple functional units
within an organisation
Projects are typically confined to a single functional
unit within an organisation
Programs are executed over a much longer timescale than
projects, often several years
Projects are typically of a shorter duration often just
a few weeks, and by definition have a finite duration
Programs have a wide scope, focussing on benefits, and
may have to change scope dramatically during their
execution to meet the changing needs of the organisation
The scope of projects is tight – they are limited to
producing deliverables
6
7. PURPOSE OF EVALUATION
v To improve health plans, health programs, health infrastructure and health services delivery
v To assess the need for funding or further coordination of fund
7
8. RESPONSIBILITY FOR EVALUATION
vThe planners
vAd hoc research groups
vThose responsible for health development
vThose responsible for implementation
vThe community itself, especially wherever primary
health programs are being carried out
8
9. TYPE OF EVALUATION
Based on the purpose, progress and
components and activities, evaluation can
be divided into:
Total Partial
Time related
a) Initial
b) Mid-term
c) Final/ terminal
In relation to planning process:
a) Baseline or bench mark evaluation
b) Appraisal evaluation
c) Concurrent evaluation
According to agency
a) Internal evaluation (sometimes called self
evaluation), in which people within a
program sponsor, conduct and control the
evaluation.
b) External evaluation in which someone from
Outside the program.
9
11. Formative evaluation:
Evaluate the degree of need for the program, or the activities used by the program to achieve
its desirable outcomes, but without evaluating the degree of outcome
Summative evaluation:
Evaluation of the degree to which a program has achieved its desired outcomes, and the
degree to which any other outcomes (positive or negative) have resulted from the program
EVALUATION CAN BE FORMATIVE OR SUMMATIVE
11
13. EVALUATION CAN BE PROSPECTIVE OR RETROSPECTIVE
Prospective Evaluation: when a new program being introduced
Retrospective Evaluation: when programs have been functioning
for some time
13
14. 1. Review of records from National Health Information systems,
2. Monitoring: Input/efforts analysis, Management by Objectives, Gantt
Charts, Program Evaluation and Review Technique (PERT), and the Critical
Path Methods (CPM) of analysis,
3. Case Studies,
4. Qualitative studies,
5. Sample surveys: Cross-Sectional studies,
6. Cohort studies,
7. Panel studies.
8. Controlled experiments and intervention studies
TOOLS OF EVALUATION
14
16. FISHBONE DIAGRAM:ROOT CAUSE ANALYSIS
16
“ Root cause analysis is a structured team process that
assists in identifying underlying factors or causes of an
adverse event or near-miss ”
18. DELAY FOR OPD
CONSULTATION
PEOPLE PROCESS
EQUIPMENT MANAGEMENT
Delay in MRD file
Appointment system (only 10 patients)
Wrong reporting
Busy schedule
of doctors
Communication
gap between
Doctor and Staff
OT
Cases
Rounds
Emergency Cases
Walk in patients
Patients take time in filling
registration form
Language problem
Breakdown of equipment
HIS system is slow
Non availability of pen
Unexpected leave by consultant
Difficulty in taking lift and finding place
Queue
system
not
followed
Non
availabil
ity of
Queue
barriers
Height of the desk is
more
Delay in Registration process
Doctors will be available in IMS instead of
HRC OPD (Sometimes)
19. All the steps and standards of evaluation help in fulfilling the major objective of any
health program.
MAJOR OBJECTIVES OF HEALTH PROGRAM:
v Accessibility
v Equitability
v Quality
v Effectiveness
v Efficiency
v Sustainability
19
20. ACCESS:
Measure of the extent to which a population can reach the health services it
needs delivered by either the public or private sector.
v Financial access- (also referred to as economic access) measures the extent
to which people are able to pay for health services.
v Physical access- (also referred to as geographic access) measures the extent
to which health services are available and reachable.
20
21. EQUITY:
Gender
equality
Equity
Human
Rights
It is a normative issue that refers to fairness in the allocation of resources or the
treatment of outcomes among different individuals or groups
v Horizontal equity- “equal treatment of equal needs”
v Vertical equity- concerned with the extent to which individuals with different
characteristics should be treated differently
21
22. QUALITY
Which is expected to maximize an inclusive measure of patient welfare.
Three aspects of quality ‘structural’, ‘process’ and ‘outcome’
‘Structural’ quality refers to whether appropriate resources are in place to provide health care
of a minimum standard (personnel trained for their tasks, well maintained equipment and
buildings, a regular drug supply)
‘Process’ quality generally refers to activities occurring during the interaction between the
health system and the client (i.e. Whether good quality Care is actually delivered).
‘Outcome’ quality in addition to health status, can include patient satisfaction and perceived
quality.
Outcome flows from process and refers to “changes in a patient’s current and future health
status that can be attributed to the antecedent health care.”
22
24. Effectiveness
Effectiveness is the extent to which planned outcomes, goals, or objectives are
achieved as a result of an activity, strategy, intervention or initiative intended to
achieve the desired effect, under ordinary circumstances
Efficiency
Refers to obtaining the best possible value for the resources from all stakeholders and sectors used
or using the least resources to obtain a certain outcome
- Technical efficiency(operational level , methods, manpower ,finance ,facilities etc)
- Cost efficiency(cost of vaccine, staff time, transportation, publicity is Rs.1000
a total of 100 children got vaccination, then cost efficiency ratio is Rs.10/Vac24
25. We must understand what is cost and various types of costs involved in program
Cost :factor of production
It is a monetary valuation applied to an asset or service that has been obtained by an
expenditure of cash by a commitment.
Cost-Direct
Indirect Cost
Fixed Cost
Incremental Cost
Cost Operating
Opportunity Cost
Production Cost
Recurrent Cost
Cost effective Analysis
Cost-benefit Analysis
Cost-utility Analysis
25
26. SUSTAINABILITY
Capacity of the system to continue its normal activities well into the future
vFinancial sustainability is the capacity of the health system to maintain an
adequate level of funding to continue its activities
vInstitutional sustainability refers to the capacity of the system, if suitably
financed to assemble and manage the necessary resources to successfully carry
on its normal activities in the future.
26
27. LOGIC MODEL OF PROGRAM EVALUATION
PROCESS
Short intermediate long
OUTCOMES
Inputs
ASSUMPTIONS/ CONTEXTUAL FACTORS
Activities Outputs
27
29. Inputs: resources that go into a program or intervention—what we invest.
They include financial, personnel, and in-kind resources from any source.
e.g. Various funding sources for program
Your partners
Staff time
Technical assistance
Activities: events undertaken by the program or partners to produce desired outcomes—
what we do, e.g. Create a state-level partnership
Train health care partners
Staff in clinical guidelines
Develop a community health communication campaign on signs and symptoms of
stroke, and to call 9-1-1.
29
30. Outputs are the direct, tangible results of activities—what we get.
These early work products often serve as documentation of progress.
e.g. State-level partnerships created
Health care professionals trained in clinical guidelines
Community health communication campaigns developed.
Outcomes are the desired results of the program—what we achieve.
Describing outcomes as short, intermediate, or long term depends on the objective, the length
of the program, and expectations of the program or intervention.
Short-term outcomes are the immediate effects of the program or intervention activities.
They often focus on the knowledge and attitudes of the intended audience
e.g. Increase partner knowledge of HDSP priorities and strategies
Increase physician knowledge of clinical guidelines
Increase knowledge of signs and symptoms of stroke and of the need to call 9-1-1.
30
31. Intermediate outcomes are behaviour, normative, and policy changes.
e.g. HDSP State Plan has been developed and published with partner involvement.
Health systems implement clinical guideline
Decrease transport time to treatment for stroke victims.
Long-term outcomes refer to the desired results of the program and can take years to accomplish.
e.g. Increase in state wide policy and environmental strategies for HDSP.
Increase in blood pressure control in a health centre population.
Increase in early treatment for stroke.
Impacts refer to the ultimate impacts of the program. Take 10 or more years to achieve.
e.g. Decrease in the rate of death due to heart disease.
Eliminate disparities in treatment for stroke between general and priority populations.
31
32. Assumptions are the beliefs we have about the program or intervention and the resources involved.
Assumptions include the way we think the program will work—the "theory" we have used to develop
the program or intervention.
Examples • Funding will be secure throughout the course of the project.
• Because we teach information, it will be adopted and used in the way we intended.
• Professionals will be motivated to attend learning sessions.
• External funds and well-placed change agents can facilitate institutional change.
• Staff with the necessary skills and abilities can be recruited and hired.
• Partnerships or coalitions can effectively address problems or reach into areas we cannot.
Contextual Factors describe the environment in which the program exists and external factors that
interact with and influence the program or intervention. These factors may influence implementation,
participation, and the achievement of outcomes. Contextual factors are the conditions over which we
have little or no control that affect success.
32
36. STRENGTHS
v Team of trained
and qualified
trainers.
vInnovation and execution
vJustifying support
function for all field
WEAKNESSES
v Scattered focus/
involved in several
unrelated activities.
vShortage of Aptitude
trainer.
v Shortage of
communication
v Poor infrastructure
OPPORTUNITIES
Government tie up
with
NGOs for
evaluation and
implementation of
health program
CHALLENGES
v Have the chance
of being overcome
v How program
need to be handled
appropriately
36
