Evaluation of Projects
Evaluation
Evaluation
Judging, appraising, determining the worth,
value, or quality of a project to make
necessary decisions.
In terms of:
• Relevance
• Effectiveness
• Efficiency
• Sustainability
• Impact
Five Strategic Evaluation Questions
• Relevance: The extent to which the objectives of a
development intervention are consistent with
beneficiaries´ requirements, and partner’s and donor’s
policies.
• Effectiveness: doing right things
• Efficiency: doing things right
• Sustainability: Meeting needs without compromising
the ability of future generations to meet their needs
(project will continue after donors intervention)
Impact: Positive or negative, direct or indirect long-
• term impact produced by a development intervention.
Evaluation Provides Information
Strategy: Are the right things being done?
• Rationale or justification
• impact
Operations: Are things being right?
• Effectiveness in achieving expected outcomes
• Efficiency in optimizing resources
• Client satisfaction
Learning: Are there better ways?
• Alternatives
• Lessons learned
Forms of Evaluation
• Formative evaluation
Evaluation intended to improve performance, most
often conducted during the design and/or
implementation phases of projects or programs.
• Summative evaluation
A study conducted at the end of an intervention
to determine the extent to which anticipated
outcomes were produced.
Types of Evaluation
• Participatory evaluation
• Process based evaluation
• Outcome based evaluation
• Self-evaluation (360 degree evaluation)
• Goal-based evaluation
Types of Evaluation
• Participatory evaluation
Evaluation in which representatives of agencies and
stakeholders work together in designing, carrying
out and interpreting an evaluation.
• Process based evaluation
An evaluation of the internal dynamics of a project, its
policy instruments, its service delivery mechanisms, its
management practices, and the linkages among these.
Types of Evaluation cont….
• Outcome based Evaluation
it facilitates the asking if the organization is
doing right activities to bring about the
expected outcomes
• 360 degree Evaluation (evaluation by planner
& implementers)
An evaluation by those who are entrusted
with the design and delivery of a project.
• Goal based evaluation
Evaluating the extent to which projects are
meeting predetermined goal / impact
Relationship of Monitoring
and Evaluation
Relationship of Monitoring and
Evaluation Evaluation
Monitoring
Information Informatio
from n from
Recording
Monitoring other
sources
Analysis Information
Analysis
Reporting Recordin Recommendations
g
Corrective action at the Affirmation or modification in
operational level objectives, resources, and process
Principles of
1. Evaluation is a Evaluation
continuous process (continuity).
2. Evaluation should involve minimum possible costs (inexpensive).
3. Evaluation should be done without prejudice to day to day
work
(minimum hindrance to day to day work).
4. Evaluation must be done on a co-operative basis in which the entire
staff and the board members should participate (total participation).
5. As far as possible, the agency should itself evaluate its program but
occasionally outside evaluation machinery should also be made use of
(external evaluation).
6. Total overall examination of the agency will reveal strength
and
weaknesses. (agency / program totality).
7. The result of evaluation should be shared with workers of the agency
(sharing).
Stages in
1. Program Planning Stage.
Evaluation.
• Pre – investment evaluation or
• Formative evaluation or
• Ex – ante evaluation or
• Pre project evaluation or
• Exploratory evaluation or
• Need assessment.
2.Program Monitoring Stage.
• Monitoring Evaluation or Ongoing / interim.
• Concurrent evaluation
3.Program completion Stage.
• Impact evaluation or
• Ex- post evaluation or (Summative / Terminal /
Final)
Steps in
Evaluation:
Learning about the program
Creating on evaluation plan & Evaluation indicators
Briefing the concerned people about the evaluation plan & indicators
Revising and elaborating the evaluation plan
Initiating Evaluation
Utilizing / Sharing the Information
Types of
Evaluation can be categorized under different headings
Evaluation
A) By timing (when to evaluate)
• Formative Evaluation
• Done during the program -Development stages
• (Process Evaluation, ex-ante evaluation, project appraisals)
• Summative Evaluation
• Taken up when the program achieves a stable of operation or when it is
terminated
• (Outcome evaluation, ex post evaluation etc.)
B) By Agency. Who is evaluating?
• Internal Evaluation External Evaluation
• It is a progress / impact Unbiased, objective detailed
• Monitoring by the management it self assessment by an outsider
• (Ongoing / concurrent evaluation)
C) By Stages
• On going Terminal Ex – post
• During the implementation At the end of After a time lag
of a project or immediately from completion
after the completion of a project
Types of Evaluation
De sire d Situa
Sus ta ine d be
tion nefits and
impa ct
Pre sent Situa Time
tion
Mid-Te rm
End-o f pro ject or Ex-po s t or impact
review final evaluation evaluation
Internal / External
InternalEvaluation:
Evaluation: (Enterprise Self Audit)
• Internal evaluation (or otherwise monitoring, concurrent evaluation) is a
continuous process which is done at various points and in respect of
various aspects of the working of an agency by the agency staff
itself i.e. staff board members and beneficiaries.
External / Outside Evaluation: (This is done by outsiders
/Certified Management Audit)
• Grant giving bodies in order to find out how the money given is utilized by the
agency or how the program is implemented sent experienced and qualified
evaluators (inspectors) to assess the work E.g. Central social welfare Board
• Some donors may send consultants in order to see how far the standards laid
down are put into practice.
• Inter agency evaluation. In this type two agencies mutually agree
to evaluate their program by the other agency.
• Inter agency tours.
Methods of Evaluation:(Tools / techniques)
Over the years, a variety of the methodologies have been evolved by academicians, practitioners and professionals for
evaluating any program / project. Some of the commonly used practices are given below.
First hand Information :
• One of the simplest and easiest methods of evaluation by getting first hand information about the progress,
performance, problem areas etc,. of a project from a host of staff, line officers, field personnel, other specialists and
public who directly associated with the project. Direct observation & hearing about the performance and pitfalls
further facilitate the chances of an effective evaluation.
Formal / Informal Periodic Reports.
Evaluation is also carried out through formal and informal reports.
• Formal reports consists of
-Project Status Report
-Project Schedule chart
-Project financial status Report.
Project Status Report:
• From this one can understand the current status, performance, schedule, cost and hold ups, deviations from
the original schedule.
Project Schedule Chart:
• This indicates the time schedule for implementation of the project. From this one can understand any delay, the cost
of delay and the ultimate loss.
Project Financial Status Report:
• It is through financial report, one can have a look at a glance whether the project is being implemented
within the realistic budget and time.
Informal Reports:
• Informal reports such as anonymous letters, press reports, complaints by beneficiaries & petitions
sometimes reveal the true nature of the project even though these reports are biased and contains maligned
information.
Graphic presentations:
• Graphic presentations through display of Charts, Graphs, Pictures, Illustrations etc. in the project office is yet
another instrument for a close evaluation.
Standing Evaluation Review Committees:
• Some of the organizations have setup standing committees, consisting of a host of experts and specialists
who meet regularly at frequent intervals to discuss about problems and to suggest remedial measures.
Project Profiles:
Views about
evaluation
Evaluation primarily perceived from three perspectives.
• Evaluation as an analysis – determining the merits
or deficiencies of a program, methods and process.
• Evaluation as an audit – systematic and continuous
enquiry to measure the efficiency of means to reach
their particular preconceivedends.
In the agency context
• Evaluation of administration means appraisal or judgement of
the worth and effectiveness of all the processes
(e.g. Planning, organizing, staffing etc.) designed to ensure
the agency to accomplish its objectives.
Areas of
evaluation:
Evaluation may be split into various aspects, so that each area of the work of the
agency, or of its particular project is evaluated. These may be,
1.Purpose 2.Programs 3.Staff 4.Financial Administration
5.General.
Purpose:
• The review the objectives of the agency / project and how far these are being
fulfilled.
Programs:
• Aspects like number of beneficiaries, nature of services rendered to them, their
reaction to the services, effectiveness and adequacy of services etc. may be
evaluated.
Staff:
• The success of any welfare program / agency depends upon the type of the staff an
agency employs. Their attitude, qualifications, recruitment policy, pay and other
benefits and organizational environment. These are the areas which help to
understand the effectiveness of the project / agency.
Financial Administration:
• The flow of resources and its consumption is a crucial factor in any project /
agency. Whether the project money is rightly consumed any over
spending in some headings, appropriation and misappropriation. These are
some of the indicators that reveal the reasons for the success or failures of
any project.
General:
• Factors like public relations strategies employed by the project / agency, the
constitution of the agency board or project advisory committee and their
contribution future plans of the agency are important to understand the success or
failures of any project.
Criteria for Evaluating Development
Assistance
Relevance Effectiveness
= The extent to which the aid intervention is = A measure of the extent to which an aid
suited to the priorities and policies of the intervention attains its objectives
target group, partner country and donor Possible questions:
To what extent were the objectives
Possible questions:
achieved/are likely to be achieved?
To what extent are the objectives of the
What were the major factors influencing the
program still valid?
achievement or non-achievement of the objectives?
Are the activities and outputs of the
program consistent with the overall goal Impact
and the attainment of its objectives? = The positive and negative changes produced by
Are the activities and outputs of the an intervention, directly or indirectly, intended or
program unintended.
consistent with the intended impacts and Possible questions:
effects? What has happened as a result of the programme
or project?
Efficiency What real difference has the activity made to
= Efficiency measures the outputs – the beneficiaries? How many people have been
qualitative affected?
and quantitative – in relation to the inputs.
It is a term which signifies that the aid uses
Sustainability
= Sustainability is concerned with measuring
the Least costly resources in order to achieve whether the benefits of an activity are likely to
the Desired results. This generally requires continue after donor funding has been
Comparing alternative approaches withdrawn.
to achieving the same outputs, to see Possible questions:
whether the most efficient process has been To what extent did the benefits of a programme or
adopted project continue after donor funding ceased?
Possible questions: What were the major factors which influenced the
achievement or non-achievement of sustainability
Were the activities cost-efficient?
of the program or project?
Were objectives achieved on
time?
Difference between Monitoring and Evaluation
Monitoring Evaluation
Object To track To validate what results
ive changes were achieved, and how
from baseline
conditions to and why they were or were
desired outcomes. not achieved.
Metho Tracks and Evaluates achievement or
assesses outcomes by comparing
dology performance indicators before and after
through analysis the intervention.
and comparison Involves Value Judgment
of indicators over Relies on monitoring
time.
data
and information from
Continuous and Time-bound,
systematic by periodic, in-depth.
Charact
Programme/
eristics
Project Internal or External
Managers and evaluators and
key partners. partners.
Uses Alerts managers Provides managers /
about problems in Donors /
performance, stakeholders with
provides options strategy
for corrective actions and policy options,
and helps provides basis for
demonstrate learning and
accountability. demonstrates
accountability.
Key Uses of M&E
• Improve performance of ongoing
• projects Accountability
— Warn of deviations from goals
— Review inputs, activities, and
results
— Performance Reporting
• Decision
m–akiInmgproving
– implementation
– Periodic review
Improving planning
What to Monitor?
• Focus on key needs of management
and different stakeholders
• Maintaining minimum accountability
and transparency standard
• Key outputs of the project and
program
Lessons
Learnt
• There should be:
– No activities without
records
– No records without
analysis
– No analysis without
learning
– No learning without
action
