MODULE 4

Monitoring & Evaluation

Monitoring is the systematic, regular collection and occasional analysis of information to
identify and possibly measure changes over a period of time.
Evaluation is the analysis of the effectiveness and direction of an activity and involves making
a judgment about progress and impact.
PURPOSE OF MONITORING & EVALUATION

➢ To assess project results - to find out if and how objectives are being met and are resulting
in desired changes.

➢ To improve project management and process planning: to better adapt to contextual and risk
factors such as social and power dynamics that affect the research process.

➢ To promote learning:

• To identify lessons of general applicability

• to learn how different approaches to participation affect outcomes, impact, and reach
• to identify what contextual factors enable or constrain the participatory research

➢ To understand different stakeholders' perspectives:

➢ To ensure accountability: to assess whether the project is effectively, appropriately, and

efficiently executed.
DEFINITION OF MONITORING
Monitoring means keeping a track of implementation process. Monitoring involves watching
the progress of a project against time, resources and performance schedules during the
execution of the project and identifying lagging areas requiring timely attention and action.
Monitoring means periodic checking of progress of works against the targets laid down in order
to ensure timely completion of the project.
PURPOSE OF MONITORING

➢ Project monitoring helps to provide constructive suggestions like.

➢ Rescheduling the project (if the project run behind the schedule)

➢ Re budgeting the project (appropriating funds from one head to another; avoiding expenses

under unnecessary heading).

➢ Re – assigning the staff (shifting the staff from one area to other; recruiting temporary staff

to meet the time schedule)

Monitoring Steps
Step 1: Develop objectives
Step 2: Design and implement management
Step 3: Design the monitoring methodology
Step 4: Implement monitoring as a pilot study
Step 5: Implement monitoring
Step 6: Report and use result
Types of monitoring
Monitoring is the systematic and routine collection of data during project implementation for
the purpose of establishing whether an intervention is moving towards the set objectives or
project goals.
In this case, data is collected throughout the life cycle of the project.
Types

➢ Process monitoring/ physical progress monitoring

➢ Technical monitoring

➢ Assumption monitoring

➢ Financial Monitoring

➢ Impact Monitoring

➢ Result Monitoring

Definition of Evaluation
Evaluation is the process for determining the value of anything. Project evaluation is the
determination of the extent to which the desired objectives have been attained or the amount
of movement that has been made in the desired direction. Evaluation is a comparison of the
situation before and after a project has operated within it for a predetermined period.
Purpose of Evaluation
From an accountability perspective:

➢ To make the best possible use of funds by the program managers who are accountable for

the worth of their programs.

➢ Measuring accomplishment in order to avoid weaknesses and future mistakes.

➢ Observing the efficiency of the techniques and skills employed

➢ Scope for modification and improvement.

➢ Verifying whether the benefits reached the people for whom the program was meant.

Form a knowledge perspective:

➢ To establish new knowledge about social problems and the effectiveness of policies and

programs designed to alleviate them.

➢ Understanding people’s participation & reasons for the same. Evaluation helps to make
plans for future work.
STEPS IN EVALUATION

➢ Learn about the program

➢ Creating an evaluation plan and evaluation indicators

➢ Briefing the concerned people about the evaluation plan and indicators

➢ Revising and elaborating the evaluation plan

➢ Initiating evaluation

➢ Utilizing /sharing the information

PROJECT REVIEW
A Project Review is an assessment of the status of a project, at a particular point in time.
During this project review, a decision is made as to whether or not the team has met the
objectives and is approved to proceed to the next project phase.
There will usually be at least three project reviews during the project life cycle: at the end of
the Initiation, Planning and Execution project phases.
Points to Review

➢ Determine whether the project goals were achieved.

➢ Determine the satisfaction of stakeholders

➢ Determine the project's costs and benefits

➢ Identify areas for further development.

➢ Identify lessons learned.

➢ Report findings and recommendations

How to Review
Define the scope of the review beforehand - Clarify your objectives for the review, and make
your intentions clear.
Review key documents - Gather together the key project documents
Consider using independent reviewers - If possible, use outside people in your review process
to get an objective, unclouded view of the project.
Use appropriate data collection - Collect information in the most appropriate way, for example,
by using interviews and surveys.
Deliver appropriate reports – Report your findings, and publicize the results.
Present recommendations – Present the detailed recommendations to the organization and the
project
leaders, as well as to customers and other stakeholders.
Variance Analysis
Variance analysis is the quantitative investigation of the difference between actual and planned
behaviour.
This technique is used for determining the cause and degree of difference between the baseline
and actual performance and to maintain control over a project.
Cost and schedule variances are the most frequently analysed measurements.
Variance analysis is the variation between plan and actual project performance. It further helps
to identify causes and assess severity of deviation.
STEPS IN VARIANCE ANALYSIS

➢ Identify the affected key performance indicators.

➢ Assess the quantum of deviation.

➢ Estimate the degree of impact on project performance.

➢ Identify the causes of variation.

➢ Establish the corrective action.

➢ Estimate the resources required to implement the corrective action.

➢ Establish time schedule required to implement the corrective action.

➢ Recommend preventive action.

VARIANCE ANALYSIS IN PROJECT MANAGEMENT

PMBOK 5th edition identifies variance analysis as one of the eleven analytical techniques. It
is an effective tool to control various aspects of project performance such as scope, schedule,
cost and risk.
• Scope Control
• Schedule Control
• Cost Control
• Risk Control
CAUSES OF VARIANCE IN PROJECTS
● Changes in project scope triggered by end-user or project team in order to meet the
contractual obligations.
● Lack of resources such as skilled manpower, availability of equipment and material.
● Wrong activity duration estimates.
● Improper identification of critical schedule activities.
● Improper project reviews.
● Lack of or poorly implemented project monitoring and controlling processes.
PERFORMANCE ANALYSIS
Performance analysis is a measure of the success or failure of a project using various
parameters. It helps in developing a positive culture of project management that yields
excellent results. A good program performance typically needs:
• Proper management of stakeholders
• An effective strategy for program process.
• Good execution
• Strong program teams
• Change management
Metrics for Performance Analysis
• Return on Investment
• Productivity
• Cost of Quality
• Cost Performance
• Schedule Performance Index (SPI)
Types of Evaluation
Evaluation
Evaluation can be defined as a process which determines as systematically and as objectively
as possible the relevance, efficiency, sustainability and impact of activities in the light of a
project focusing on the analysis of the progress made towards the achievement of the stated
objective.
TYPES
• Formative Evaluation
• Process Evaluation
• Outcome Evaluation
• Summative Evaluation
• Impact Evaluation
• Real Time Evaluation
• Participatory Evaluation
• Thematic Evaluation
• Cluster or Sector Evaluation
• Meta Evaluation
Internal Evaluation
• Self- and internal evaluation is defined by EVAL as an evaluation that is coordinated by
someone internal to the project or programme for the purposes of organizational learning.
• Self-or internal evaluations may be conducted by someone who is part of the project
management team (self-Kit evaluation), a qualified ILO official (internal evaluation), or an
external consultant (internal evaluation).
• The main difference between independent and self- or internal evaluation is evaluation
management. In self- and internal evaluation, the project team initiates the work and selects
an evaluation coordinator.
External evaluation
• External project evaluation is the classical form of evaluation. In this context “external”
means that the evaluators are not project members, but experts from the North, South or East
who, owing to their specific skills and in line with specifications, are called in for a period of
one to three weeks to investigate the defined questions.
• External evaluations are therefore suitable for factual issues, relatively complex situations,
and overall appraisals.
• With their outsider’s view the external evaluators use appropriate methods to collect
information on the project and its environment, in order to assess and answer the questions
posed.
EVALUATION FOCUS
At the planning stage of a project decisions can be made about what to focus on in particular
evaluations. The focus might be:
• geographic area
• Groups of beneficiaries
• Types of activities
• Time period
A particular kind of activity over a group of projects to compare effectiveness (a thematic
evaluation) a group of projects all working in the same area to see how they interact with each
other.
STEPS IN EVALUATION
• Develop program logic and review needs
• Develop the evaluation brief
• Commission the evaluation project
• Manage development of the evaluation design
• Manage development of the evaluation work plan
• Manage implementation of the work plan, including production report(s)
• Disseminate report and support use of the evaluation
DIFFERENCE BETWEEN MONITORING AND EVALUATION

MONITORING EVALUATION
Continuous: start and ends with a One shot operation: At a point of time
programme. [usually at completion or mid-way of
programme.
Required for immediate use and midcourse Used for future planning / replication/
correction expansion
Done by implementing personnel Usually by outside agency
Quick but covers all units In-depth; covers a sample
 Correcting / managing Learning process
Symptomatic, early warning system Diagnostic

CRITERIA FOR EVALUATION

Relevance: Assesses the extent to which the objectives of a development intervention are
aligned and consistent with target beneficiaries’ needs and requirements, national needs and
priorities, global priorities and partner’ and donor’s policies.

Effectiveness: Measures the progress of the project/program implementation towards

attainment of set objectives, i.e. a comparison of the actual attained results vs. the planned
results in the results framework. Also looks at incorporation of: results based approach, human
rights based approaches and cross cutting issues in programming (design and implementation).

Efficiency: Measures the outputs (qualitative and quantitative) of a program/project in relation

to the inputs (resources; financial, technical and time) i.e. Value for money and
efficiency/utilisation ratio of the utilised resources.

Impact: Assessing impact involves examining in detail, the main effects resulting from the
intervention on the local social, economic, environmental and other development indicators;
disaggregated by sex or other relevant social groups, such as minorities.

Sustainability: Assesses the continuation of benefits from a development intervention after

major development assistance (mostly donor funding) has been completed or withdrawn, i.e.
the probability of continued long-term benefits of the project, after the project ends.

STEPS IN EVALUATION PROCESS

Planning: The relevant questions during evaluation planning and implementation involve
determining the feasibility of the evaluation, identifying stakeholders, and specifying short-
and long-term goals.

• Defining and identifying stakeholders is a significant component of the planning stage.

Stakeholders are people or organizations that have an interest in or could be affected by
the program evaluation. They can be people who are involved in program operations,
people who are served or affected by the program, or the primary users of the
evaluation. The inclusion of stakeholders in an evaluation not only helps build support
for the evaluation but also increases its credibility, provides a participatory approach,
and supplies the multiple perspectives of participants and partners.
• Stakeholders might include community residents, businesses, community-based
organizations, schools, policy makers, legislators, politicians, educators, researchers,
media, and the public. For example, in the evaluation of a program to increase access
to healthy food choices in and near schools, stakeholders could include store merchants,
 school boards, zoning commissions, parents, and students. Stakeholders constitute an
important resource for identifying the questions a program evaluation should consider,
selecting the methodology to be used, identifying data sources, interpreting findings,
and implementing recommendations.
• Once stakeholders are identified, a strategy must be created to engage them in all stages
of the evaluation. Ideally, this engagement takes place from the beginning of the project
or program or, at least, the beginning of the evaluation. The stakeholders should know
that they are an important part of the evaluation and will be consulted on an ongoing
basis throughout its development and implementation. The relationship between the
stakeholders and the evaluators should involve two-way communication, and
stakeholders should be comfortable initiating ideas and suggestions. One strategy to
engage stakeholders in community programs and evaluations is to establish a
community advisory board to oversee programs and evaluation activities in the
community. This structure can be established as a resource to draw upon for multiple
projects and activities that involve community engagement.
• An important consideration when engaging stakeholders in an evaluation, beginning
with its planning, is the need to understand and embrace cultural diversity. Recognizing
diversity can improve the evaluation and ensure that important constructs and concepts
are measured.

Implementation — Formative and Process Evaluation: Evaluation during a program’s

implementation may examine whether the program is successfully recruiting and retaining its
intended participants, using training materials that meet standards for accuracy and clarity,
maintaining its projected timelines, coordinating efficiently with other ongoing programs and
activities, and meeting applicable legal standards. Evaluation during program implementation
could be used to inform mid-course corrections to program implementation (formative
evaluation) or to shed light on implementation processes (process evaluation).

For community-engaged initiatives, formative and process evaluation can include evaluation
of the process by which partnerships are created and maintained and ultimately succeed in
functioning.

Completion — Summative, Outcome, and Impact Evaluation: Following completion of the

program, evaluation may examine its immediate outcomes or long-term impact or summarize
its overall performance, including, for example, its efficiency and sustainability. A program’s
outcome can be defined as “the state of the target population or the social conditions that a
program is expected to have changed, Once the positive outcome of a program is confirmed,
subsequent program evaluation may examine the long-term impact the program hopes to have.

Dissemination and Reporting: To ensure that the dissemination and reporting of results to all
appropriate audiences is accomplished in a comprehensive and systematic manner, one needs
to develop a dissemination plan during the planning stage of the evaluation. This plan should
include guidelines on who will present results, which audiences will receive the results, and
who will be included as a coauthor on manuscripts and presentations. Although the content and
format of reporting may vary depending on the audience, the emphasis should be on full
disclosure and a balanced assessment so that results can be used to strengthen the program.
Dissemination of results may also be used for building capacity among stakeholders.

RELEVENCE OF PERSONNEL MANAGEMENT

Personnel management can be defined as obtaining, using and maintaining a satisfied workforce. Itis a
significant part of management concerned with employees at work and with their relationship within the
organization. According to Flippo, “Personnel management is the planning, organizing,
compensation, integration and maintenance of people for the purpose of contributing to organizational,
individual and societal goals.” According to Brech, “Personnel Management is that part which is
primarily concerned with human resource of organization.
• Management is a process whereby various resources are integrated into a total system
for objective accomplishment. It tries to bring about a co-ordinated effort of many
individuals or many groups to achieve specified goal. Management is a 'System of
Authority', a legal right to command or direct others to act in a prescribed manner.
Qualified and trained personnel selected as managers manage the organization.
Management process is guided by definite objectives; performance of organized
activities; maintenance of certain definite relationships.

• Management process involves decisions to get the tasks done through resources. These
decisions are the policies which help the organization and the personnel work smoothly
towards achievement of , organizational goals. The word 'personnel' in an organization
is used as a synonym to 'manpower' or people. Megginson defined 'Manpower' as the
total knowledge, skills, creative abilities, talents and aptitudes of an organization's work
force as well as the value, attitude and beliefs of the individuals involved.
• Manpower management includes planning, selection, training and development of
human resources of the organisation. Personnel management is the management of
manpower. This deals with planning, organizing, directing and controlling the
personnel functions of the organization. It is a major part of the managerial process. A
personnel department facilitates personnel management.
• To form an Organisation there is a need for more people. These people should be
suitable to accomplish the tasks specified to achieve the organizational objectives. The
personnel management as a major sub-system or an expanded area of the organization
is concerned with the recruitment, selection, development, utilisation and motivation of
manpower towards the success of the organization. To make the tasks easy, personnel
departments are established whose managers are entrusted with the responsibilities of
managing the problems of the personnel employed in that department. Nursing service
department is an example in which nursing manpower is appointed and their personnel
matters are dealt with by the personnel manager.
• Personnel Management is people-centred and its objective is the maintenance of better
human relations in. the organization by development, application and evaluation of
 personnel policies, procedures and programmes relating to the individuals in the
organization to optimize the contribution towards organizational objectives.
• Its aim is to make the best use of the human resources for which it includes the
recruitment of right kind of people according to jobs, their training and development to
do the work, their welfare to maintain their morale and incentives to boost their interest
in order to reach the ultimate objectives of the organization.