Monitoring and evaluation

  • Performance monitoring helps assess programme/policy implementation against a set of indicators. These should measure key outputs and results of the initiative. They should capture the evolution throughout the implementation process.
  • Evaluation makes judgements about the extent to which the programme or policy meets its objectives. These judgements are also made based on series of indicators. Evaluations do not only present the data on outputs and results; rather they assess whether these results can be considered as ‘good’ performance or not. Evaluations provide insights that should be used to refine and improve programmes or policies and make them more effective.

Defining what is to be monitored and/or evaluated

Step1. Defining the scope of an evaluation

The first step in developing a monitoring framework or an evaluation approach is defining what is to be covered. This implies clarity about the scope of the subject of monitoring or evaluation. This can concern aspects such as:

  • the exact activities covered
  • the period covered
  • the geographical scope of the evaluation
Step 2. Developing the programme theory or intervention logic

The second step is developing the programme theory, or the ‘intervention logic’. This explains how the programme is expected to work. It clarifies the logical chain from inputs and activities to outputs, results and impacts. It articulates why a certain activity is expected to lead to certain changes. It identifies the intermediary changes that are needed to achieve the expected results and impacts.

 
Step 3. Formulate evaluation questions

Based on the programme description, scope definition and intervention logic, it is important to formulate a set of evaluation questions. These should capture those aspects of the programme on which decisions need to be made. This depends very much on the stage of programme design and delivery, and the nature of decisions that the evaluation is expected to inform:

  • In some instances it may be more relevant to ask questions about the implementation process. For instance, this could be the case if the aim is to refine an ongoing programme and improve its implementation.
  • In others more emphasis will be put on the results and impacts. For instance, if the purpose of the evaluation is to know if a pilot programme has the desired impact, before deciding to establish the programme more widely.

Evaluation questions reflect the purposes of the evaluation, and help determine when to conduct the evaluation:

PURPOSE OF THE EVALUATION EXAMPLES OF EVALUATION QUESTIONS WHEN TO CONDUCT THE EVALUATION

- Determine whether activities are being implemented as intended

- E.g. What are the barriers faced by practitioners in the implementation of activities?

- As soon as programme / policy implementation begins

- Measure the results of the programme / policy

- E.g. What impact does the programme have on the rate of early leavers in our region?

- When (part of) the target group has participated in (some of) the planned activities.

- Assess if the programme / policy has attained its ultimate goals, e.g. reduce the rate of early leavers

- E.g. What effect does the programme have on participants’ attitudes towards school?

- At the end of the programme / policy or one of its cycles, e.g. when participants of a specific year have taken part in all the planned activities.
- The evaluation period covered has to take into account the time needed for impact to happen. To measure changes in early leaving rates, the evaluator needs to wait for participants to reach at least the theoretical age of completion of upper secondary education.