Monitoring and Evaluation

The lack of interaction and instant feedback from users of information mean that monitoring and evaluation are essential parts of the process of information provision. An adviser can quickly check if a client hasn't understood. A teacher can spot the glazed look of incomprehension and try another explanation. But information staff have no immediate way of knowing whether their information has been understood or has answered a user's questions.

It is useful to treat monitoring and evaluation as linked but distinct activities. Below we discuss them separately and explain the differences between them.

Monitoring

Monitoring the work of a project is essential if you want to evaluate its effectiveness.

Monitoring allows you to know how many people saw or used your information, and basic details about them. It may include some details on who it was that used your information and what for, and feedback from advisers or intermediaries about how they believe it was received by your audience. These details will help you design your evaluation.

All of this will help provide the core knowledge you will need for your evaluation.

Evaluation

Most evaluation that does occur tends to focus on the piece of information itself. For example, is it presented in an attractive manner? Did readers understand it?

Is it presented in attractive manner? This is crucial, but we can go beyond this tight focus to look at broader questions.

Was the information used and if it was, who by, how, and why? Did using the information benefit users in tangible ways, and if yes, what were they? What action did they take as a result of using the information?

Casting a wider net enables us to relate our knowledge of how the information was used and what happened as a result to the needs of our own organisation, its other services and policy goals.

Evaluation also has longer-term benefits; it enables us to identify best practice, and steadily improve the quality of our information. Each evaluation feeds into the next one, building up knowledge and understanding of how to do things well, what to avoid, and where we still need to learn.

If we don't evaluate, we aren’t able to learn and improve. Each time we produce new information, we will do it as though this was the first time. If we do it badly, we will continue to do it badly, making the same mistakes each time. Improvements might occur randomly or through a combination of good luck, skill and experience - but they may not be recognised and repeated.

Evaluation can be time consuming and expensive, and although it is clearly desirable to evaluate every aspect of every piece of information, it will rarely be possible. However, evaluations can be very effective without doing this. You will still learn a lot from evaluating one or two aspects of the information (methods used, language used, presentation techniques, design, etc), or the effects for a single group of target users. Each evaluation can build on the results of the last one, or what you know from the evaluations produced by other organisations with a similar audience. What you test can be decided by what you already know, so that you slowly plug the gaps in your knowledge.

Broadly speaking, the goal of your evaluation must be realistic. We suggest that in deciding what is feasible, you should look at what you already know and build on that. The goal is not some notion of complete knowledge, but of making progress towards a better understanding of how to do things well, in order to do it better next time.

Evaluation needs to be planned and built into the production process from the start. When evaluation is shaped at an early stage, it is possible to ensure that monitoring information is collected at the right stages, and that participants can agree to do this well in advance.

The results of evaluation shouldn't only come at the end of a project. Early evaluations can inform revisions and improvements to the project as you go along, giving the information the best chance of success.

‘How will you monitor and evaluate it?’ provides practical guidance on these issues.