Measuring success

"Participation works best when people feel that they can make a difference, when they have the time to fully engage with the issues, and when there is a healthy relationship of mutual respect.

It works worst when it is rushed, ill-informed and vague about the links to formal decision-making or when it allows the loudest voices to dominate."

Geoff Mulgan, Involve Chair, United Kingdom.

A lot of effort can be wasted duplicating or partially duplicating information that is already known to others in the sector, because those asking for the information – or those being asked – either don't know where to find it, or do not have the time to locate the material or to assess its usefulness. They can, therefore, assume that something completely new needs to be undertaken.

This can lead to a lot of one-off, small-scale projects, with short deadlines that may be useful immediately to one group of people but the findings are never used again or shared with others.

The result can be repeated cycles of work that use up scarce resources (time, money) and don't build on previous learning.

To build knowledge in smarter ways:

• think about how to use administrative data (the information that organisations collect for reporting to funders and for keeping track of what they are doing)
• find out what research and evaluation and reports other agencies undertake
• use skilled librarians to help with key word-synonym development so published material can be sourced (not everything useful is published on google or in libraries).

Don't automatically discount information published more than five years ago. While policy or legal frameworks may alter, much human behaviour does not alter very quickly. Depending on the issue, work done well 20 or 30 years ago may still be very relevant.

If you decide research or evaluation is still needed, the next decision is to get advice from experienced people about resources likely to be needed and whether the project is actually feasible in research/evaluation terms. If you are still proceeding, then the project needs some scoping and decisions made about whether all or part of the project work can be done in-house, or externally.

Another good starting point is the Social Policy Evaluation and Research (SPEaR) website and Secretariat. SPEaR (through its Secretariat housed by the Ministry of Social Development) is a co-ordination hub for government agencies and the social research and evaluation sector to improve research capability, capacity and outcomes.

The team undertakes an annual mapping exercise of research and evaluation activity and can potentially identify previous work that could be useful to your project.

SPEaR is developing Good Practice Guidelines for social policy research and evaluation undertaken by government agencies or commissioned by these agencies.

So far, material has been developed to identify principles of good practice and the principles have been extended into four areas:

• Research and Evaluation Contracting
• Research and Evaluation Ethics
• Research and Evaluation involving Maori
• Research and Evaluation involving Pasifika

Two further areas have been identified for development: new Migrant/Refugees and Sensitive Issues (eg, trauma survivors).

Research and evaluation use depends on relevance, timing and credibility. Credibility depends on practice standards. The good practice programme is a process of pooling expertise acquired by individuals and agencies here and overseas to reflect the practical realities involved in social policy research and evaluation.