Editorial comment: Thanks to ECDG Advisory Group member, Alexey Kuzmin, for sharing his thoughts on the issue of defining an ‘adequate level’ of evaluation capacity in this week’s blog.
What is an ‘adequate level’ of evaluation capacity in an organization?
In 2010, BTW – a US-based consulting group – published the Evaluation Capacity Diagnostic Tool for organizations (link: http://btw.informingchange.com/uploads/2010/06/BTW-Evaluation-Capacity-Diagnostic-Tool_2010.pdf). This tool is based on BTW’s concept of evaluation capacity emphasizing the proper organizational context and evaluation experience of staff.
Organizational context includes:
- organizational culture and practice around evaluation;
- organizational commitment and support for evaluation;
- and using data to inform ongoing work.
Evaluation experience of staff includes:
- existing evaluation knowledge and experience;
- developing a conceptual model for the policy process / designing evaluation;
- and defining benchmarks and indicators.
The tool allows one to calculate an Evaluation Capacity score. If the score is high, then you have an ‘adequate level’ of evaluation capacity in place. The notion of the adequate capacity level is what I would like to discuss briefly.
According to my experience, different organizations need different levels of evaluation capacity. Evaluation capacity of an NGO working with disabled people and evaluation capacity of a professional consulting group specialized in evaluation should differ simply because for an NGO, this is one of the useful skills (not the key one) and for a consulting group, this is one of the key assets. Hence, an adequate level of evaluation capacity will be different for different organizations. An NGO should not be as advanced in evaluation as the consulting group. Developing evaluation capacity beyond what an organization actually needs will be a mistake. I believe that an ‘adequate level’ of evaluation capacity is something toward which an organization should aim. Adequacy should be measured with consideration of the organization’s specialization.
Alexey Kuzmin
ECDG Advisory Group
