Clear articulation of evaluator competencies is essential for evaluation capacity development. In order to effectively develop evaluation capacity we need to be aware of what evaluators should know and be able to do. Numerous publications on evaluator competences in recent years confirm the importance of the subject. Probably, the most frequently cited are the articles by Jean King et al. (USA) and the Canadian Evaluation Society materials related to certification of evaluators. In these as well as in many other publications on evaluator competencies, evaluation is understood as an empirical study conducted to inform the intended users.

In my view, it is very important to consider that in addition to an empirical approach, there are two other approaches to evaluation that require specific competences: evaluation based on expertise and evaluation based on indicators. In fact, any of the following three distinct but complementary approaches could be used at any stage of the program life cycle:
1) Expert Evaluation or evaluation based on expertise (knowledge that subject experts already have).
2) Indicator-Based Evaluation or evaluation based on indicators (which are measured in the course of evaluation).
3) Empirical Evaluation or evaluation based on empirical study.

Three examples will help clarify this typology. A project proposal assessment conducted by a group of pre-selected subject experts is an example of Expert Evaluation. Monitoring is an example of Indicator Based Evaluation. The systematic collection of data about a program using social science methods to identify a program’s impact is an example of Empirical Evaluation.

Expert evaluation is based mainly on pre-existing knowledge and limited information about the program. Indicator-based evaluation combines pre-existing knowledge (we develop indicators based on our understanding of how things work) and some information reflecting the reality of the program (actual values of indicators). Empirical evaluation is mainly based on information collected in the course of an evaluation (studying the program as it is) and depends less on pre-existing knowledge.

As evaluation consultants, we are often invited to participate in or organize proposal assessment processes; are heavily involved with design, implementation, and maintenance of monitoring systems; regularly conduct traditional empirical evaluations.

That is why I believe that evaluator competencies (at least for our region – Russia and Newly Independent States) should include:
1. Design and implementation of expert evaluation of project and program proposals.
2. Design, implementation and maintenance of project and program monitoring systems.
3. Design and implementation of empirical evaluations of projects and programs.

Alexey Kuzmin
ECDG Advisory Group