ECDG Toolkit

To Rent or to Own

The first tool addresses the question of whether a nonprofit should hire an external evaluator or develop internal evaluation capacity. We used the analogy of renting an apartment or buying a house. When moving to a new area, if a person plans to stay for a short-time, it makes sense to rent. If one is going to be there for a number of years, that individual might want to invest in a house and build some equity. The same is true for evaluation. If a nonprofit has one-time funding for a special project; hire an external evaluator. However, if it is going to be conducting evaluation for multiple projects over an extended period of time, it may want to invest in developing some evaluation capacity.

Includes

Checklist for determining whether to conduct internal or external evaluation

return to top

Purpose and Shared Vision

If a nonprofit decides it wants to develop its own evaluation capacity, it needs to decide on the purpose of evaluation. Three commonly accepted purposes are:

To provide feedback for program improvement
To demonstrate accountability to funders
To promote organizational learning

These are distinct but not mutually exclusive.

Once a nonprofit determines the purpose, it should create a shared vision for evaluation. This imagines the potential or desired future for the role of evaluation. This process should involve the whole organization, not just the Executive Director and the Board. It should also be aligned with the organizational vision.

Includes

List of visioning do's and don'ts
Step by step instructions for visioning

return to top

Organizational Design

Once the evaluation purpose has been determined and a shared vision for evaluation has been created, the nonprofit needs to put the organizational design in place to achieve the vision. In Tool 3, we recommend that nonprofits pay particular attention to four elements of organizational design:

Division of Labor — who will do the evaluation
Authority — relationships among coworkers
Departmentalization — separate evaluation unit or integrated into program areas
Span of Control — reporting relationships

Includes

Three column T-chart for determining organizational design

return to top

Organizational Culture

Tool 4 deals with creating a culture of evaluation. Organizational culture is like the collective personality of the nonprofit. It can be viewed from three dimensions.

Observable Artifacts — reports, records, stories, myths, symbols
Values — beliefs about whether evaluation is a good thing or a bad thing
Assumptions — beliefs about evaluation that we think to be true without questioning them

Includes

* Organizational culture rating scale

return to top

Evaluation Policies

In Tool 5, we assert that organizational design and culture are put in place by the policies that the nonprofit implements. ECDG provides tools for nonprofits to do informal policy analysis in order to determine if evaluation policies that are implemented will have the desired results.

Includes

Procedure for doing informal policy analysis
Brainstorming activity
Multi-voting form
Force field analysis form
Action plan form

return to top

Budgets

Tool 6 deals with budgeting for evaluation. It is important to realize that the policies to which we referred in the previous tool are operationalized through budgets.

Have you ever heard, "Every dollar spent on evaluation is one dollar less for programming?". Not true! The increased service capability and reduced expenses that come from an effective evaluation system should more than pay for itself.

Includes

Budgeting rules of thumb
Sample evaluation budget

return to top

Processes

In Tool 7, the internal and external processes that nonprofits carry out are discussed. Internal processes maintain the organization and external processes help it adapt to its environment. Both types of processes use the same mechanism. They take inputs from the environment, convert or transform them, and return them to the environment as outputs. Evaluation is an important part of the process mechanism. It can tell the nonprofit whether the output is aligned with the organization's mission and goals. In other words, "Are we doing things right?". Evaluation can also tell the nonprofit whether its mission and goals are aligned with the needs of the external environment. In other words, "Are we doing the right things?".

Includes

Input-Process-Output form
Process flowchart

return to top

Approaches to Training

At this point, it would be appropriate for a nonprofit to start thinking about training. Training that is conducted before the necessary organizational architecture is in place is a waste of resources. Tool 8 helps nonprofits determine their evaluation training needs by using the following process:

Determine what skills and abilities should be in place and what information should be known by whom at what point in time
Ascertain the current situation
The gap between the identified needs and current realities will be filled by evaluation training

return to top

Action Research

If a nonprofit has followed ECDG's advice, it may have resulted in changes to the organizational design, organizational culture, policies, budgets, and training. ECDG would expect the nonprofit to take a critical look at these changes to see if they are working or not. One way that ECDG suggests doing this is through participatory action research. In Tool 9, we outline the process.

Includes

7 step action research process guide
Action research plan form

return to top

Standards for Internal Evaluation

In order to determine if the nonprofit is using its newfound evaluation capacity well, ECDG recommends applying a set of standards especially developed for internal or self-evaluation by the UK Evaluation Society.

return to top