What Matters Most...Evaluation as Learning

Poet Charles Bukowski is famous for a lot of things, but one of them is writing the line, "What matters most is how you walk through the fire."

I don't mean to be melodramatic, but I think many nonprofits approach evaluation like it is walking through fire. They avoid it unless they know the results will make their organization look like a shining beacon of best practices. This is unfortunate because it is not about making it through the fire (i.e. getting positive results from your evaluation), but HOW you make it through (i.e. what did you learn from it, what changes are made because of it).

Last year, as an organization, we made a commitment to share our learnings as we evaluated our work (here's the first post, and here's the second) - be they things that we were doing well or things that we could improve upon. In this spirit, the following is a summary of an evaluation we just concluded.

What We Learned

During the second half of 2010, we received funding to launch a pilot project to work with immigrant and refugee-led nonprofits. The goal was to prepare these organizations for a review and to meet all standards by providing hands-on technical assistance on a one-to-one basis.

We've all heard the old saying, a picture is worth a thousand words. Well, here is a thousand words (or there abouts) as a picture that summarizes the responses of the nonprofit leaders who participated in the project.

Project Participant Responses

Clearly, the most common responses of "helpful" and "information" jump out immediately, but also the words "support" and "services" stand out as common themes. These are music to my ears as they validate the work we were doing as beneficial.

But as I wrote earlier, it's easy (and tempting) to bask in the positive good news evaluation can bring and stop there. There is a great deal to be learned by looking at a similar word cloud of those nonprofit leaders that we met with initially, but who didn't choose to participate in the project for a variety of reasons.

Project Non-Participant Responses

Although there is some overlap between the two word clouds, tellingly there are two other words that stand out - "relationship" and "work."

Remember that this second image summarizes the responses of nonprofit leaders who chose not to engage with us in trying to meet all standards. Why? First and foremost, there was no established relationship between our organizations. With no relationship, there is little trust and little understanding of why go through this process.

There is a recognition of how much work this process is, even with help. Most of these nonprofit leaders are the sole employee of their organization and have to balance multiple demands on their time.

What We're Doing Now

We're using this information in a couple ways going forward. First, we recognize that developing a relationship is very important, but even more important is doing the work to stay in that relationship. There's a balance that we are learning between being available, but not hounding. Giving space, but not disappearing.

Second, we are streamlining the process as much as possible for nonprofits engaged in this project. An example of this has been to prioritize those "easy steps" first, set realistic timelines, and engaging board members to share the work so it doesn't solely rest on the shoulders of the nonprofit leader.

We'll continue sharing our evaluation results as we get them, but in the meantime it's important to remember and support one another in sharing the areas where improvement is needed.

After all, that is what matters most.