Get Started with Impact Evaluation Quickly

The multiple social and ecological crises in 2020 so far have fuelled a lot of new interest in impact measurement. More than ever before people want to know how, and whether, their work results in desirable social, ecological, and economic results. 

I am a Champion for both Demonstrating Value and the Common Approach to Impact Measurement, because each distils dozens of different (and sometimes conflicting) approaches and guides to impact measurement for social impact organizations into clear guiding principles and steps to follow. 

The Common Approach to Impact Measurement has just published a new website with lots of useful information including the five essential practices—the “how” of impact measurement.’ The Demonstrating Value Getting Started Toolkit provides invaluable step-by-step guidance. But even these ‘essentials’ guides can present a steep learning curve if you’re new to the concepts and practices.

Get Started in Just Four Hours 

In this post I will describe how you can “quick start” an impact evaluation program for your own small business or nonprofit organization. Block out just one afternoon in your calendar. You can have the foundations in place for measuring and managing your impact and showing it in a snappy one-pager. Four hours is all it takes to get started, and here’s how to do it. Hypothetically starting at 1 pm, right after lunch, so you can be finished by the end of the day. 

1:00pm to 1:30pm – Start with the end in mind

Start by looking at some examples of impact snapshots like Comox Valley Farmers Market or Squamish Rebuild—these are examples of what can be created fairly quickly using the guidance in the Demonstrating Value workbook and tools. These examples were created with simple tools like Excel and PowerPoint, but you can use whatever tools you prefer.

1:30pm to 2:10pm – What do you want to ‘know and show?’

Write down three to five statements about your organization’s work that you would like to be able to share, with confidence, as part of your ‘impact story’—the story about the difference you make through your work. Here are some common types of statements you can use as a starting point, and adapt to your particular case:

• The problem/issue that we address with our products/services is significant and meaningful.
• We have an innovative and scalable business/service model based in a deep understanding of the issue.
• Our products/services are valuable and appealing to the audiences/stakeholders that we serve.
• Our products/services make an impact on the problem that we aim to address. (This one may be related to your organization’s vision statement. If you have one, it should answer the question, “How is the world made better by our work?”)
• We are an efficient, effective organization with a coherent management approach.

It helps to start your measurement effort with a clear idea of the story you’re trying to tell, so get it down in some crisp short statements.

2:10pm to 2:30pm – Who is your audience?

Who cares about your impacts? Or put another way, with which audience do you want to share your impact story? Choose one audience to focus on as you work through this process. It’s possible to do more, but trying to do several at once quickly becomes complicated.

Very often the priority audience is the people or organizations you’re asking for money, whether that’s grant-makers, government, investors, or individual donors. But it could also be the beneficiaries of the impact you strive for, or local community members, or policy makers, or any number of others. 

2:30pm to 3:00pm – What’s the structure of your impact story?

It’s time to think about how you will present your key points (what you want to know and show) to your audience of interest. Linked below you’ll find some more snapshot examples that were developed by Bryn Sadownik at Demonstrating Value using PiktoChart, a free and easy to use design tool. Each different template lends itself to a different ‘story structure’ that can guide your narrative. Some example structures include:

• Issue, Solution, Results
• Past, Present, Future
• Steps in a client/customer journey
• A dashboard giving the impression of ‘live’, up-to-the-minute results and information

There is no limit to the number of possible structures or visual metaphors you can use to tell your story.

Now grab a blank piece of paper and a pen or pencil and sketch out your own snapshot with no detailed content. Just include placeholders for content—i.e. the three to five key things you want to know and show, and a space for some detail with each one.

• Which key point is most important i.e. in the ‘headline’ position?
• Is there a key point that might fit best with a chart, or numerical data to support it? 
• Which point might best be supported by a photograph, or an anecdote, or a testimonial from someone (and who)?

Use your imagination! Play with some different arrangements, and see what emerges. There’s no wrong way to do this part, so be as creative as you want—but do consider whether your choices are ‘on brand’ for the organization. When you’ve landed on something you’re happy with, take a look over it and reconfirm for yourself that it covers the key points of the story in the right order. Is it relevant for the interests and information needs of your selected audience?

3:00pm to 4:00pm – Create an Information Map

At this point you can think in more detail about the specific evidence you will actually use for each section in your impact story. For each thing you want to know or show, what evidence might support your story? On your first pass through this question, don’t limit yourself only to information you already have. Consider what evidence (‘hard data’, stories, anecdotes, etc.) would be most compelling if you had no constraints on collecting data. Aim for a possible two or three points of evidence for each section, and remember to consider both quantitative (numbers) and qualitative (stories, descriptions) data. This gives you a ‘long list’ of potential information you could include.

Then think through what is actually reasonable to collect by considering, for each point of evidence:

• How valuable is this information to your audience? Is it ‘nice to know,’ or critical for them to make a decision (e.g. to support you or not)? (You could give each information point a value rating on a scale of one to five, for example.)
• What might be the costs be of acquiring and analysing this information? Cost drivers include (but aren’t limited to):
  • Do you have a system to manage this information, or do you have to create one?
  • Do you have to collect it yourself, or is it available from a third party?
  • How frequently do you need to collect and update this information?
  • How much analysis is required before this information is ‘presentation ready?’
  • Are there legal/ethical issues attached to collecting or using this information?

4:00pm to 4:45 pm – What is worth doing? What are the alternatives?

Now you can exercise your judgment and trim down your long list to a short list: decide where high-value information is worth the cost of collection, or not. Trim items from the list that have a high cost to collect, but lower value to your audience. In about an hour of focused work you should be able to create a fairly complete ‘short list’ of data collection and analysis tasks that will help you support your impact story with evidence.

The goal is to pare down the list to a set of information that is within your budget and capacity to collect and use on an ongoing basis without underinvesting and missing out on valuable evidence that you need to show your impact.

Some ‘real talk’ here: If you cut down the list to only include things that have little or no cost or effort attached, it’s unlikely you’ll have a useful result. Conversely it’s also a perpetual challenge, especially for nonprofits, that funders expect a lot of evidence-based rigour in programs, but don’t adequately fund evaluation work, or make good use of the information they’ve requested. If possible, use this exercise to advocate for appropriate funding to do the work properly. Don’t settle for a partial and underfunded approach to evaluation, or a time-and-resource wasting ‘fishing expedition’ that’s a burden to everyone involved.

4:45pm to 5:00pm – Who can help you check your assumptions?

Before you proceed into the work of data collection and preparing a more polished ‘snapshot’ with real data, save yourself some effort by validating your assumptions with two or three groups:

• Your intended audience: Make sure that they actually value the information you intend to present, and that you’re not missing some critical evidence. 
• The people who will provide and/or collect the data: ”You can’t ask people that,” “this would take too long,” “we’d need a new database to track this….” You can imagine the conversations, right? It’s important to check explicitly that it’s legal, ethical, and technically feasible to collect the information you want before you ask people to do it for you. 
• The beneficiaries of your work, if applicable: If you claim that you’re making an impact, but it doesn’t wash with the people you’re claiming to help, then you’ve got trouble coming sooner or later. 

It doesn’t take long to write down some names of who you’d like to ask for help with validating how you’ll tell your story and support it. In each case, you will need to find someone (or several people, if you can) and show them your draft snapshot. You can say: “Here’s how we plan to tell the story about our impact, and it looks like this, and it’s supported by this evidence. Is it sufficient for you to decide (to support us, fund us, partner with us, etc. as appropriate)?” It’s much better to know you’re putting in the effort to collect the right data before you go and collect it!

If you get through this list of tasks, you’ve got what it takes to start measuring and managing the impact you make with confidence, and you can still get to dinner on time! Of course, once you’ve got the high-level plan, there’s still work to be done! You need to implement a system, design a polished version of your snapshot, do user-testing with surveys, set up data integrations, and align all of your evaluation work with your existing brand and strategy…. Please turn to us for support if you need it. Junxion has helped dozens of organizations get on sound footing for their impact strategy and reporting. We’d like to help you too.

After all, showing your impact is but one step in solving for the big crises we face!