Photo Credit: Animating Democracy

Not only can storytelling be evaluated, within limits, but it can also be a method of evaluation, providing a picture of the change you make. Here’s how to make it useful.

Identify your goals.

Before you can evaluate whether your storytelling met its goals, you have to be clear about what those goals are. Start by considering whether you’re trying to change public knowledge, attitudes, behavior, or policy. Animating Democracy, a project of Americans for the Arts, has a page on social-impact indicators to assess changes in such areas as people’s knowledge, attitudes, and actions, and in policies. (It’s part of a broader set of resources on the group’s website.)

“Not everything that can be counted counts, and not everything that counts can be counted.”—William Bruce Cameron

Go too narrow with what you’re evaluating, and you risk showing only what’s self-evident; go too broad, and you risk trying to prove something that it’s impossible to document. The sweet spot is where social impact overlaps with your desire and ability to evaluate.

Determine your metrics and your methods.

Once you’ve determined your goals, ask yourself exactly what it will look like if you succeed. More media coverage? More small donors contributing? New volunteers walking through the door? Peg your goals to a time period and specific numbers, and these become your metrics. If your storytelling is part of a web-based fundraising campaign, the metrics might be “10 new donors per fundraising email” and “$500 raised per fundraising email.” Depending on what you’re looking to evaluate, the mechanisms of evaluation may include surveys, focus groups, comments on social media, voting patterns, web-traffic data, fundraising data, or reviews of policy changes.

Measure for learning.

Evaluation can be used to compare one storytelling method to another, and to generate lessons for your group, your funders, and the field. You might pose a strategy question up front that you’ll address over the course of a project: What kinds of stories raise the most money in fundraising appeals? What media is most effective in communicating with target audiences?

To answer such questions, you’ll have to check the indicators on your work against whatever you’re comparing it to. For example, if you’re checking how much traffic a web video drives to your site, you’ll have to measure traffic before and after the time you post the video. Or you might use a control group to determine if some stories work better than others; for example, you might send out two fundraising emails to different but roughly equal email lists, one with a story from one of your clients, and one with a story from a board member. (This is similar to how the 2012 Obama campaign tested the response rate of various email subject lines. One of the most successful was simply “Hey.”) Once you have the first round of results, refine your strategy and try again.

Don’t evaluate if you won’t learn from it.

Doing a project assessment can be useful in the same way that writing a grant proposal is: It forces you to articulate what you want to accomplish and why. If you have an idea of what you want to learn and why, evaluation can help. But it should be paired with a strategy for making a difference; otherwise, it’s a little like telling only the end to a story: You say where you end up but not the path you took to get there. Or if you’re not under pressure to evaluate, or won’t learn anything from it, don’t bother. That’s the gist of a 2014 article in the Stanford Social Innovation Review.

Gather stories to assess needs and strengths and document impact.

“Rebecca’s life has come to an end with doctor’s announcement that she was pregnant…” That’s the first line of one of over 57,000 “micro-narratives” collected in Kenya and Uganda by GlobalGiving, a website that allows users to donate to vetted development projects around the world. The organization’s Storytelling Project hires and trains “scribes” to gather stories in the areas where beneficiary groups are located. Those stories get fed through a host of tools GlobalGiving created to filter data about community needs, possible solutions, and innovative organizations that it might add to its web platform. The data also supports other groups in international development, especially the small ones that can’t afford such a project, so one funder tells the Stanford Social Innovation Review. And the project’s principle applies to groups of any size: Gathering stories can help a group assess the needs and strengths of the community in which it works and document its impact.

Further exploration:

• Chapter from Hatch on evaluation has tips about how to create good key progress indicators, or KPIs.
• How Do We Know is a project of Active Voice Lab and has resources on evaluating how media contributes to social change.
• Animating Democracy, a project of Americans for the Arts, has a page on social-impact indicators, and a broader set of a resources.
• Narrative Arts has a blog post on using the social-impact indicators of Animating Democracy to evaluate storytelling work.

How do we know?

Photo Credit: Active Voice Lab

For many storytellers, there is no more loathsome question than “How are you going to measure your impact?” It clashes with the creative impulse behind storytelling.

The folks at Active Voice Lab—a San Francisco-based group that helps people use stories and culture to advance social change—have artfully reframed this question to get at something more essential: “How do we know if we’re making a difference?”

That’s the topic—and the name—of a new website they’ve launched on media impact measurement. It includes a blog, a resource page, a set of “horticulture” garden tools that serve as metaphors to explore media impact, and other features.

“I can’t think of a social-justice movement that doesn’t need stories, just as they need organizers, money, research, leadership,” says Active Voice Lab director Ellen Schneider. “But stories alone are just characters, settings, plot, and so on. They’re powerful as part of an ‘ecosystem of change’”—a big picture that includes all the other elements in a social-change movement.

The work of social change can feel abstract, especially if you sit at a desk much of the day, or if you’re not sure who sees your stories or if they care. You may wonder if your dedication to your cause is worth anything. You may despair of the possibilities for change, when the forces of apathy and injustice seem so entrenched. It’s in those moments especially when the question of “How Do We Know?” matters most—for it is a prompt to stay connected to the work itself and to the other people doing it.