Now that you have a map, filled with landmarks, you have to figure out how you know you’re making progress towards your destination. What kind of data can you collect and how do you make sense of it? (These are the feedback loop spaces on the map.) In public engagement work, the landmarks are often nebulous. You’re looking for changes in attitudes, increases in the amount and kind of attention paid to a topic, participation in events, specific online actions (retweets, likes, etc.), the quality of stories told, and the list goes on. So how do you identify what’s important?
Even though you set out with good intentions to measure what’s important, often reality gets inverted, and you start to think only that which is measurable is important. It’s good to recognize this bias early on in order to correct for it. Once you’ve done that, you can figure out how to make sense of it all.
Start with your destination and work backwards to all the places you’ll have to pass through before you get there. For example, in order to accomplish a change in policy, you have to generate greater attention to the topic that can lead to an increase in social media activity about the topic, high-level influencers talking about the topic, and perhaps an increase in people attending related events. Once you have these things identified (in no particular order), then start to think about metrics. Are there things you can count? Do you have the tools you need to count them? If not, can you get them?
If there is nothing to count, is there something to describe? Have the quality or style of images people are using in online conversations changed? How do you know? Are more people showing up to meetings? Are meeting organizers describing their process more clearly? Are communities using the resources you’ve made available to them?
In addition to the things you can count and describe, can you create opportunities for more feedback? That is, if one of your landmarks is a change in people’s attitude about a topic, can you ask people questions? Can you circulate a survey online? Can you distribute a paper survey? If so, what kinds of questions will you ask to gain insight on people’s attitudes?
The key is to define the landmark and then figure out how you will describe it.
Things to Keep in Mind
- Metrics matter. With each landmark on your map, determine how you’re going to measure progress towards reaching it. Does the data exist? Will you have to make it?
- There are many ways to see the landscape. Consider all mechanisms for collecting data, including online and paper surveys, interviews, or online metrics (i.e. Google analytics).
- Don’t wait until the end to analyze your data. You should be narrativizing and evaluating your data throughout a process.