Stories can be used to track complex processes of social and environmental change, which often slip through the cracks of traditional indicators and metrics. 

In February, ImpactMapper Founder, Alexandra Pittman, PhD, led a webinar sharing a diverse set of examples on leveraging storytelling and impact stories as datasets to understand social impact., drawing from experiences in Azerbaijan, the Balkans, to the US. We have compiled some common questions from the webinar for you here. 

If you want to receive other tips and insights for collecting and leveraging stories in your evaluation, sign up to our newsletter!

1. How can organizations embed stories in their monitoring and evaluation to track changes over time, using a pre-post analysis approach?
   

There are many unique ways in which a storytelling approach can be embedded in evaluations using pre-post methods, with the intention of assessing changes triggered by a particular intervention.

If we are interested in tracking changes within complex concepts like leadership, resilience or empowerment - stories can be first leveraged to understand how these concepts are framed by a person or a community before taking part in an intervention. After the intervention has been implemented, you can ask the person to share a similar story, describing that concept and track if and how that concept has changed over time. Were the goals of the intervention achieved or not? You can also pair this with other interview questions or survey follow-up techniques to ask more specifically if the person felt they had a shift based on the program, and if so, how; to gain further validation. To share a few analysis possibilities, you can then code both sets of stories for the main themes and give stories a binary code if conceptualizations changed before and after the program or not. 

Framing key concepts that a program is aiming to shift through the lens of participants helps us to understand what change actually looks like in a particular context and can help track these changes over time. Indeed, aggregating stories and comparing shifts in narrative trends collected before and after an intervention is implemented can be a powerful way to unearth social impact.

2. How can organizations start a story collection process and embed them within their M&E frameworks?
   

Having a guiding set of questions that aligns with your pre-existing metrics and the role of the organization in a change process or the group of stakeholders that are coming together is important. 

Stories can be collected through interviews, annual touch base conversations, site visits, data collection workshops, focus groups, video blogs, surveys, etc.  The important thing to remember when collecting these stories - no matter what the data collection method - is creating a safe space where participants feel comfortable opening up and sharing. This may mean that it is better to have someone external gathering the data or a member of their own community. To read more about collecting stories through participatory workshops, read our last blog post here. 

3. How can stories be aggregated as data points to track trends without losing the nuance?
   

When looking at stories as data points, it is important to remember that often there are multiple changes described within one story. Likewise, when we are thinking about change interventions, in general, there are different types and levels of shifts within theories of change too. 

For instance, the starting point of a longer-term outcome, like enhanced protection for women surviving gender based violence (GBV), could be a change in awareness of what constitutes violence or awareness of services offered in a community. After such a shift in awareness, someone could be moved to share this information with others, or to take collective action and join a group that's working to enhance referral mechanisms for women surviving GBV in their community or support local safe houses. Then it could be that this group does something collectively to strive for a larger collective change, like pushing for policies and legislations protecting and supporting GBV survivors at the national level. 

Stories can clearly reflect this change trajectory, echoing the pathway and how changes happened. This complexity can be captured when coding stories too, using multiple level or hierarchical coding/tagging to identify and track the frequency of changes and high-level trends. This allows us to aggregate common codes across a set of stories and grantees to observe how frequently a particular type of outcome or trend has taken place. Complementing this process, you can then pull out exemplary stories to depict nuance and complexity of the aggregate trends that emerged from the trend analysis. Moving between both aggregation and nuance is  key and is what we do in our work with our partners at ImpactMapper. 

We hope these insights are useful - if you want to hear more about how you actually use stories as datasets for impact measurement and why this really matters, get in touch! ImpactMapper organizes participatory workshops that show your organization how to use report and storytelling data more effectively and can help you to analyse data through our software tool or Impact Reports.