Participatory story collection workshops are a powerful way to collect outcomes and learn more about a change process from diverse groups' perspectives. Stories have the power to capture the nuance of individual and collective experiences and context. They also can help to move away from top-down or donor-centric evaluation frameworks. If managed correctly, these workshops can help participants to share, listen, and learn. This honors their voices and ways of defining success, builds ownership and supports an empowering self-reflection process. If workshops embed program learning and improvement activities, they can also help to disseminate evaluation learnings on the spot.  This method can be used by foundations to track grantees outcomes, non-profit organizations to measure the success of their interventions, by businesses to understand consumer behavior, and impact investors to track company portfolio impacts. 

Read on for 3 practical tips to holding a participatory workshop to collect impact stories and create an outcomes database, which you can then analyze as a dataset. If you are interested in more tips for collecting and leveraging stories in your evaluation, sign up to our newsletter.

1. Create a safe space
   

Stories are often filled with personal or sensitive information. Wouldn't it be odd if a stranger walked up to you and asked you to disclose this type of information?  In this sense, creating a safe space for participants is the most important first step. It is essential for people to feel comfortable, engaged, respected, and open to share. 

Building rapport with the group is the first step. Start with an ice-breaker, something that helps the group get to know each other a little better, will be a great start. In addition to icebreakers, we often start off workshop sessions with the participants defining the rules for our engagement together, e.g., to co-define what this safe space looks like and describe the types of interactions that we would like to have together.  

To support open sharing, it is important to note that there are no right or wrong stories and that this is an opportunity to learn from everyone’s personal experiences. We want to create an environment where participants feel encouraged and safe to show vulnerability and share things that didn’t quite work too.

In a research environment, we always let people know how the story data collection will be used. It is important to document and let them know that before any information is shared externally that they will be contacted and asked if they would like this story to be shared, if there is any sensitive information to delete, if they want their name or organizational affiliation to be public or not. This is a critical part of informed consent and creating a safe space to share.  

Typically when running a workshop, we like to begin by pairing people up with only one other person to share and the other person documents that story and asks follow up questions. This creates a more intimate space for sharing. This then becomes the primary data source, and we then move into larger groups to do additional analysis work from there.  

‍

2. Choose the right questions
   

Choosing the right questions is important to collecting meaningful impact stories narratives for the purposes of impact assessment. A few tips to remember on strong question creation appear below. 

• Be mindful of the questions that are asked that frame storytelling exercises. Ensure that questions are specific to the role of the intervention being evaluated or the contribution of different groups to a change process.
• When creating an evaluation question, it is important that it is: (i) time-bound (ii) specific (iii) open-ended (does not close off answers by soliciting yes/no responses) (iv)  linked to the program focus area or outcomes and your organization’s theory of change (otherwise it will be difficult to make any claims about your organization’s contribution to change).
• Include follow up questions to get deeper information on what they believe led to the change happening, in terms of factors and actors. This helps to start surface change pathways and factors that make a difference to change actually happening.

To guide this process, you can draw from some monitoring and evaluation methodologies and data collection techniques that are appropriate for changes in complex settings, where a variety of actors come together to contribute to a change process. These sorts of methodologies are especially helpful in longer term attitude and norm change work, such as human rights, women's rights, economic justice, environmental and indigenous rights, etc. 

Some common story collection methods that we often adapt are Outcome Harvesting and Most Significant Change (MSC). As opposed to working with predefined outcomes,  Outcome Harvesting methods collect evidence of what has changed and, then, working backwards, determines whether and how an intervention contributed to these changes.  In MSC,  narratives (or stories) are collected in a participatory way framing questions to identify the most important or significant change. A sample question for a donor using the MSC approach could be: 

Looking back over the grant period, from your point of view, is there a story that exemplifies the most significant change that has resulted from your work? Please remember that change also includes maintaining past gains when women’s, girls’ and trans people’s rights are under attack. Let us know why you chose this story over all possible stories.

3. Use participatory methods for coding stories
   

Once you have built a story or outcomes dataset, you can now code the data uncovering outcome trends. There are many ways to do this, such as drawing from your Theory of Change, impact KPIs or the sustainable development goals (SDGs). Another option is to have the participants code the stories themselves. 

Collaborative qualitative coding of stories helps to make evaluations and its results more democratic. It is a powerful tool for evaluations to put the voice of participants at the center, asking them to classify and value aspects of the program or context based on stories shared.

You can engage in this participatory coding technique in a workshop setting. A strong facilitator or trainer with evaluation and research knowledge, especially qualitative coding skills, helps participants frame experiences. In this process, participants assign meaning and outcome trends to their own stories and experiences and have agency in summarizing the most important dimensions of the change process.

Codes defined are relevant to participants’ actual experiences and meaning making of the situation, ensuring coding content validity (or how well the code represents what it is supposed to be measuring). As a result, outside parties do not make assumptions about what changes are happening.

Tip: Here is a low-tech and inexpensive option for conducting a coding exercise in a participatory way with beneficiaries. All you need are some flipcharts, sticky notes, (or some other small cards  and tape) and an engaged audience. In this exercise, everyone can share a story of  most significant change around a subject, and then reflect on the most significant ones to be included in the final evaluation report.

If you prefer to transfer these participatory codes to a software tool, or code your data in alignment with your existing TOC and indicator framework, you can use ImpactMapper software to do so.  You can even embed your indicators into our survey tool and have participants tag their open ended text responses for you, engaging a participatory coding process while collecting your survey data!

We hope these tips are useful - if you want to hear more about how you actually use stories as datasets for impact measurement and why this really matters, get in touch! Impact Mapper organizes participatory workshops that show your organization how to use report and storytelling data more effectively and has software and reporting tools to support you in your impact journey.

Photo credit: ImpactMapper storytelling workshop hosted by Oak Foundation and Trag Foundation in Sarajevo.