What We Set Out to Learn: The Story Behind WomenStrong’s Outcome Evaluation

In the Fall of 2024, WomenStrong International partnered with Ignited Word to conduct an outcome evaluation focused on two primary objectives: 1) to co-develop and apply an organizational strengths framework to understand the key milestones and challenges partners have experienced during the time they have been WomenStrong grantees and (2) to explore whether and how WomenStrong’s support contributed to partners’ strength or resilience in those moments of change.

As the evaluation concluded earlier this month, Amy Gregowski and Mara Steinhaus from WomenStrong’s Knowledge and Learning team joined Malaika Cheney-Coker and Abbie Cohen, the evaluation leads at Ignited Word, for a conversation with WomenStrong Executive Director Chisina Kapungu. Together, they reflected on the evaluation’s approach and methodology and shared insights into how the learnings (coming in June) can inform more effective, collaborative grantmaking.

The following conversation has been edited for clarity and length.

Chisina: So happy to have all of you here! My first question is for the WomenStrong team. What were you most hoping to learn from this evaluation?

Amy: Like most evaluation and learning teams, we have worked with our colleagues to develop a theory of change–how we think change happens to get us to our ultimate goal or objective–which for WomenStrong is supporting stronger, more resilient partner organizations that can better respond to the needs of their communities in their areas of work. Ignited Word created the visual for our theory of change on our website. As part of that, they conducted a literature review, which was really helpful for us in understanding our ideas about how change happens while reflecting on the literature about the programs we are implementing and how they relate to our outcomes. 

So, one priority of the evaluation was to better understand the pathways, to reflect on our theory of change and how change may happen. And a big part of that is to get our partners’ perspectives. We’ve come up with the ideas in our theory of change based on the work we’ve done with them so far,  but it’s really important for us to hear what matters to the partners. What are the outcomes that matter to them when it comes to organizational resilience and health?

Chisina: Malaika or Abbie, what made you interested in conducting this outcome evaluation for WomenStrong International and how did you decide what methodologies to propose that best suited the evaluation objectives?

Malaika: We had already been working with WomenStrong to conduct a sort of re-imagination of the theory of change and also the literature review. The literature review research confirmed even more strongly that there are just so few grant makers relative to the need for supportive grantmaking, not only providing unrestricted funding but also walking hand-in-hand with their partners.

And then, it was clear in the RFP that the preference was for qualitative approaches, and that qualitative approaches were entirely appropriate for what WomenStrong was seeking. This is also an area that Ignited Word specializes in. Outcome harvesting, one of the methods we chose, is really a great method to cast a broad enough net to capture a range of outcomes, whether anticipated or unanticipated, and not assume that everything is just cut and dried and very linear. The other approach, critical moments, was a method that I’ve used before and loved because of its ability to draw partners into the reflection process around what has been pivotal to them. It makes the exercise much more participatory and not too extractive from the partners standpoint, which I know is also in keeping with WomenStrong’s approach and ours.

Chisina: How were grantee partners engaged in the evaluation, and why was this important? What was the added value of including partners within the evaluation?

Abbie: Our research design was really informed by the principles of liberatory, creative, and value-based evaluation. This means engaging partners in a meaningful way, not just as a checkbox. We also focused on offering easy and accessible ways to engage with the evaluation, recognizing that meaningful participation starts with lowering barriers to entry. So we used a data collection tool that gave flexibility in how people answered. Respondents could write, audio record, or video record their responses. Partners were encouraged to have their partners outside of WomenStrong use the data collection tool, and they can request their data so they can use it as a point of reflection and a point of learning. So the idea is that we’re not just taking the data for WomenStrong, but they can use it as a learning tool.

Malaika: I’ll just add that in keeping with how WomenStrong works, all of this is done in three languages. And so working in the language of preference for partners has certainly been part of the approach that we’ve taken.

Chisina: I liked the fact that the evaluation was very inclusive, as well as the part about ownership. Many times when we’re doing evaluation, we’re extracting information for our own purposes, but you’re actually using it in a way where the partners own their data and can use the data to actually disseminate and brand themselves or market themselves however they want.

Abbie: Yes, and I’m really looking forward to the sense-making conversation with grantee partners in early May. I’m excited to hear how they make meaning of what we are seeing and tell us if it reflects their reality. It will allow them to see the outcomes that are a result of partnering with WomenStrong and they can then use that information for their own needs.

Amy: This journey for me at WomenStrong has been learning and unlearning a lot over the years in terms of how I initially learned to do research. This evaluation continued that journey. Outcome harvesting has that openness that you mentioned, Malaika, and it aligns with our values of trust-based giving and sharing power. 

We want to make sure that the definition of what matters is not what we come up with at WomenStrong, but instead what really matters to partners. It’s not just about whether we’re going to have really interesting stories or data that WomenStrong can say “this is great.” It’s about partners getting shared value from it. 

Malaika: You don’t have to twist our arms to develop and implement creative research approaches and to do our best to share power within the research process. We don’t often get the opportunity to do what we love to do with such a willing and like-minded organization.

Chisina: I think that’s the exciting part of the way that you all have designed this evaluation; it is left open for other outcomes that we may not have even thought of. And so with that, without getting into the findings, how do you feel the evaluation is going so far? Are there any surprises or challenges you can share?

Abbie: I’m very excited about what’s been bubbling up organically. One thing that has been surprising to me is that there is significant evidence that WomenStrong’s engagement with partners leads to more funding for a variety of reasons. The unrestricted funding allows them to focus on internal processes. In addition to the financial flexibility, WomenStrong’s trust-based and non-hierarchical model is also widely recognized by partners as a defining strength of the relationship, and that’s something that they’re also trying to mimic in other donor engagements. 

Chisina: I think we’re in such a critical moment in history right now in terms of foreign aid and development, and reflecting on how we reimagine philanthropy. I think the results can be a way forward to thinking differently. How do you decolonize philanthropy? How do you shift power? We hope to take those findings to different spaces, to share the lessons in different formats, including at conferences that we’ll be attending.

Mara: In terms of thinking about where we envision applying these findings, as a learning organization, we are always taking whatever feedback we can get and engaging in learning activities internally as well. We’re going to be using some aspects of Emergent Learning to frame discussions internally with staff about what upcoming opportunities we have to revise our approaches and the support that we provide to partners based on what we’re seeing in these results. 

Chisina: Malaika and Abbie, how do you envision the findings of this evaluation influencing future programming or strategy?

Malaika: Chisina, it’s interesting you mentioned earlier the dramatic challenges of this moment. It’s dramatic across the board and there’s an even sharper edge to it for women’s rights organizations.

I feel like a key purpose of the findings and their application going forward has to be how we help organizations be as resilient as possible. I also hope that the findings give some granularity to help define programs according to the stage of the organization. There is quite a spread of different developmental stages and different priorities of the grantee partner organizations. 

Abbie: I think WomenStrong is on the right track in terms of sharing the findings with other like-minded donors, but also donors that might be a little farther out, but may be able to be brought in, in terms of the model that you’re using. I think the outcome evaluation is going to give you so much fodder for advocating for your approach.

Chisina: Thank you, Malaika, Abbie, Mara, and Amy, for your dedication and your commitment to living our values within the evaluation. We look forward to hearing the results and the sense-making workshop with our partners.

Interested in learning more about our outcome evaluation results? Sign up here, or send us an email: [email protected]