Authors: Rachel Norton, Kanmani Venkateswaran
Summary: To build resilience to climate hazards, practitioners are shifting to implementing systems change-oriented programmes that sustainably reduce hazard vulnerability at scale. Reporting on the impact of complex change processes, adequately capturing their depth and breadth, and leveraging the data to inform learning requires a different kind of reporting system. The Zurich Climate Resilience Alliance’s system addresses these needs.
Evolving needs for evolving programmes
Monitoring, Evaluation, Research, and Learning (MERL) systems are key tools for tracking progress towards programme objectives, informing programme strategy and adaptation, and ensuring accountability, transparency, and credibility. As climate resilience programmes have grown in scope, we have seen a corresponding shift from output-based to outcome-based reporting. This shift helps programmes better capture the breadth, depth, and significance of their progress and impact. It also supports applied research to build out the evidence base and facilitate shared learning between stakeholders, both critical for building resilience to climate hazards.
The Alliance’s complexity as a multi-partner and multi-country initiative required a MERL system capable of tracking credible and evidenced impact, while also capturing the learning, nuance, and stories behind that impact. Alliance partners and teams effect systems change across scales and through research, influence, community programming, and learning. To accommodate this diversity of work, and the knowledge needs of the Alliance and its funder, ISET-International built a monitoring system that combines leading-edge design, capacity development, and facilitation.
The Alliance MERL system
The Alliance’s MERL system is innovative, complexity-aware, and purpose-built. It accounts for diverse change trajectories under common objectives, outcome areas, and programme design principles. The system includes a workbook for reporting people impacted, a progress reporting component where teams can report activities and efforts that have not yet led to outcomes, and a template for collecting Outcome Harvesting data.
It is innovative in how it captures and integrates both qualitative (narrative progress and outcome reporting) and quantitative (people impacted numbers) data. While the data input system relies on well-known tools – Microsoft Word, Microsoft Excel – the methodologies behind data collection and analysis are innovative. In particular, Outcome Harvesting is a useful approach for capturing qualitative data in a systematic and structured way while also allowing for the development of impact stories (see our impact story on provincial government adoption of technologies and tools to improve flood risk management in Vietnam for an example). These stories help validate and provide context for the reported impact numbers. The system also enables the visualization of trends in support of accountability and learning.
It is complexity-aware in how it supports tracking progress towards system change and impact numbers over time. Systems change takes time, so while impact numbers may not be reported every year, progress can be. Because the Alliance operates under a common framework which underlies the MERL system, the MERL team can use the data to develop a global narrative of how the Alliance effects systems change. Furthermore, the MERL team can stitch together reporting on incremental change overtime to tell stories of complex, long-term change that also acknowledge the contributions of multiple actors. For example, this year’s impact story on Concern Bangladesh’s work draws on several years of reporting on practice, policy, and spending changes that contributed to better flood outcomes in their programme communities.
It is purpose-built in that it satisfies both donor reporting needs and internal reflection at several levels. Beyond donor requirements, the Alliance governance team and country teams alike are using the reporting process and outputs to inform their strategy, programme adaptation, impact communications, profile-raising, and applied research.
What goes in to making this type of system work
It has taken many years of iterating, learning, and training to build the Alliance MERL system into the bespoke, effective system that exists now. While the Alliance’s unique enabling environment—long-term funding and the ability to build a custom-system—provided a stable foundation for innovation and iteration, there are aspects that can be more easily replicated by organizations and/or programmes:
- Providing training and guidance: To arrive at its current iteration, the MERL team has worked extensively with country teams, providing one-on-one support, developing guidance, and facilitating webinars. This has supported teams to provide increasingly high-quality reporting into the system that is evidenced, outcome-based, and supports important storytelling.
- Learning: The Alliance is committed to constantly learning and reflecting to ensure that we stay at the cutting edge of impact measurement in the development sector, and to ensure that the system is responsive to user challenges (e.g. difficult to answer questions) and needs (e.g. drawing out new types of knowledge from the reporting).
- Shifting reporting cultures and norms: Ongoing training and iterative learning have supported the transition from output- to outcome-based reporting to satisfy both funder needs and Alliance needs at multiple levels, while also accounting for organizational reporting capacities and internal reporting systems.
Expanding what MERL systems can and should do
Going beyond a pure accountability function, the Alliance is leveraging our MERL system to support peer learning on what works and doesn’t work , build credibility (i.e. back-up our numbers with validated stories), and drive planning and adaptation. This expansion of what MERL systems can and should do is occurring at an opportune time within the sector – as we seek to accelerate resilience-building efforts, MERL systems that enable learning from complexity will be crucial for understanding what works and doesn’t work.
For a glimpse into the impact and progress towards systems change captured by our MERL system, check out our latest impact report.