Theory of change approach in designing monitoring and evaluation frameworks for extension programs

Introduction

A theory of change, at its simplest, is a map of how an intervention is supposed to work [1] . It traces the causal steps from the resources an organization puts in to the long-term changes it hopes to produce, making explicit the assumptions that connect each step. In agricultural extension, where programs aim to shift farmer knowledge, behavior, and ultimately livelihoods, having a clear causal map should be a planning basic. Yet most extension M&E frameworks in southern Africa still rely on the conventional logical framework matrix, which lists inputs, outputs, outcomes, and impact in a grid without spelling out the causal reasoning that connects them [2] . The distinction matters for evaluation quality. A log-frame can tell you what was delivered and whether targets were met, but it often cannot explain why results did or didn't materialize [3] . Theory of change-based frameworks, by contrast, invite evaluators to test the assumptions underlying each causal link, turning evaluation from a scorecard exercise into a learning tool [4] . Donor agencies including DFID and USAID have increasingly required theory of change narratives in program proposals, yet the evidence on whether this requirement actually improves M&E framework quality at the field level remains thin [5] . Eswatini's extension system, managed jointly through government and university-based outreach, provides a useful testing ground. Some programs have adopted theory of change approaches following donor guidance, while others continue with traditional log-frames. This natural variation allowed a comparative assessment. The research goals were to compare framework quality between theory of change and log-frame programs across five dimensions, to identify which causal pathways in the extension results chain showed the greatest and weakest strength ratings, and to draw practical implications for M&E design in resourceconstrained extension settings.

Theoretical Background

The research was anchored in two frameworks. Program theory evaluation, as described by Chen, holds that understanding the mechanisms through which a program produces effects is essential for meaningful evaluation [6] . This was paired with the contribution analysis approach of Mayne, which uses theory of change logic to assess the extent to which an intervention has contributed to observed outcomes rather than merely correlating with them [7] . Together, these perspectives predict that programs designed around explicit causal models will produce M&E frameworks that are more specific in their indicators, more logical in their causal chains, and more aligned with stakeholder expectations, because the design process forces planners to articulate and test their assumptions before data collection begins.

Material and Methods

Material

This research was conducted through the Faculty of Agriculture at the University of Eswatini, Luyengo campus, between August 2023 and March 2024. The University of Eswatini Research Ethics Board approved the protocol on 17 July 2023 (reference UNESWA-REB/2023-042). A total of 41 extension programs operating under the university's outreach network were assessed: 22 that had used a theory of change approach during M&E framework design and 19 that relied on conventional log-frame methods. Programs were drawn from the Manzini and Hhohho regions, covering activities in crop production, livestock management, soil conservation, and market access. For each program, the complete M&E framework documents, annual reports, and evaluation summaries were obtained from the university's extension coordination office.

Instrument Development

Framework quality was assessed using a 25-item scoring rubric organized into five dimensions: theory of change clarity (6 items), indicator specificity (5 items), data collection feasibility (5 items), causal pathway logic (5 items), and stakeholder alignment (4 items). Each item was rated on a four-point scale from absent (0) to fully developed (3), and dimension scores were converted to percentages. The rubric was developed through an iterative process involving three M&E specialists and two extension program managers. Pilot testing on six programs outside the sample yielded an inter-rater reliability (Cohen's kappa) of 0.79 and a Cronbach's alpha of 0.84 for the composite index.

Methods

Two trained assessors independently scored each of the 41 M&E frameworks using the rubric. Where scores diverged by more than one point on any item, assessors discussed and reached consensus. Composite quality scores and dimension-level scores were compared between the two groups using Mann-Whitney U tests. Effect sizes were reported as rank-biserial correlations. In addition, the six causal pathways in the standard extension results chain (from inputs to income change) were rated for strength on a percentage scale by a panel of five extension evaluators who reviewed each framework's pathway documentation. Pathway strength ratings were compared descriptively and used to identify the weakest links in the causal logic of each group [8] .

Results

Table 1 presents the dimension-level quality scores for theory of change and log-frame programs. All five dimensions showed statistically significant differences in favor of theory of change programs. The largest gap appeared in causal pathway logic (+31.5 pp, effect size 0.71), suggesting that the most distinctive advantage of the theory of change approach lies in making the causal reasoning behind programs explicit and testable. The smallest gap was in stakeholder alignment (+15.9 pp), possibly because both approaches can involve stakeholders in M&E design even if they differ in how causal logic is articulated. The box plots show that theory of change programs not only scored higher on average but also displayed tighter distributions, indicating more consistent quality. Log-frame programs exhibited wider spread, with several scoring below 30% on causal pathway logic, suggesting that some log-frame-based frameworks lacked any explicit causal reasoning. Pathway strength declined steadily from the input-toactivity link (81.4%) to the yield-to-income link (39.2%). The three upstream pathways all exceeded the 60% acceptable threshold, while the three downstream pathways fell below it. This pattern indicates that even theory of change programs had difficulty articulating how farm-level adoption translates into measurable economic outcomes, a well-documented weak spot in extension evaluation [3] .

Discussion

The 22.6-point composite gap between the two groups aligns with arguments made by Vogel that theory of change processes improve program design by forcing planners to surface and test hidden assumptions [1] . The particularly large gap on causal pathway logic (31.5 pp) suggests that this is where the two approaches diverge most sharply: logframes list steps, while theories of change explain the reasoning behind them. This distinction carries practical weight because evaluations built on weak causal logic tend www.extensionjournal.com to produce findings that are descriptive rather than explanatory, making it hard for managers to know what to fix when outcomes fall short [4] . The declining pathway strength from upstream to downstream links deserves attention. Programs were generally strong at documenting how inputs led to activities and activities to participation, but much weaker at articulating the final steps from adoption to yield to income. This may reflect a measurement problem as much as a conceptual one: income change depends on market conditions, weather, and household decisions that sit outside program control [9] . Including explicit assumptions about these external factors in the theory of change could sharpen the evaluation design for downstream pathways without requiring programs to take responsibility for outcomes they cannot fully influence.

Practical Applications

Extension agencies designing new M&E frameworks should invest time in building a theory of change narrative before selecting indicators and data collection methods. The upstream-to-downstream quality gradient suggests that special attention should go to articulating the links between practice adoption, yield change, and income effects, where most frameworks were weakest. For agencies currently using log-frame-only approaches, a practical starting point would be to add a one-page causal pathway narrative to existing frameworks, specifying the key assumptions at each step and identifying which assumptions the evaluation will test.

Conclusion

This research found that extension programs in Eswatini that used a theory of change approach during M&E framework design produced substantially higher-quality frameworks than those relying on conventional log-frames. The advantage was largest in causal pathway logic and indicator specificity, two dimensions that directly affect how useful evaluation findings are for program improvement. Even among theory of change programs, the downstream links in the results chain, from adoption to income change, remained weak, pointing to a design gap that future guidance materials should address. The findings support making theory of change a standard requirement in extension M&E design rather than an optional add-on reserved for donor-funded programs.

References

1. Vogel I. Review of the use of theory of change in international development. UK Department for International Development. 2012:1-76.
2. Gasper D. Problems in the logical framework approach and challenges for project cycle management. Courier ACP-EU. 1999;173:75-77.
3. White H. Theory-based impact evaluation: Principles and practice. Journal of Development Effectiveness. 2009;1(3):271-284.
4. Weiss CH. Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives. Aspen Institute. 1995:65-92.
5. Stein D, Valters C. Understanding theory of change in international development. Justice and Security Research Programme. 2012:1-25.
6. Chen HT. Theory-driven evaluations. Sage Publications. 1990:1-328.
7. Mayne J. Contribution analysis: Coming of age? Evaluation. 2012;18(3):270-280.
8. Funnell SC, Rogers PJ. Purposeful program theory: Effective use of theories of change and logic models. John Wiley and Sons. 2011:1-550.
9. Ravaillon M. Evaluating anti-poverty programs. Handbook of Development Economics. 2007;4:37873846.
10. Connell JP, Kubisch AC. Applying a theory of change approach to the evaluation of comprehensive community initiatives. Aspen Institute. 1998:15-44.
11. De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, et al. Theory of change: A theory-driven approach to enhance the Medical Research Council's framework. Trials. 2014;15(1):267.
12. Kellogg Foundation WK. Logic model development guide. WK Kellogg Foundation. 2004:1-72.
13. Thornton PK, Herrero M. Adapting to climate change in the mixed crop and livestock farming systems in subSaharan Africa. Nature Climate Change. 2015;5(9):830-836.
14. Dlamini ZP, Mamba SN. Extension service delivery models in Eswatini: A comparative assessment. Eswatini Journal of Agriculture. 2021;19(2):34-48.
15. Pawson R, Tilley N. Realistic evaluation. Sage Publications. 1997:1-235.