Meta-analysis techniques for synthesizing effect sizes across agricultural extension intervention research

Introduction

On one hand, a trial in western Kenya reports that farmer field schools doubled maize yields; on the other, a similar trial in eastern Uganda found no significant yield difference between participants and controls [1] . Both investigations were well designed, yet their conclusions point in different directions. This kind of inconsistency is common in agricultural extension research, where context, crop, intervention design, and measurement approaches vary enormously from one investigation to the next [2] . Narrative reviews attempt to reconcile such contradictions, but they rely on subjective judgement and are prone to selective emphasis on dramatic results [3] . Meta-analysis addresses this problem by converting findings from individual investigations into a common metric, the effect size, and statistically pooling them to estimate an overall effect while quantifying the variation around it [4] . The technique has been used extensively in medical and educational research for decades but remains underused in agricultural sciences, particularly in extension research where it could inform policy decisions about which intervention models deserve investment [5] . This research applied meta-analytic techniques to a corpus of 84 agricultural extension intervention investigations published between 2010 and 2024. The objectives were threefold: to estimate the overall pooled effect of extension interventions on agricultural outcomes, to assess and explain heterogeneity across investigations using moderator analysis, and to evaluate the presence and potential impact of publication bias on pooled estimates [6,7] .

Theoretical Framework

Meta-analysis rests on the statistical principle that combining results from multiple independent investigations increases the precision of effect estimation beyond what any single investigation can achieve [4] . The random-effects model, adopted in this research, assumes that true effect sizes vary across investigations due to real differences in populations, interventions, and contexts, rather than sampling error alone [8] . Heterogeneity, measured by the Qstatistic and expressed as the I² percentage, indicates the proportion of total variability attributable to betweeninvestigation differences. When I² exceeds 75%, moderator analysis is warranted to identify sources of variation [9] . Effect size metrics in this domain include Cohen's d and Hedges' g, with the latter preferred for its correction of small-sample bias, an important consideration given that many extension trials involve fewer than 100 participants [10] .

Materials and Methods

Materials

A systematic literature search was conducted across five electronic databases: Scopus, Web of Science, CAB Abstracts, AGRIS, and Google Scholar. The search used Boolean combinations of terms including 'agricultural extension,' 'intervention,' 'impact,' 'effect,' 'randomised,' 'quasi-experimental,' and 'outcome.' Searches were limited to English-language, peer-reviewed articles published between January 2010 and December 2024. Grey literature was excluded to maintain quality control. The search was executed between January and March 2024 at the Central University of Venezuela library system. Inclusion criteria required: quantitative outcome reporting with means, standard deviations, or sufficient statistics to compute effect sizes; a comparison group (control or pre-intervention); and extension as the primary intervention [11] . Ethical review was not required as this research involved secondary data analysis of published works.

Methods

The systematic review followed PRISMA guidelines [12] . Two reviewers independently screened titles, abstracts, and full texts, with disagreements resolved by consensus. Methodological quality was assessed using a modified Newcastle-Ottawa Scale adapted for extension intervention research. Effect sizes were calculated as Hedges' g with 95% confidence intervals. Where investigations reported multiple outcomes, the primary outcome specified by the authors was selected to avoid dependency. Pooled effects were estimated using a DerSimonian-Laird random-effects model in R software (version 4.3.2) using the metafor package [13] . Heterogeneity was assessed via Q-statistic and I². Moderator analyses used mixed-effects models with categorical moderators: intervention type, geographic region, outcome category, and investigation design. Publication bias was evaluated through visual funnel plot inspection, Egger's regression test, and the trim-and-fill method [14] .

Coding Instrument Development

A standardised coding instrument was developed to extract data from included investigations. The instrument captured 34 variables organised in five domains: investigation identification, methodological characteristics, intervention description, outcome metrics, and effect size data. The instrument was pilot-tested by both reviewers on 10 randomly selected investigations, achieving inter-rater agreement of 91.4% (Cohen's kappa = 0.87). Discrepancies were resolved through discussion, and coding definitions were refined before full extraction began. For investigations reporting only F-statistics or chi-square values, conversion formulas from Borenstein and colleagues (2009) [4] were applied to derive standardised effect sizes. The overall pooled effect of g = 0.53 represents a medium positive impact. Participatory approaches produced the largest effect (g = 0.71), more than double that of information-only interventions (g = 0.31). Among outcome categories, yield outcomes showed the largest pooled effect (g = 0.58), while income outcomes displayed the highest heterogeneity (I² = 82.6%), suggesting that income effects are most context-dependent. Sub-Saharan Africa contributed the most investigations (k = 34) and showed the largest regional effect (g = 0.59). However, the between-group Q-test was not significant (p = 0.428), indicating that geographic region alone does not explain the observed heterogeneity. Within-region variation remained substantial across all regions. The majority of investigations reported medium effects (37.8%), followed by small effects (28.4%). Large effects (22.1%) and very large effects (11.7%) were less common, with the latter concentrated among participatory and training-plus-inputs interventions. The flowchart traces the screening pathway from 1,847 initial records through title/abstract screening (412 retained), full-text assessment (138 evaluated), and quality appraisal, yielding 84 investigations for final inclusion. Exclusion reasons at each stage are displayed alongside.

Results

Discussion:

The overall pooled effect of g = 0.53 provides a benchmark that individual investigations cannot offer. By Cohen's (1988) [15] conventions, this represents a medium effect, suggesting that agricultural extension interventions, taken as a class, produce meaningful but not transformative improvements in farming outcomes. This estimate is comparable to the pooled effects reported by Waddington and colleagues (2014) [16] in their meta-analysis of farmer www.extensionjournal.com field schools (d = 0.48) and slightly higher than the pooled impact found by Magruder (2018) [17] for technology adoption interventions (d = 0.41).

The finding that participatory approaches (g = 0.71) substantially outperformed information-only methods (g = 0.31) carries direct policy implications. Extension agencies investing in passive dissemination strategies, such as leaflets, radio messages, or one-off demonstrations, can expect roughly half the effect of programs that actively engage farmers in decision-making and experimentation [18] . This pattern held across regions and outcome types, suggesting it reflects a genuine feature of extension effectiveness rather than a contextual artefact.

The high overall heterogeneity (I² = 78.3%) is both expected and informative. Extension interventions differ in nearly every dimension: crop, climate, institutional context, and target population. That moderator analysis explained some but not all of this variation underscores the importance of context in extension programming and the limits of what pooled estimates alone can tell us [19] .

Limitations

Several limitations should be noted. The restriction to English-language publications may have excluded relevant investigations from Francophone and Lusophone Africa and Latin America. Grey literature exclusion, while maintaining quality, likely introduced publication bias favouring positive results, as confirmed by Egger's test (p = 0.037). The coding instrument, though reliable, required judgement calls when investigations reported outcomes ambiguously. The use of single primary outcomes per investigation may have missed important secondary effects. Finally, the analysis treated diverse agricultural contexts as comparable, which, despite moderator adjustments, remains an inherent simplification [20] .

Conclusion

This meta-analysis demonstrates that agricultural extension interventions produce a medium-sized positive effect on farming outcomes, with participatory approaches generating substantially larger impacts than passive information delivery. The overall pooled effect of g = 0.53 offers a robust benchmark for policymakers and program designers seeking evidence-based guidance on extension investment. Substantial heterogeneity across investigations confirms that context matters and that no single intervention model works uniformly well everywhere. Meta-analysis provides the methodological rigour needed to move extension research beyond narrative reviews toward quantitative evidence synthesis, but its value depends on the quality and transparency of primary investigations. These findings argue for greater adoption of meta-analytic methods within agricultural extension research communities and for improved reporting standards in primary investigations that would make future syntheses more precise and informative.

References

1. Davis K, Nkonya E, Kato E, Mekonnen DA. Impact of farmer field schools on agricultural productivity and poverty in East Africa. World Dev. 2012;40(2):402413.
2. Birner R, Davis K, Pender J, Nkonya E. From best practice to best fit: A framework for designing and analyzing pluralistic agricultural advisory services worldwide. J Agric Educ Ext. 2009;15(4):341-355.
3. Slavin RE. Best evidence synthesis: An intelligent alternative to meta-analysis. J Clin Epidemiol. 1995;48(1):9-18.
4. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester: John Wiley & Sons; 2009. p. 1-421.
5. Anderson JR, Feder G. Agricultural extension: Good intentions and hard realities. World Bank Res Obs. 2004;19(1):41-60.
6. Higgins JPT, Thomas J, Chandler J, editors. Cochrane handbook for systematic reviews of interventions. Version 6.3. London: Cochrane Collaboration; 2022. p. 1-694.
7. Cooper H, Hedges LV, Valentine JC, editors. The handbook of research synthesis and meta-analysis. 3rd ed. New York: Russell Sage Foundation; 2019. p. 1632.
8. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188.
9. Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-1558.
10. Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. J Educ Stat. 1981;6(2):107-128.
11. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and metaanalyses: The PRISMA statement. Ann Intern Med. 2009;151(4):264-269.
12. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.
13. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1-48.
14. Duval S, Tweedie R. Trim and fill: A simple funnelplot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455-463.
15. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Lawrence Erlbaum Associates; 1988. p. 1-567.
16. Waddington H, Snilstveit B, Hombrados JG, Vojtkova M, Phillips D, Davies P, et al. Farmer field schools for improving farming practices and farmer outcomes: A systematic review. Campbell Syst Rev. 2014;10(1):1335.
17. Magruder JR. An assessment of experimental evidence on agricultural technology adoption in developing countries. Annu Rev Resour Econ. 2018;10:299-316.
18. Feder G, Murgai R, Quizon JB. The acquisition and diffusion of knowledge: The case of pest management training in farmer field schools in Indonesia. J Agric Econ. 2004;55(2):221-243.
19. Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks: SAGE Publications; 2001. p. 1-247.
20. Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002.