Propensity score matching evaluation of extension program impact on smallholder productivity and household welfare

Introduction

Does agricultural extension actually improve farmer outcomes, or do the observed differences between participants and non-participants reflect pre-existing advantages? This question has troubled extension evaluation for decades [1] . Farmers who seek out extension tend to be more educated, more commercially oriented, and better connected to markets than those who do not, making simple comparisons between participants and non-participants misleading [2] . Without accounting for this selection bias, impact evaluations risk attributing to extension what is really the effect of motivation, resources, or social networks [3] . Propensity score matching offers a statistical approach to this problem by constructing comparison groups that resemble treatment groups on observed characteristics, thereby reducing selection bias without requiring randomized assignment [4,5] . While PSM has gained traction in extension evaluation literature, its application in Central American contexts remains limited, and few evaluations have assessed impacts across multiple outcome dimensions simultaneously [6] . Guatemala presents a compelling case for rigorous extension evaluation. The country has invested substantially in smallholder extension through the Programa de Extensiion Rural since 2012, reaching an estimated 180,000 households annually [7] . Yet evidence of program impact relies largely on monitoring data and participant satisfaction surveys rather than designs capable of addressing selection bias [8] . This research applied PSM to estimate causal effects of extension participation on productivity, income, technology adoption, market access, and food security among smallholder maize producers in four departments of the Western Highlands.

Theoretical Framework

The Rubin causal model provides the theoretical foundation for this evaluation [9] . Under this framework, the causal effect of extension is defined as the difference between what a farmer achieves with extension participation and what the same farmer would have achieved without it. Since only one outcome is ever observed for each individual, the fundamental problem of causal inference must be addressed through statistical methods that construct credible counterfactuals [4] . PSM addresses this by estimating the probability of treatment (extension participation) conditional on observed covariates, then matching treated units to untreated units with similar propensity scores. The validity of PSM rests on two assumptions: conditional independence (no unmeasured confounders) and common support (overlap in propensity score distributions) [5,10] .

Materials and Methods

Materials

Instrument Development

The outcome measurement battery combined validated scales with locally adapted indicators. Technology adoption was measured through a 12-item index covering improved seed use, fertilizer management, soil conservation, and postharvest handling (Cronbach alpha = 0.81). Food security used a 9-item adapted Household Food Insecurity Access Scale (alpha = 0.87). Market access was assessed through 6 items on marketing channels, price information, and collective selling (alpha = 0.79). All scales were pilot-tested with 35 households in Solola department and refined based on field experience.

Methods

Propensity scores were estimated using logistic regression with treatment (extension participation) as the dependent variable and 11 covariates: household head age, education, gender, household size, farm size, land tenure, distance to market, prior extension contact, cooperative membership, credit access, and department. Nearest-neighbor matching (1:1) with a caliper of 0.05 was applied without replacement. Balance diagnostics assessed standardized bias before and after matching, with a threshold of 10%. Average treatment effects on the treated (ATT) were estimated for five outcomes. Rosenbaum bounds tested sensitivity to hidden bias. Analysis used Stata 17.0. All five outcome indicators showed statistically significant treatment effects after matching. Technology adoption showed the largest relative gain (30.6%), followed by maize yield (23.8%). Food security, while showing the smallest percentage gain (16.0%), was still significant at p = 0.004.

Results

Comprehensive Interpretation

The grouped bar chart illustrates the importance of matching: unmatched control groups showed larger apparent gaps with treatment groups than matched controls, indicating that naive comparisons overestimate extension impact. After matching, treatment effects were smaller but still substantial and statistically significant. Rosenbaum sensitivity analysis showed that an unmeasured confounder would need to increase the odds of treatment assignment by a factor of 1.45 to invalidate the yield result, suggesting moderate robustness to hidden bias.

Discussion

The PSM results demonstrate that extension participation generates meaningful, statistically significant gains across multiple outcome dimensions even after accounting for selection bias [1,4] . The 23.8% yield gain is consistent with meta-analytic evidence from sub-Saharan Africa reporting 15-30% productivity effects for extension participation, and the multi-dimensional impact pattern aligns with the theory that extension operates through multiple channels simultaneously: knowledge transfer, input access, market linkage, and social network effects [11,12] . The large pre-matching imbalances, particularly in education and cooperative membership, underscore why selection bias is a serious concern for extension evaluation in Guatemala [2,3] . Without matching, the estimated treatment effects would have been substantially larger, meaning that a significant portion of the apparent extension benefit actually reflects pre-existing advantages of participants. This has practical implications for program accountability: monitoring systems that track only participant outcomes without comparison to credible counterfactuals will systematically overestimate program impact [13] .

Technology adoption showed the largest treatment effect (30.6%), which is expected given that technology promotion is the most direct and immediate extension activity. Food security, positioned further along the impact pathway and influenced by many non-extension factors, showed the smallest effect (16.0%), consistent with the attenuation pattern typically observed in program impact chains [14,15] .

Practical Applications

These results carry several practical implications for extension program management. First, program managers should incorporate PSM or similar quasi-experimental into routine evaluation frameworks rather than relying solely on before-after or participant-only assessments. The infrastructure required is modest: systematic baseline data collection covering key covariates and comparable non-participant sampling. Second, the multi-dimensional impact pattern suggests that extension programs should be evaluated on outcome portfolios rather than single indicators; a program might show modest yield effects but strong market access gains that represent equally important welfare contributions. Third, the sensitivity analysis provides a template for communicating evaluation confidence: reporting the degree of hidden bias needed to overturn findings gives decision-makers practical information about result reliability.

Conclusion

Propensity score matching revealed that Guatemala's extension program produced significant positive effects on smallholder maize yields (23.8%), household income (18.8%), technology adoption (30.6%), market access (19.1%), and food security (16.0%) after controlling for selection bias. Pre-matching covariate imbalances confirmed that naive comparisons would have substantially overestimated program impact, highlighting the importance of rigorous evaluation methods. www.extensionjournal.com Extension programs that demonstrate impact through methods capable of addressing selection bias build stronger cases for continued investment than those relying on monitoring data alone. Routine integration of quasiexperimental evaluation into extension program management would strengthen both accountability and learning across the sector.

References

1. Birner R, Davis K, Pender J, Nkonya E, Anandajayasekeram P, Ekboir J. From best practice to best fit: a framework for designing and analyzing agricultural advisory services. Journal of Agricultural Education and Extension. 2009;15(4):341-355.
2. Heckman JJ, Ichimura H, Todd PE. Matching as an econometric evaluation estimator: evidence from evaluating a job training programme. Review of Economic Studies. 1997;64(4):605-654.
3. Godtland EM, Sadoulet E, de Janvry A, Murgai R, Ortiz O. The impact of farmer field schools on knowledge and productivity. Journal of Development Economics. 2004;74(1):27-49.
4. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41-55.
5. Caliendo M, Kopeinig S. Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys. 2008;22(1):31-72.
6. Ragasa C, Mazunda J. The impact of agricultural extension services in the context of a heavily subsidized input system. World Development. 2018;105:25-47.
7. MAGA. Memoria de labores del programa de extension rural 2012-2022. Guatemala City: Ministerio de Agricultura Ganaderia y Alimentacion; 2023. 128 p.
8. Berdegue JA, Fuentealba R. Latin American rural development: an inventory of programs and indicators. Santiago: RIMISP; 2011. 94 p.
9. Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology. 1974;66(5):688-701.
10. Dehejia RH, Wahba S. Propensity score matching methods for nonexperimental causal studies. Review of Economics and Statistics. 2002;84(1):151-161.
11. Anderson JR, Feder G. Agricultural extension: good intentions and hard realities. World Bank Research Observer. 2004;19(1):41-60.
12. Davis K, Nkonya E, Kato E, Mekonnen DA, Odendo M, Miiro R, Nkuba J. Impact of farmer field schools on agricultural productivity and poverty in East Africa. World Development. 2012;40(2):402-413.
13. White H. Theory-based impact evaluation: principles and practice. Journal of Development Effectiveness. 2009;1(3):271-284.
14. Rola AC, Jamias SB, Quizon JB. Do farmer field school graduates retain and share what they learn? An investigation in Iloilo, Philippines. Journal of International Agricultural and Extension Education. 2002;9(1):65-76.
15. Waddington H, Snilstveit B, Hombrados J, Vojtkova M, Phillips D, Davies P, White H. Farmer field schools for improving farming practices and farmer outcomes. Campbell Systematic Reviews. 2014;10(1):1-335.
16. Rosenbaum PR. Observational studies. New York: Springer; 2002. 377 p.
17. Imbens GW, Wooldridge JM. Recent developments in the econometrics of program evaluation. Journal of Economic Literature. 2009;47(1):5-86.
18. Feder G, Murgai R, Quizon JB. The acquisition and diffusion of knowledge: the case of pest management training in farmer field schools. Journal of Agricultural Economics. 2004;55(2):221-243.