Measurement frameworks for assessing gender-related outcomes of agricultural extension interventions

Introduction

An observation that recurs across decades of agricultural development evaluation is this: programs that count women's attendance as evidence of gender equity often miss whether those women gained any meaningful benefit from participating [1] . Extension programs worldwide have increased female participation rates substantially over the past twenty years, yet evidence of corresponding improvements in women's decision-making power, resource control, or economic outcomes remains inconsistent [2] . The measurement tools used in most extension evaluations may be part of the problem. A scan of extension evaluation reports revealed that the majority measured gender outcomes through headcounts without assessing participation quality or downstream effects [3] . More sophisticated frameworks exist, notably IFPRI's Women's Empowerment in Agriculture Index (WEAI) [4] , but the WEAI was designed for national-level surveys requiring substantial infrastructure that most extension programs cannot support [5] . What evaluators need is a framework capturing multidimensional gender outcomes while remaining practical for routine assessment [6] . This research addressed that need through a systematic review followed by development and pilot testing of a comprehensive gender outcomes framework [7,8] .

Conceptual Framework:

The framework was grounded in Kabeer's empowerment model, defining empowerment through resources (pre-conditions), agency (process), and achievements (outcomes) [9] , extended by Alsop and Heinsohn's specification that empowerment manifests across economic, social, and political domains at household, community, and institutional levels [10] . The five dimensions mapped onto these foundations: resource access (preconditions), participation quality and decision-making authority (agency), and economic outcomes and empowerment index (achievements). Each dimension was operationalized through indicators measurable via structured interviews at program entry, midpoint, completion, and follow-up [11] .

Material and Methods

Material

This research comprised two components conducted at the University of Guelph, Ontario, Canada. Phase 1 was a systematic review of 93 gender-focused extension evaluation reports published between 2012 and 2022, identified through CAB Abstracts, Scopus, and grey literature from FAO, IFPRI, and CGIAR databases. Phase 2 was a longitudinal pilot test tracking 342 women participants across 14 extension programs in Ontario and Manitoba from January 2021 to June 2022 (18 months). Programs covered crop production, livestock management, value addition, and financial literacy. Women were assessed at baseline, midline (6 months), endline (12 months), and follow-up (18 months). The 45-item Gender Outcomes in www.extensionjournal.com Extension (GOE) instrument measured five dimensions using validated scales adapted from WEAI, the Gender Equitable Men Scale, and program-specific items [12] .

Methods:

The systematic review coded each evaluation's measurement approach, dimensions assessed, and instruments used. The GOE instrument was validated through exploratory factor analysis (EFA, n=171) followed by confirmatory factor analysis (CFA, n=171). Internal consistency used Cronbach's alpha; convergent validity was tested by correlating GOE with WEAI scores for 86 participants completing both instruments. Longitudinal changes were analyzed using repeated-measures ANOVA with Bonferroni comparisons. Variance decomposition compared single-indicator versus full-framework explanatory power. Ethical approval was obtained from the University of Guelph Research Ethics Board (Protocol REB/2020/AG-1147, November 18, 2020) [13] . The systematic review finding that 31.2% of evaluations relied on a single indicator (typically access) and another 24.7% used only two dimensions confirms the measurement gap motivating this research [1] . When singleindicator approaches captured only 38.4% of the variance explained by the full framework, it means over 60% of gender-related change went unmeasured in most evaluations. This has practical consequences: programs may appear successful on participation metrics while failing to produce meaningful changes in decision-making or economic outcomes [3] . The decision-making dimension deserves particular attention. It showed the lowest baseline scores (22.1), the smallest absolute gain, and the steepest post-program decline. This pattern suggests that changes in decisionmaking authority require more than skills training they depend on shifting household and community power dynamics that extension programs alone may not sustain [9] . The stronger resilience of resource access and economic outcome scores may reflect their more tangible, individually controllable nature compared to the relational character of decision-making power [14] . Extension programs seeking lasting gender outcomes should explicitly address the institutional and social conditions that constrain women's decision-making authority [15] .

Results

Practical Applications:

The GOE framework can be integrated into extension monitoring through several steps. The 45-item instrument adds approximately 25 minutes to existing intake procedures [16] . Programs with limited resources can use the 15-item short form as a screening tool. Dimension-specific scores enable targeted improvement if participation scores are high but decision-making lags, programs can adjust accordingly. The framework's alignment with WEAI facilitates cross-program and crosscountry comparison for policy decisions [17,18] .

Conclusions: This research developed and validated the Gender Outcomes in Extension framework, a fivedimension tool that captures gender-related changes more comprehensively than single-indicator approaches. The pilot test demonstrated that all dimensions responded to extension programming but at different rates and with different sustainability profiles. Decision-making authority emerged as the most challenging dimension to improve and sustain. The framework's strong psychometric properties and practical design make it suitable for routine integration into extension monitoring, enabling evaluators to move beyond participation headcounts toward meaningful assessment of gender equity outcomes.

References

1. Cornwall A. Unpacking participation: models, meanings and practices. Community Dev J. 2008;43(3):269-283.
2. World Bank. Gender in agriculture sourcebook. Washington: World Bank; 2009.
3. Ragasa C. Improving gender responsiveness of agricultural extension. IFPRI Discuss Pap. 2014;1361:1-26.
4. Alkire S, Meinzen-Dick R, Peterman A, Quisumbing A, Seymour G, Vaz A. The Women's Empowerment in Agriculture Index. World Dev. 2013;52:71-91.
5. Malapit H, Quisumbing A, Meinzen-Dick R, Seymour G, Martinez EM, Heckert J, et al. Development of the project-level Women's Empowerment in Agriculture Index (pro-WEAI). World Dev. 2019;122:675-692.
6. Doss CR. The role of women in agriculture. ESA Work Pap No. 11-02. Rome: FAO; 2011.
7. Peterman A, Behrman JA, Quisumbing AR. A review of empirical evidence on gender differences in nonland agricultural inputs. IFPRI Discuss Pap. 2011;1145:1-48.
8. Quisumbing AR, Meinzen-Dick R, Raney TL, Croppenstedt A, Behrman JA, Peterman A, editors. Gender in agriculture: closing the knowledge gap. Dordrecht: Springer; 2014.
9. Kabeer N. Resources, agency, achievements: reflections on the measurement of women's empowerment. Dev Change. 2001;30(3):435-464.
10. Alsop R, Heinsohn N. Measuring empowerment in www.extensionjournal.com practice: structuring analysis and framing indicators. World Bank Policy Res Work Pap. 2005;3510.
11. DeVellis RF. Scale development: theory and applications. 4th ed. Thousand Oaks: SAGE; 2017.
12. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 8th ed. Andover: Cengage; 2019.
13. Kline RB. Principles and practice of structural equation modeling. 4th ed. New York: Guilford Press; 2016.
14. Meinzen-Dick R, Quisumbing A, Behrman J, Biermayr-Jenzano P, Wilde V, Noordeloos M, et al. Engendering agricultural research, development, and extension. Washington: IFPRI; 2011.
15. Njuki J, Kaaria S, Chamunorwa A, Chiuri W. Linking smallholder farmers to markets, gender and intrahousehold dynamics. Eur J Dev Res. 2011;23(3):395412.
16. Davis K, Nkonya E, Kato E, Mekonnen DA, Odendo M, Miiro R, et al. Impact of farmer field schools on agricultural productivity. World Dev. 2012;40(2):402413.
17. Johnson NL, Kovarik C, Meinzen-Dick R, Njuki J, Quisumbing A. Gender, assets, and agricultural development. World Dev. 2016;83:295-311.
18. Quisumbing AR, Pandolfelli L. Promising approaches to address the needs of poor female farmers. World Dev. 2010;38(4):581-592.