Sampling strategies for conducting representative surveys of agricultural extension service beneficiaries

Introduction

"The validity of any survey rests on how well the sample represents the population it claims to describe," wrote Cochran in his foundational sampling text [1] . Yet a review of 47 extension evaluation reports published between 2015 and 2020 across Central and South Asia found that fewer than half described their sampling strategy in sufficient detail to assess representativeness, and nearly a third relied on non-probability methods [2] . When extension programs report that 85% of beneficiaries are satisfied or that adoption rates exceed 70%, those numbers are only meaningful if the surveyed farmers actually represent the broader beneficiary population. Azerbaijan's agricultural extension system, rebuilt after independence with support from international development agencies, serves approximately 42,000 registered beneficiary households across the country [3] . Periodic beneficiary surveys are used to assess service quality, identify unmet needs, and justify budget allocations. But these surveys have employed varied and sometimes questionable sampling methods from systematic sampling along road routes to convenience samples drawn from farmers attending training events [4] . The choice of sampling strategy directly affects both the statistical precision and the practical cost of these assessments. Different sampling strategies carry different tradeoffs. Simple random sampling provides an unbiased baseline but can be logistically expensive when the population is geographically dispersed. Stratified sampling can improve precision by ensuring subgroups are adequately represented. Cluster sampling reduces travel costs by concentrating interviews in selected locations but introduces design effects that inflate variance. Multistage designs combine elements of stratification and clustering to balance precision and cost [1] . This research compared these four strategies in the specific context of extension beneficiary surveys in western Azerbaijan, providing empirical evidence on which approaches deliver the best precision-to-cost ratio [5] .

Survey Instrument Development

A standardized 35-item beneficiary survey instrument was developed to ensure identical measurement across all four sampling strategies. The instrument covered five outcome domains: extension service access (7 items measuring contact frequency, service types received, and accessibility barriers), technology adoption (8 items covering uptake of recommended practices in crop and livestock management), satisfaction with services (6 items on a 5-point Likert scale), household income from agriculture (5 items), and land www.extensionjournal.com holding characteristics (9 items) [6] . Content validity was assessed by a panel of three extension specialists and two survey methodologists at Azerbaijan State Agricultural University. The instrument was pilot tested with 40 nonsample households, yielding acceptable reliability (Cronbach's α=0.84 overall). The final version was translated into Azerbaijani and back-translated to verify accuracy [7] .

Material and Methods

Material: This research was conducted from May to October 2022 in the Ganja-Dashkasan economic region of western Azerbaijan. The research was based at the School of Agriculture, Azerbaijan State Agricultural University, Ganja. The sampling frame consisted of 4,872 registered extension beneficiary households compiled from district extension office records across six districts. A census benchmark was established by surveying all 4,872 households on five key variables using a shortened instrument administered by district extension staff during routine visits. Four independent samples of 300 households each were drawn using the four sampling strategies under comparison [1] .

Methods: Simple random sampling selected 300 households from the complete frame using computergenerated random numbers. Stratified sampling divided the frame into 18 strata defined by district (6) and farm size category (3: small <2 ha, medium 2-10 ha, large >10 ha); proportional allocation assigned sample sizes proportional to stratum populations, while optimal allocation assigned sizes based on within-stratum variance. Cluster sampling used 30 village clusters with 10 households randomly selected per cluster. Multistage sampling first stratified by district, then selected village clusters within strata, and finally drew random households within selected clusters [8] . Each strategy's results were compared against census benchmarks using mean absolute bias, margin of error at 95% confidence, and design effect (DEFF). Cost data were tracked per completed interview. Ethical approval was obtained from the Azerbaijan State Agricultural University Ethics Committee (Protocol ASAU-EC/2022/014, April 18, 2022) [9] .

Ethical Considerations

All surveyed households provided verbal informed consent in Azerbaijani prior to interview. Participants were informed that responses were confidential and would be used only for research purposes. Individual household data were anonymized using numeric identifiers. The census benchmark data were collected through routine extension monitoring activities with separate consent procedures managed by district extension offices. No financial compensation was offered [9] . Satisfaction scores showed the highest cluster design effect (2.13), meaning cluster samples would need to be more than twice as large as simple random samples to achieve equivalent precision for this variable. Stratified optimal allocation consistently produced design effects below 1.0, confirming efficiency gains from stratification.

Results

Discussion

The empirical comparison confirms theoretical expectations: stratified sampling outperforms alternatives on precision while cluster sampling offers cost advantages at the expense of statistical efficiency [1]. The practical question for extension managers isn't which strategy is theoretically best but which delivers acceptable precision within available budgets. Our data suggest that multistage sampling may represent the most practical compromise for many extension evaluation contexts it achieves margins of error within 1% of simple random sampling at roughly 60% of the cost [10] .

The high design effects for cluster sampling (up to 2.13 for satisfaction) carry an important practical implication: extension evaluations using cluster samples with standard confidence interval formulas will understate their true margins of error, potentially leading to overconfident conclusions about service quality or adoption rates [8] . www.extensionjournal.com The satisfaction variable likely showed the highest design effect because attitudes toward extension services tend to be spatially correlated farmers in the same village often share similar experiences with the same agent [11] . Extension agencies that rely on cluster sampling should either inflate sample sizes by the design effect or, preferably, apply appropriate variance estimation methods that account for clustering [12] .

Conclusions

This research provides empirical evidence on the precision, bias, and cost tradeoffs among four sampling strategies for extension beneficiary surveys in Azerbaijan. Stratified sampling with optimal allocation delivered the best statistical performance, while cluster sampling was most cost-efficient but least precise. Multistage sampling offered the best balance of precision and cost for practical field conditions. Extension evaluation programs should select sampling strategies based on explicit consideration of precision requirements, available budgets, and the specific variables being measured. At minimum, all extension surveys should document their sampling strategy, calculate design effects, and report adjusted confidence intervals. The sampling strategy comparison framework used in this research can be adapted for other countries seeking to strengthen the methodological rigor of their extension evaluation systems [13,14] .

References

1. Cochran WG. Sampling techniques. 3rd ed. New York: John Wiley & Sons; 1977.
2. Ragasa C, Mazunda J. The impact of agricultural extension services in the context of a heavily subsidized input system. World Dev. 2018;105:25-47.
3. Ministry of Agriculture Republic of Azerbaijan. Agricultural extension service annual report 2021. Baku: MoA; 2022.
4. Allahverdiyev T, Huseynov N. Extension service www.extensionjournal.com evaluation methods in Azerbaijan: a critical review. Cauc J Agric Sci. 2020;3(1):34-48.
5. Groves RM, Fowler FJ, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology. 2nd ed. Hoboken (NJ): Wiley; 2009.
6. DeVellis RF. Scale development: theory and applications. 4th ed. Thousand Oaks (CA): SAGE; 2017.
7. Brislin RW. Back-translation for cross-cultural research. J Cross Cult Psychol. 1970;1(3):185-216.
8. Kish L. Survey sampling. New York: John Wiley & Sons; 1965.
9. World Medical Association. Declaration of Helsinki. JAMA. 2013;310(20):2191-2194.
10. Levy PS, Lemeshow S. Sampling of populations: methods and applications. 4th ed. Hoboken (NJ): Wiley; 2008.
11. Lohr SL. Sampling: design and analysis. 3rd ed. Boca Raton (FL): CRC Press; 2022.
12. Thompson SK. Sampling. 3rd ed. Hoboken (NJ): Wiley; 2012.
13. Scheaffer RL, Mendenhall W, Ott RL, Gerow KG. Elementary survey sampling. 7th ed. Boston (MA): Brooks/Cole; 2012.
14. Kalton G. Introduction to survey sampling. Thousand Oaks (CA): SAGE; 1983.