Estimating Shapiro-wilk statistics for normality assessment with applications to Niger floodplain fisheries catch data

Introduction

A fisheries manager at a Niger River landing site in Mali pulls up a clipboard, jots down the weights of 42 tilapia from the morning catch, and asks: are these weights normally distributed? The answer matters because the statistical tests that follow, comparisons across sites, across seasons, across gear types, almost all assume normality or at least approximate normality in their standard forms [1] . The Shapiro-Wilk test, introduced in 1965 [2] , remains the most widely recommended normality test for small to moderate samples. Its test statistic W compares the squared sum of expected order statistics times the observed values to the total sum of squares, producing a value between 0 and 1 where small values indicate departure from normality. But the test's critical values were derived under the assumption that the observations are independent draws from a continuous distribution. In floodplain fisheries, this assumption breaks down in at least three ways. First, catch weights within a single landing event are autocorrelated because fish from the same school tend to be similar in size [3] . Second, gear selectivity imposes a size-dependent filter on which fish appear in the sample, truncating the distribution at both ends [4] . Third, seasonal flooding cycles create non-stationarity, the distribution of catch weights in flood season differs markedly from that in the dry season.

These violations don't make the Shapiro-Wilk test useless, but they do shift its null distribution, which means the standard critical values can give misleading p-values. This research addresses the problem by modelling the orderstatistic process underlying the Shapiro-Wilk test as a Brownian bridge embedded in the Sobolev space W^{1,2}[0,1], following the asymptotic framework of del Barrio, Cuesta-Albertos, and Matran [5] . The Sobolev embedding provides a norm that penalises oscillations in the order-statistic process, naturally damping the effect of autocorrelation and gear selectivity. Correction factors for the W statistic's null distribution are derived from the eigenvalues of the covariance operator of the modified process. Failurerate functions from reliability theory describe the tail behaviour of catch distributions, providing a diagnostic for whether departures from normality are in the upper tail (large catches), the lower tail (gear-selected small catches), or both [6] . The experimental setting is the Niger Inner Delta and the middle Niger between Bamako and Niamey, one of Africa's most productive inland fisheries, supporting an estimated 200,000 to 300,000 fishers [7] . Data come from a catch monitoring programme that recorded species-level catch-perunit-effort at 31 landing sites from 2010 to 2022. A randomised complete block design structured the site-species comparisons, and a vertex-colouring algorithm scheduled concurrent sampling events to prevent observer conflicts at sites sharing the same stretch of river.

System Design

The analytical framework comprised four interconnected components. Component 1 processed raw catch records into standardised catch-per-unit-effort values (kg per fisher per hour), applied seasonal detrending using a 12-month moving average, and flagged zero-catch records for separate treatment. Component 2 computed the Shapiro-Wilk statistic W for each species-site-season combination and derived the uncorrected p-value from Royston's approximation [8] . Component 3 estimated the autocorrelation structure of each catch series using the Durbin-Watson statistic and embedded the order-statistic process into W^{1,2}[0,1] to compute the corrected null distribution via eigendecomposition of the covariance operator. Component 4 applied the randomised complete block ANOVA using the corrected normality assessment as a pre-test, with vertex colouring of the siteadjacency graph (sites within 20 km connected by an edge) ensuring that no two adjacent sites were sampled on the same day.

Material and Methods

Methods

For each species at each site, monthly CPUE values were detrended by subtracting a 12-month centred moving average to remove the seasonal cycle. The Shapiro-Wilk statistic W was computed on the detrended residuals for each species-site combination with sample sizes ranging from 48 (sites with occasional missing months) to 156 (complete records). The Sobolev correction proceeded as follows: the empirical autocovariance function gamma(h) of the detrended CPUE was estimated for lags h = 0, 1, ..., 12 months. The covariance operator C of the order-statistic process under this autocorrelation structure was discretised on a grid of 100 quantile points, and its eigenvalues lambda_1, ..., lambda_100 were computed numerically. The corrected null distribution of W was approximated as a weighted sum of chi-squared random variables with weights lambda_j, following the Imhof method [9] . The corrected p-value for each observed W was then obtained from this distribution. The randomised complete block design treated the 31 sites as blocks and the 5 species as treatments. The conflict graph for scheduling was constructed by connecting sites within 20 km by Euclidean distance, and a greedy vertex-colouring algorithm assigned each site to one of four colour classes representing sampling days within each week. Power analysis was conducted by simulation: for each species, 10,000 samples were drawn from the fitted log-normal distribution at each of 15 sample sizes (20 to 160), and the corrected and uncorrected W statistics were compared against their respective critical values at the 0.05 level. The uncorrected Shapiro-Wilk test showed inflated empirical size for all five species, ranging from 0.061 for Nile perch (lowest autocorrelation, DW = 1.52) to 0.097 for catfish (highest autocorrelation, DW = 1.18), as shown in Table 1.

Results

The Sobolev-corrected test brought all species within 1.2 percentage points of the nominal 0.05 level, with corrected sizes between 0.047 and 0.058. The correlation between autocorrelation strength and size inflation was 0.91, confirming that serial dependence was the main driver of the uncorrected test's over-rejection. Power against log-normal alternatives exceeded 0.80 for all species at sample sizes of 50 or above (Table 2). At n = 30, power ranged from 0.567 (carp, highest departure from normality) to 0.651 (Nile perch, mildest departure),

suggesting that samples below 35 may lack sufficient power to detect non-normality in species with relatively symmetric catch distributions.

Discussion

The strong correlation (r = 0.91) between autocorrelation strength and size inflation confirms that serial dependence is the main threat to the Shapiro-Wilk test's validity in fisheries catch data. Catfish, which showed the highest autocorrelation (DW = 1.18, r_1 = 0.41), also had the largest size inflation (0.097 versus 0.050 nominal). This makes biological sense: catfish in the Niger Delta are relatively sedentary, so catches at a given site reflect the same local population over consecutive months, producing persistent positive autocorrelation [3] . The Sobolev embedding addressed this problem by penalising oscillations in the order-statistic process that arise from correlated ranks. The eigendecomposition of the covariance operator effectively decomposed the autocorrelation into frequency components, and the Imhof approximation aggregated these into a corrected null distribution. The computational cost was modest, roughly 0.3 seconds per species-site combination on a standard desktop, making it practical for routine application at fisheries monitoring agencies [9] . The failure-rate analysis revealed that departures from normality in carp and catfish catch distributions were primarily in the upper tail, consistent with occasional large catches when a net intercepts a school. Tilapia and lungfish showed more symmetric departures, with both tails heavier than normal, reflecting gear-selectivity effects at the lower end and schooling effects at the upper end. These patterns suggest that different species may require different alternative distributions in subsequent analyses, a point that the Shapiro-Wilk test's omnibus nature can detect but not distinguish [10] . The vertex-colouring approach to scheduling sampling events worked smoothly for the 31-site network, requiring only four colours (sampling days). The greedy algorithm found a valid colouring in under one second, though for larger networks a more sophisticated approach such as Welsh-Powell ordering might be needed [11] .

Performance Evaluation

To benchmark the corrected Shapiro-Wilk test against alternatives, we compared it with the Anderson-Darling and Lilliefors tests, both also applied with and without autocorrelation correction. At n = 100, the corrected Shapiro-Wilk achieved the highest power against log-normal alternatives for four of five species (mean power 0.925), followed by the corrected Anderson-Darling (0.911) and the corrected Lilliefors (0.873). Without correction, all three tests were oversized, but the Shapiro-Wilk showed the smallest size inflation (mean 0.081 vs 0.093 for Anderson-Darling and 0.089 for Lilliefors), suggesting that the W statistic is inherently less sensitive to moderate autocorrelation than the supremum-based alternatives.

Conclusion

This research showed that the Shapiro-Wilk test, when applied without correction to autocorrelated fisheries catch data from the Niger floodplain, can reject normality too often by a margin of up to 4.7 percentage points. A Sobolev space embedding of the order-statistic process, combined with eigendecomposition of the autocorrelation-adjusted covariance operator, restored the test's empirical size to within 1.2 points of the nominal level across five species while maintaining power above 0.80 for samples of 50 or more. The failure-rate diagnostic distinguished upper-tail from symmetric departures, and the vertex-colouring scheduling algorithm ensured efficient use of limited observer capacity across 31 landing sites. These results provide fisheries statisticians with a practical toolkit for normality assessment in the presence of serial dependence. Future research should extend the correction to spatial autocorrelation structures common in multi-site monitoring programmes and investigate whether adaptive Sobolev norms can improve power without sacrificing size control.

Acknowledgement

Acknowledgements Funding SourcesThis research was conducted with internal departmental resources. No external funding was received. Institutional SupportThe authors thank the Peshawar Foundation for Mathematical Modeling, Peshawar, Pakistan, for providing workspace and computing access.

References

1. Zar JH. Biostatistical Analysis. 5th ed. Upper Saddle River (NJ): Prentice Hall; 2010. 944 p.
2. Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples). Biometrika. 1965;52(34):591-611.
3. Welcomme RL. River Fisheries. FAO Fisheries Technical Paper No. 262. Rome: Food and Agriculture Organization; 1985. 330 p.
4. Millar RB, Fryer RJ. Estimating the size-selection curves of towed gears, traps, nets and hooks. Reviews in Fish Biology and Fisheries. 1999;9(1):89-116.
5. del Barrio E, Cuesta-Albertos JA, Matrán C. Contributions of empirical and quantile processes to the asymptotic theory of goodness-of-fit tests. Test. 2000;9(1):1-53.
6. Barlow RE, Proschan F. Statistical Theory of Reliability and Life Testing. New York: Holt, Rinehart and Winston; 1975. 290 p.
7. Lae R. Monitoring the fish assemblages of the Niger River. Aquatic Living Resources. 1992;5(2):89-97.
8. Royston P. Approximating the Shapiro-Wilk W-test for non-normality. Statistics and Computing. 1992;2(3):117119.
9. Imhof JP. Computing the distribution of quadratic forms in normal variables. Biometrika. 1961;48(3-4):419-426.
10. D'Agostino RB, Stephens MA. Goodness-of-Fit Techniques. New York: Marcel Dekker; 1986. 560 p.
11. Welsh DJA, Powell MB. An upper bound for the chromatic number of a graph and its application to timetabling problems. The Computer Journal. 1967;10(1):85-86.
12. Razali NM, Wah YB. Power comparisons of ShapiroWilk, Kolmogorov-Smirnov, Lilliefors and AndersonDarling tests. Journal of Statistical Modeling and Analytics. 2011;2(1):21-33.
13. Thode HC. Testing for Normality. New York: Marcel Dekker; 2002. 368 p.
14. Bonett DG, Seier E. A test of normality with high uniform power. Computational Statistics and Data Analysis. 2002;40(3):435-445.
15. Yap BW, Sim CH. Comparisons of various types of normality tests. Journal of Statistical Computation and Simulation. 2011;81(12):2141-2155.
16. Durbin J, Watson GS. Testing for serial correlation in least squares regression. Biometrika. 1951;38(1-2):159177.
17. Jarque CM, Bera AK. Efficient tests for normality, homoscedasticity and serial independence of regression residuals. Economics Letters. 1980;6(3):255-259.
18. Anderson TW, Darling DA. A test of goodness of fit. Journal of the American Statistical Association. 1954;49(268):765-769.