Comparison of the Friedman, modified Friedman, and Quade tests under symmetric and asymmetric distributions

Introduction

Experimental designs that involve both within-subject and between-subject factors are widely used in applied research, particularly in fields such as psychology, medicine, agriculture, and industrial quality control. Classical general linear models assume normality and homogeneity of variance. Although modern linear mixed models allow flexible variance-covariance structures and generalized mixed models accommodate non-normal distributions, these approaches still require correct model specification. In practice, misspecification or complex data features such as skewness, heavy tails, or outliers can affect inference, motivating the use of nonparametric methods. In such cases, nonparametric tests provide more robust and distribution-free alternatives, making them suitable for analyzing data that deviate from normality or containing outliers (Hollander, Wolfe, & Chicken, 2014) [7] . Classic references emphasize that rank-based tests require substantially weaker assumptions than their normaltheory counterparts and often exhibit high asymptotic relative efficiency even under normality (Conover, 1999; Gibbons & Chakraborti, 2011; Lehmann, 1975) [2,5,9] . Lehmann (1975) [9] established the theoretical foundations for rank-based inference, demonstrating that nonparametric procedures maintain validity across a wide range of distributional forms while achieving high relative efficiency under normality. In randomized complete block designs (RCBDs), the Friedman test (1937) has been the most used nonparametric procedure for detecting treatment effects when blocking is present. However, the Friedman test assumes that the variability among blocks is uniform, which may not hold when block variances differ substantially. To address this, the Modified Friedman test (Xu, J. et. al (2017); Jan & Shieh, 2025) [13] and the Quade test (1979) [8] were proposed as generalizations that incorporate block variability through ranking or weighting adjustments. These tests improve sensitivity to treatment differences when block effects are not homogeneous. More recently, Jan and Shieh (2025) [8] proposed an improved nonparametric test and accompanying sample size determination procedures for RCBDs, addressing limitations of traditional Friedman-type methods in maintaining Type I error control and maximizing power under small to moderate sample conditions. Their findings emphasize the ongoing relevance And potential for methodological innovation within the RCBD framework. Related studies have revisited the small-to-moderate sample accuracy of Friedman-type procedures, proposing F-transformations and explicit power approximations that improve Type I error control and power relative to the classical χ² approach (Jan & Shieh, 2025) [8] . Building upon these developments, the present research aims to systematically evaluate existing RCBD tests (Friedman, Modified Friedman, and Quade) for both k=3 and k=5 populations under both symmetric and asymmetric distributions. The general recommendation is to use the Quade test for a small number of treatment and the Friedman test if the number of treatments is 5 or more (Theodorsson-Norheim, 1987) [11] . This study aims to bridge this methodological gap and provide researchers with more reliable tools for inference when traditional assumptions of normality and homogeneity are violated. Furthermore, through Monte Carlo simulations under Normal, Exponential, Chi-Square, and t-distributions spanning symmetric, skewed, and heavy-tailed error structures components, this research will offer valuable insights into the robustness, efficiency, and power of the 3 nonparametric tests for an RCBD (Hettmansperger & McKean, 2011) [6] . The study will estimate and compare the statistical power and Type I error of these tests across different settings of the number of populations (k = 3, 5), the number of blocks (b = 10, 20). In parallel, we will evaluate the robustness of the proposed tests under four probability distributions Normal, Exponential, Chi-Square, and t-distribution (df = 3) using large-scale Monte Carlo simulations.

The friedman test (1937)

The Friedman test (Friedman, 1937, 1940) is a nonparametric procedure used to detect differences among treatments in a Randomized Complete Block Design (RCBD). It evaluates whether treatment effects differ significantly when blocking is used to control nuisance variability. The hypotheses are stated as:

Where 𝑀𝑀 𝑗𝑗 represents the population median of the 𝑗𝑗𝑗𝑗ℎ treatment. The test statistic is computed as:

Where: 𝑏𝑏= number of blocks 𝑘𝑘= number of treatments 𝐹𝐹 𝑗𝑗 = sum of ranks assigned to the 𝑗𝑗𝑗𝑗ℎ treatment within each block Under the null hypothesis, FR approximately follows a chi-square distribution with (k-1) degrees of freedom. The null hypothesis is rejected for large values of F, based on the critical value from the chi-square table. Definitions, assumptions, and asymptotic reference distributions for the Friedman rank tests follow Gibbons and Chakraborti (2011) [5] , who provided detailed derivations of their chi-square approximations and critical regions. Original development and rank-ANOVA justification appear in Friedman (1937, 1940), with practical χ² approximations reviewed in standard texts (Conover, 1999) [2] .

The modified friedman test

The Modified Friedman test (Xu, J. et. al, 2017) [13] improves the traditional Friedman test by incorporating within-subject variability through ranking based on sample variances. Let 𝑋𝑋 𝑖𝑖𝑗𝑗 be the observation for the 𝑖𝑖th subject and 𝑗𝑗th treatment, and let 𝑟𝑟 𝑖𝑖𝑗𝑗 denote its rank within subject𝑖𝑖. The sample variance of subject 𝑖𝑖is 𝑉𝑉 𝑖𝑖 , and its rank among all subjects is 𝜏𝜏 𝑖𝑖 . The hypotheses are stated as:

Where 𝑀𝑀 𝑗𝑗 represents the median of the jth treatment. The adjusted rank is defined as

∑ 𝑘𝑘 𝑗𝑗=1 ( 𝐹𝐹 ¯⋅𝑗𝑗 -𝐹𝐹 ¯⋅⋅ ) 2 Where:

𝐹𝐹 𝑖𝑖𝑗𝑗 is the mean adjusted rank for treatment 𝑗𝑗,

is the grand mean of adjusted ranks.

Under𝐻𝐻 0 , the statistic 𝑀𝑀𝑀𝑀𝑀𝑀𝐹𝐹 is asymptotically chi-square distributed with (𝑘𝑘 -1)degrees of freedom.

The Quade Test (1979)

The Quade test (Quade, 1979) is a nonparametric alternative to the two-way analysis of variance (ANOVA) for data obtained from randomized complete block designs (RCBD). It is particularly useful when block variances differ, because it assigns weights to blocks according to their importance, as reflected by the ranges within each block. Blocks with larger internal variability receive larger weights, which allows the test to incorporate block heterogeneity directly into the ranking structure and improves its sensitivity under non-additivity or unequal block variances. The Quade test is traditionally formulated with an F-distribution reference. For integration into our combined mixed-design framework, we transform the Quade F-statistic to a χ²-scale equivalent (see Transformation section below), maintaining consistency with the Friedman.

The hypotheses for the Quade test are stated as:

𝐻𝐻 1: At least one treatment median differs, Where 𝑀𝑀 𝑗𝑗 represents the median of the 𝑗𝑗th treatment.

The procedure begins by calculating the range within each block as:

so that blocks with a larger spread of observations are identified. Each block range is then ranked from smallest to largest, and its rank is denoted by 𝑄𝑄 𝑖𝑖 . Next, within each block, the observations are ranked, and a weighted score is calculated as:

Where 𝐹𝐹(𝑋𝑋 𝑖𝑖𝑗𝑗 )is the rank of the 𝑗𝑗th treatment within the 𝑖𝑖th block, and 𝑘𝑘is the number of treatments. The Quade test statistic is then obtained by applying standard one-way ANOVA to the weighted scores 𝑆𝑆 𝑖𝑖𝑗𝑗 , producing an 𝐹𝐹 𝑄𝑄𝑄𝑄statistic that approximately follows an F-distribution with degrees of freedom:

Where SSA is the treatment sum of squares, SSE is the error sum of squares, and 𝑏𝑏 is the number of blocks. Under 𝐻𝐻 0 , the classical rejection rule is:

Materials and methods

The performance the three nonparametric test statistics was examined through a comprehensive Monte Carlo simulation study conducted in SAS, where data generation followed established simulation principles (Wicklin, 2013) [12] . The primary objective of this simulation was to evaluate and compare the behavior in a randomized complete block design (RCBD) under a wide range of experimental conditions and probability distributions differing in symmetry, skewness, and tail behavior. Specifically, the study incorporated the following three well-established within-subject tests:

• The Friedman test (1937)

• The Modified Friedman test (1979)

• The Quade test (1979) All datasets were generated using the SAS random number generator function RAND ( ) with (SEED = 0) allowing the system clock to produce different random values in each run. These data-generation and computation steps were programmed in SAS using standard statistical programming practices (Bailer, 2010) [1] . Five thousand random samples were generated for each scenario, with the number of blocks for each scenario either 10 or 20. The number of populations considered was either 3 or 5.

The significance levels of each test under each of the various scenarios were estimated by counting the number of times the null hypothesis was rejected when the null hypothesis was true divided by 5000. Powers were estimated by counting the number of times each test rejected for a particular type of distribution and set of parameters and then divided by 5000. Using the maximum significance error a 95% confidence interval for each probability estimate would be est(p)±0.014. The data were generated from two symmetric distributions being the normal and the t-distribution with 3 degrees of freedom. The data were generated from two skewed distributions being the chi-square distribution with 2 degrees of freedom and the standard exponential distribution.

The rejection regions determined using the Chi-Square and quantile corresponding to a 0.05 significance level (Conover, 1999) [2] . Well-established references justify the large-sample χ² reference for the Friedman and Modified Friedman tests, and the F reference for the Quade test, with specific cautions regarding small-sample performance (Conover, 1999; Gibbons & Chakraborti, 2011; Hollander, Wolfe, & Chicken, 2014) [2,5,7] . These classical sources provide the theoretical foundations for using asymptotic χ² and F approximations in rank-based procedures, emphasizing that accuracy improves as the number of blocks or treatments increases. Through this simulation framework, the study aims to determine how effectively each test maintains Type I error control. If Type I error is controlled, the estimated powers are compared among the three tests.

Results and discussion

Results are given in Tables 12345678for all distributions studied with 20 blocks. Tables 1234included the results for 3 populations and Tables 5-8 include the results for 5 populations. The results for 10 blocks were similar. The first line of every table gives the estimated alpha values for each test under that scenario. Al tests had alpha values that were close to the stated alpha value of 0.05 with close defined to be in the interval (0.04, 0.06).

Results for the Normal distribution with 3 populations are given in Table 1. One can see that the power of the Quade test is generally the largest followed by the power of the Modified Friedman and then the Friedman. The same pattern is found for 5 populations in Table 2. The powers of the tests appear to be closer together for 5 populations than for 3 populations.

Results for the t-distribution are found in Tables 34. One can see that the estimated power of the Friedman test is higher than the power for the Modified Friedman or the Quade test. This is true for both 3 and 5 populations. The Quade test has the second highest power followed by the Modified Friedman. It appears that the powers between the tests are closer together for 3 populations than for 5 populations. Results for the exponential distribution are given in Tables 56. The same pattern holds as in the t-distribution case. The power of the Friedman test is higher than both the power of the Quade test and the power of the Modified Friedman. The Quade test has the second highest power. This is true for both 3 and 5 populations. It once again appears that the powers are closer together between the tests for 3 populations than for 5 populations.

Results for the chi-square distribution with 2 degrees of freedom are given in Tables 78. The same pattern holds as in both the tdistribution and the exponential distribution. The Friedman test has the highest powers. The Quade test has the second highest powers. The powers again appear to be closer together for 3 populations than for 5 populations.

Conclusion

The findings from this research do indicate that the Quade test has the highest powers for when the underlying distributions are normal and this was true for both 3 and 5 populations. The powers of the tests did become closer as the number of populations increased. The Quade test is recommended if one has fewer than 5 populations and thinks that the underlying distributions are symmetric with light to medium tails. For the heavier-tailed t-distribution with 3 degrees of freedom, the Quade test had the second highest powers with the Friedman test having the highest powers. The powers were closer together for 3 populations than for 5 populations. The Friedman test is recommended for heavy-tailed symmetric distributions and especially if the number of populations becomes 5 or greater.

Results for both the exponential and chi-square with 2 degrees of freedom distributions, the Friedman test had the highest powers followed by the Quade test. The powers appear to have a greater separation for 5 populations over 3 populations. The Friedman test is recommended if one suspects the underlying distributions to be skewed. This is especially true when the number of populations exceed 3.

References

1. Bailer, A. J. (2010). Statistical programming in SAS. SAS Institute Inc.
2. Conover, W. J. (1999). Practical nonparametric statistics (3rd Ed.). Wiley.
3. Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675-701.
4. Friedman, M. (1940). A comparison of alternative tests of significance for the problem of matched samples. Annals of Mathematical Statistics, 11(1), 86-92.
5. Gibbons, J. D., & Chakraborti, S. (2011). Nonparametric statistical inference (5th ed., updated version). CRC Press.
6. Hettmansperger, T. P., & McKean, J. W. (2011). Robust nonparametric statistical methods. CRC Press.
7. Hollander, M., Wolfe, D. A., & Chicken, E. (2014). Nonparametric statistical methods (3rd ed.). Wiley.
8. Jan, S. L., & Shieh, G. (2025). Improved nonparametric procedures and sample size determination for randomized block designs. Journal of Nonparametric Statistics. (In press).
9. Lehmann, E. L. (1975). Nonparametrics: Statistical methods based on ranks. Holden-Day.
10. Quade, D. (1979). Using weighted rankings in the analysis of variance. Journal of the American Statistical Association, 74(367), 680-683.
11. Theodorsson-Norheim, E. (1987). Friedman and Quade tests: BASIC computer program to perform nonparametric two-way analysis of variance and multiple comparisons on ranks of several related samples. Computers in Biology and Medicine,Vol. 17, Issue 2, pp 85-99.
12. Wicklin, R. (2013). Simulating data with SAS. SAS Institute Inc.
13. Xu, J. Shan, G., Amei,A., Zhao, J., Young, D., and Clark S. (2017). A Modified Friedman test for randomized complete block designs. Communications in Statistics -Simulation and Computation,Vol. 46. Issue 2, pp 1508-1519.