A permutation test for nonindependent matched pair data

A permutation test for nonindependent matched pair data

Kang, Seung-Ho

The paired t-test and the Wilcoxon signed-rank test are often conducted to compare two continuous outcomes from paired observations. An assumption underlying these tests is that the responses from pair to pair are mutually independent. However, the assumption is violated in certain applications such as site-specific data in periodontal research. An adjustment to the paired t-test to account for the clustering effect has been well developed. But the adjustment relies on either large sample theory or the assumption that the observations being analyzed follow a normal distribution. In this paper we propose a permutation test for matched pair clustered data which are valid in small samples. We developed and reviewed software to carry out the proposed test. The proposed test is applied to real-life data.

Key Words: Paired t-test; Wilcoxon signed-rank test; Intraclass correlation; Dentistry


MATCHED PAIRS OF data often arise in medical statistics for comparing two treatments. In clinical trials subjects matched on the basis of characteristics that are associated with the response being studied are randomized to the treatments independently within each matched group. The purpose of matching in clinical trials is to increase the precision of the comparisons among the treatments. The matched pair samples may also represent two sets of measurements on the same patient. The most common paired design results when one group is measured twice. Oftentimes the first measurement occurs before treatment and the second measurement occurs after treatment.


There has been substantial research on exact inference in the past two decades (6) and permutation tests are valuable tools in exact tests. An advantage of permutation tests is that they are guaranteed to control the type I error rate under the nominal level, so that they can be valid even in small samples. However, the computation required made it impracticable to carry out permutation tests in the past. In this paper we proposed a permutation test for nonindependent matched pair data and developed and reviewed software to conduct the permutation test. A disadvantage of permutation tests is that we may not be able to achieve a type I error rate of 100(1 – ex)%, because the permutation distribution is discrete. This conservatism is the price that we should pay for exactness. Upton (7) argues for simply reporting the p-value and not making comparisons to arbitrary nominal levels so that one can reach one’s own conclusion based on the losses associated with each type of error.

Most research in exact tests has focused on the cases of independent data. The area of correlated data has received little attention to date. We hope that this work will stimulate much research in this area.

Acknowledgment-This work was supported in part by NIH grant MO1RR02588.


1. Blomqvist N. On the choice of computational unit in statistical analysis. J Clin Periodontology. 1985;12: 873-875.

2. Emrich LJ. Common problems with statistical aspects of periodontal research papers. J Periodontology. 1990;61:206-208.

3. Donner A, Eliasziw M. Application of matched pair procedures to site-specific data in periodontal research. J Clin Periodontology. 1991;18:755-759.

4. Donner A. A review of inference procedures for the intraclass correlation coefficient in the one-way random effects model. Int Stat Review. 1986;54:67-82.

5. Barnett JD, Melloneg JT, Gray JL, Towie HJ. Comparison of freeze-dried bone allograft and porous hydroxylapatic in human periodontal defects. J Periodontology. 1989;60:231-237.

6. Agresti A. A survey of exact inference for contingency tables. Stat Sci. 1992;7:131-177.

7. Upton GJG. A comparison of alternative tests for the 2 x 2 comparative trial. J Royal Stat Society A. 1982; 145:86-105.


Assistant Professor, Department of Statistics, Ewha Womans University, Seoul, Korea


Statistical Analyst, Department of Biostatistics, University of Texas, M. D. Anderson Cancer Center, Houston, Texas


Professor, Clinical Epidemiology, University of Texas Medical School, Houston, Texas

Reprint address: Seung-Ho Kang, PhD, Department of Statistics, Ewha Womans University, 11-1, Dae Hyun– Dong, SeoDaeMun-Gu, Seoul, 120-750, Korea. E– mail: seungho-kang@hanmail.net.

Copyright Drug Information Association Apr-Jun 2001

Provided by ProQuest Information and Learning Company. All rights Reserved