Adaptive Tests of Significance Using Permutations of Residuals with R and SAS

CHAPTER 1

INTRODUCTION

1.1 WHY USE ADAPTIVE TESTS?

Many adaptive tests have been developed in an effort to improve the performance of tests of significance. We will consider a test of significance to be “adaptive” if the test procedure is modified after the data have been collected and examined. For example, if we are using a certain kind of two-sample adaptive test we would collect the data and calculate selection statistics to determine which two-sample test procedure should be used. If the data appear to be normally distributed, then a Wilcoxon rank-sum test would be used; but if the data appear to contain outliers, then a median test would be used.

Adaptive tests of significance have several advantages over traditional tests. They are usually more powerful than traditional tests when used with linear models having long-tailed or skewed distributions of errors. In addition, they are carefully constructed so that they maintain their level of significance. That is, a properly constructed adaptive test that is designed to maintain a significance level of α will have a probability of rejection of the null hypothesis at or near α when the null hypothesis is true. Hence, adaptive tests are recommended because their statistical properties are often superior to those of traditional tests.

The adaptive tests described in this book have the following properties:

• The actual level of significance is maintained at or near the nominal significance level of α.
• If the error distribution is ...