Adaptive Tests of Significance Using Permutations of Residuals with R and SAS

PREFACE

This book was written to introduce researchers to adaptive tests of significance and to describe the advantages of using these testing methods. Traditional tests of significance, such as the two-sample t test and the t test for slope, are robust in the sense that non-normality of the error distribution often does not dramatically change the level of significance. So, why should we use adaptive tests?

Adaptive tests are used to increase the power when the errors are not normally distributed. In real-world testing situations we rarely know the distribution of the errors, so it is important to know just how the traditional tests compare to adaptive tests with a variety of normal and non-normal error distributions. The power comparisons, which are displayed throughout the book, show that the adaptive test is often much more powerful than the traditional test with many non-normal error distribution.

Adaptive tests use the data to adjust the test procedures. For example, if a re-searcher wants to perform a two-sample test and the data suggests that the error distributions may be normally distributed, the traditional test procedure is not modified a great deal, so that the resulting test will approximate a two-sample t test. However, if the data contain a few outliers, then the test procedure will be modified to down weight the importance of those outliers.

At first glance these adaptive tests of significance are suspicious. It does not seem right to use the data to modify the ...