Assessing the Sensitivity and Robustness of Randomization Test and F-Test in Analysis of Repeated Measures Design with Missing Observations

Abstract

ABSTRACT In this work, the sensitivity and robustness of randomization test and F-test were assessed in a single factor repeated measures design with missing observations under cases when the data were normal, non normal, contained outliers and when sphericity condition was met or not met under varied sample size and number of treatments. The assessment was done by comparing the statistical power and p-value of the randomization test with that of F-test. The Monte Carlo approach was used in the simulation study on sensory adaptation experiment. The results showed that in the complete case analysis with normal data, the randomization test was approximately as sensitive and robust as the F-test, while it was more sensitive than the F-test when data had skewed distributions (Chi-square, exponential, lognormal and Weibull distributions) used in this work. The randomization test was more sensitive and robust than the F-test in the presence of an outlier, with and without missing observations. When observations were missing, the randomization test was mildly more sensitive than the F-test for normal data, while it was substantially more sensitive than the F-test when the data distribution were skewed as the percentage of missingness increased. When sphericity condition was met, the randomization test and the F-test were approximately equally sensitive; whereas the randomization test was more sensitive and robust than the F-test when sphericity condition was not met. The randomization test was therefore recommended to replace the F-test in analyzing single factor repeated measures design.