Association for Behavior Analysis International

The field of behavior analysis has traditionally eschewed the use of statistical tests in the analysis of single-case experimental design (SCED) data. In particular, behavior analysts have argued against parametric statistics (e.g., t-tests and ANOVAs) for multiple reasons. Rather than use statistical tests, behavior analysts have relied upon inter-ocular trauma tests, where the visual analysis of SCED results hits you right between the eyes. The field has, generally speaking, overlooked the fact that parametric tests are only a few hammers in a much larger toolkit of statistical procedures. It is possible and beneficial for behavior analysts to use both methods—visual analysis and statistical tests—in conjunction with one another to analyze their data. In this symposium, the presenters will review three different non-parametric statistical tests that can be used in basic and applied behavior analytic research: Randomization tests, general estimating equations (GEE), and multilevel modeling. Each has its own unique merits and uses within SCEDs, and can function to augment other methods of analysis and replace more commonly used statistical tests.

Master's level and doctoral level BCBAs; Graduate Students in Behavior Analysis; Basic Researchers; Applied Researchers; Scientist-Practitioners

The frequent and repeated measurement of behavior often precludes behavior analysts from making statistical inferences about data obtained from single-case experimental designs (SCED). Parametric tests assume a random sample, independent observations, and a normal distribution. SCEDs violate one or more of these assumptions, and even worse, are considered quasi-experimental because subjects are not randomly sampled from a defined population or randomly assigned to treatments. Recent advances in computing, however, have brought an old and readily applicable test of significance to fore: Randomization Tests (Fisher, 1935; Pitman, 1937). Unlike conventional Null Hypothesis Significance Tests (NHST), randomization tests are non-parametric, distribution-free tests of statistical significance. They are particularly applicable to SCEDs, so long as treatments are randomly assigned to observations. The requirement that SCEDs include random assignment increases their methodological rigor by controlling for unknown variables and addresses the charge that SCEDs are quasi-experimental. While randomization tests cannot supplant the experimental control already built into SCEDs, they can certainly supplement the conclusions behavior analysts might make about treatment effects. The purpose of this presentation, then, is to elucidate the origins of randomization tests, explicate their applicability to SCEDs, and warn against the pitfalls of NHSTs when making inferences.

The field of Behavior Analysis has historically been conflicted over the use of inferential statistical methods in the analyses of data from single-subject designs. Valid concerns with the use of inferential statistics include the suppression of important behavioral variability at the individual level and the over-reliance on and misinterpretation of the p-value. This conflict has commonly resulted in two strategies: first, reliance on visual analyses and the out-right rejection of any inferential statistics, or second, the application of more "basic" inferential tests that may or more not be appropriate for single-subject design data. Multilevel modeling (e.g., mixed-methods or hierarchal regression) is a more advanced statistical analysis that addresses many of the concerns that the field of Behavior Analysis has with inferential statistics including quantifying the contribution of individual behavioral variability to the results and the compression of many data-points into a single comparison. The benefits of multilevel modeling will be demonstrated using several single-subject design datasets. A guide of how researchers can implement multilevel modeling including a priori recommendations before beginning data collection will be offered.