|
| Evaluating Data Integrity, Behavioral Variability, and Observational Reactivity in Skill Acquisition Within Behavior Analytic Research |
| Saturday, May 24, 2025 |
| 4:00 PM–4:50 PM |
| Convention Center, Street Level, 150 AB |
| Area: EAB; Domain: Translational |
| Chair: Andrea Nicole Michaels (University of Nevada, Reno) |
| Abstract: This symposium explores critical factors influencing data collection and reliability, behavioral assessments, and natural variability in skill acquisition within behavior analytic research. The first presentation investigates how treatment integrity impacts the reliability and accuracy of data collection during service delivery. Implications of treatment implementation errors are discussed. The second presentation aims to address natural variability in skill acquisition, which highlights importance of replication and mastery criteria when interpreting differences in performance. Emphasis on the need to account for inherent variability when drawing conclusions from skill acquisition research are discussed. The third presentation examines student reactivity to different observer conditions during academic tasks. Results indicate that both the location and relationship to the observer significantly impact on-task behavior. Practical implications for designing more ecologically valid observational assessments are discussed. Together, these studies provide valuable insight into data reliability, quantifying natural variability in skill acquisition research, and considering the implications of observers within assessments. |
| Instruction Level: Intermediate |
| Keyword(s): Behavioral Variability, Data Reliability, Observational Reactivity |
| |
| An Evaluation of the Effect of Treatment Integrity Errors and Observation Conditions on the Accuracy and Reliability of Data Collection |
| (Applied Research) |
| SWATHI RAGULAN (University of Nevada, Reno), Andrea Nicole Michaels (University of Nevada, Reno), Bethany P. Contreras Young (University of Nevada, Reno) |
| Abstract: Within the applied domain of behavior analysis (ABA), it is common practice to collect data reliability and data accuracy measures. These measures are often collected as part of an ongoing evaluation of behavioral services. In other words, critical decisions related to the presumed success or failure of specific ABA interventions and procedures are based on data that are collected during the course of service delivery (Vollmer et al., 2008). However, several factors in relation to staff-delivered consequences in the context of service delivery may impact the accuracy and reliability of collected data, including the treatment integrity of the services delivered along with the observation conditions of the data collection period. Data that are dependent on staff-delivered consequences, instead of the client behaviors, may result in making data-based decisions that can detrimentally impact the client and the success of the intervention (Vollmer et al., 2008). As such, it is crucial that data that are collected in the context of service delivery are both reliable and accurate in relation to the client’s behaviors. This study evaluated the effects of treatment integrity errors and observation conditions on the reliability and accuracy of data. |
| |
| Empirical Null Distribution of Differences in Sessions to Mastery: Implications for Comparative Skill Acquisition Research |
| (Applied Research) |
| ABRAÃO FIGUEIRA DE MELO (University of Nevada, Reno), Anna Ingeborg Petursdottir (University of Nevada, Reno), Faith Horne (Texas Christian University), Nicole M. Rodriguez (University of Nebraska Medical Center's Munroe-Meyer Institute) |
| Abstract: In comparative skill acquisition studies, two sets of instructional targets are often taught simultaneously under different conditions. The present study sought to quantify natural variability in simultaneous acquisition of two sets of conditional discriminations taught identically. Data from 72 college-student participants were used to construct an empirical null distribution of differences in sessions to mastery, from which we calculated the empirical probability of obtaining differences of various sizes. Using stringent mastery criteria, the empirical probability of obtaining at least a one-session difference in sessions to mastery was .75, and the probability of obtaining a relative difference of at least 10% was .63 in the absence of an independent variable. The probability of obtaining a directionally consistent difference in consecutive evaluations fell below .05 with three replications of a 1-session or greater difference and with two replications of a 10% or greater difference. Using more lenient mastery criteria increased the number of replications needed to have confidence in the results. However, regardless of mastery criteria, the number of needed replications decreased as the criterion for considering data to be differentiated became more demanding. The data highlight factors to consider when evaluating the believability of data from comparative skill acquisition studies. |
| |
| Effects of Reactivity on Student On-Task Behavior |
| (Applied Research) |
| DARIA GRIFFITH FUELL (Utah State University; University of Kansas), Audrey N. Hoffmann (Utah State University) |
| Abstract: This study addresses student on-task behavior during academic tasks and examines student reactivity to observer presence. The investigation aimed to understand how different observer conditions—no observer, observer behind an observation window, researcher observer in the session room, and caregiver observer in the session room—influence student on-task behavior. The study assessed on-task behavior and academic work completion using a multi-element design. The literature review highlights the scarcity of recent studies on reactivity, particularly concerning how students respond to adult observers during academic tasks. Three elementary-aged students were observed in a controlled environment, with results indicating measurable differences in on-task behavior across observer conditions. The study found that observer locality and relationship can influence student behavior, with unfamiliar or hidden observers having reactive effects of higher levels of on-task behavior. Limitations of the study include a small sample size, concerns about ecological validity, and brief observation periods. Despite these limitations, the research provides valuable insights for Applied Behavior Analysis professionals, emphasizing the need to consider observer effects in behavioral assessments and interventions. |
|
| |