Association for Behavior Analysis International

Evaluating procedural fidelity—the extent to which procedures are carried out as designed—is essential to providing training and supervision to behavior-change agents and monitoring the quality of behavior-analytic services. Nevertheless, there are barriers to collecting procedural-fidelity data in practice, so more research is needed to enhance our understanding of procedural-fidelity measurement in real-world settings. This symposium presents three studies addressing this topic. First, Resendez et al. will discuss a study examining the use of multiple measurement systems to assess procedural fidelity of discrete-trial instruction procedures, highlighting the differences observed with each system. Next, Blackman et al. will present findings from procedural-fidelity data collection by providers at 29 organizations, providing an overview of the current state of procedural fidelity in ABA services and its implications. Finally, Echeveria et al. will review a study examining an assessment-based intervention designed to increase the frequency with which behavior analysts measure procedural fidelity in practice. These studies collectively offer valuable insights for supervisors to improve fidelity measurement to support staff in providing high-quality services.

supervising BCBAs and behavior-analytic researchers

Assessing procedural fidelity is essential for evaluating staff performance and treatment effectiveness, yet research comparing different measurement methods remains limited. To address this gap, we systematically replicated Bergmann et al. (2023), which explored how procedural fidelity in discrete-trial-instruction (DTI) sessions varies based on the measurement system and calculation method. In contrast to their approach, our study simulated real-time conditions by viewing recorded sessions without pausing or rewinding, making the process more representative of actual practice. Interobserver agreement was calculated across studies to make conclusions about the reliability of each measurement system in real-time conditions. We will discuss the key differences observed between the systems, with particular emphasis on connecting our findings to the utility of procedural-fidelity assessment in practice. This study highlights the importance of selecting the appropriate measurement system based on the specific goals and needs of the evaluation, ensuring more accurate and meaningful fidelity assessments during supervision sessions.

In the pursuit of excellence, understanding and implementing treatment fidelity (Gresham, 1989) is paramount. Treatment fidelity serves as the backbone of quality service delivery in applied behavior analysis (ABA) therapy, directly impacting client outcomes (e.g., DiGennaro et al., 2007) and organizational standards. Despite its importance, many in the industry face considerable challenges in effectively collecting, tracking, and analyzing fidelity data (Colon et al., 2024). This presentation aims to bridge that gap by offering an overview of aggregated fidelity data from across the industry and highlighting its profound implications. We will discuss findings from an extensive analysis of de-identified fidelity data collected from 29 different organizations, showcasing the real-world state of treatment fidelity in ABA (Blackman et al., under review). These data reveal significant gaps in protocol adherence, and these shortcomings have a direct impact on both the quality of services delivered to consumers and the turnover rates among service providers.