Association for Behavior Analysis International

Single-case experimental design researchers have long emphasized the importance of replicating effects within and across participants. Replication is critical for both internal and external validity. One potential implication of this dual role is that single-case researchers may experience a trade-off between robust internal validity through clear demonstration of experimental control and exploring the boundary conditions of interventions. The emphasis on strong and clear experimental control and aversion to publishing null or ambiguous results may result in stifling precise understanding of external validity. Single-case research has produced considerable advances toward better understanding of effective interventions and improving skills for individuals requiring support; however, serious challenges in defining what works for whom and under what conditions remain.

There is evidence that SCED studies not demonstrating robust experimental control are less likely to be published. As a result, some SCED researchers may exclude data that shows weak or no experimental control from research reports submitted for peer review. In other cases, researchers may omit procedural components to enhance the appearance of robust experimental effect from study descriptions, resulting in attribution of effects entirely to the described intervention(s). Traditional views of replication, valuable as they are, may not be sufficient to advance understanding about when, how, and why interventions are less effective or ineffective. This presentation will focus on what SCED researchers can do, individually and as a community, to address these and related challenges.

In the first part of the presentation, we will briefly overview concerns about replication identified in research domains outside of single-case research (e.g., “the replication crisis”) and how these map onto SCED research. We will discuss examples of systematic replication in thematic lines of research and how these culminate in identification of interventions and strategies as evidence-based. We then will illustrate potential ways the function of replication might be strengthened or weakened through researcher decisions.

In the second part of the presentation, we will present research scenarios and datasets  that highlight the described challenges with replication. For example, we will present SCED datasets to attendees and ask them to make hypothetical procedural and reporting decisions based on the data. We will also discuss editorial and reviewing scenarios involving research reports reflecting varying degrees of experimental control. Attendees and presenters will collectively explore solutions to these scenarios with identification of specific action steps to support a robust and valid SCED research literature.

Single-case researchers, reviewers, editors, and those who teach single-case designs

Matt Tincani, PhD, BCBA-D focuses on the application of behavioral principles to improve outcomes for people with disabilities and other special learning needs. His interests include systematic reviews of behavior science research, teaching job-related skills to neurodivergent people, smart technologies to improve special education, single-case designs, meta science, and open science. His recent work has explored questionable and improved research practices in single-case designs. He currently serves on the editorial boards of several prominent journals in special education and behavior analysis and has served in several regional and national leadership roles, including as Appointee to the Nevada Governor's Councils on Development Disabilities and Autism, and as founding president of the Positive Behavior Support SIG of the Association for Behavior Analysis InternationalHe has published over 90 scholarly publications, including the books Classroom Management and Positive Behavior Support (2nd edition; Routledge) and Adults with Autism Spectrum Disorders, Evidence-Based and Promising Practices (Guildford Press). He is currently co-principal investigator of an NSF-funded research project to explore support for people with neurodevelopmental disorders in attaining employment in information technology fields.

The purpose of the Single Case Conference was to identify questionable research practices in the conduct of single-case experimental designs (SCEDs) and ways to correct them. A characteristic of this conference has been its highly interactive nature. Each session includes small group discussions to foster the discussion of the speakers’ presentations. Points and questions raised in the small-group discussion will be posted in a dynamic group document. In this session, these points are examined and discussed as a community because the participants are part of the larger community of people applying SCEDs in their practice and in their science. Questions from small group discussions that occur across all sessions will be posed to the speakers, who will have an opportunity to respond and further the discussion. It is also anticipated that material from this session will inform the last session, in which journal editors will serve as a panel.

Single-case researchers, reviewers, editors, and those who teach single-case designs

The purpose of the Single Case Conference was to identify questionable research practices in the conduct of single-case experimental designs (SCEDs) and ways to correct them. Briefly, the goal is culture change: developing new and more effective ways to talk about SCEDs. Accordingly, each session comprises 1) speakers focused on the strengths and limitations of specific aspects of SCEDs and 2) small group discussions to foster the discussion of the speakers’ presentations. Points raised in the small-group discussion will be posted in a dynamic group document. In the penultimate session, these points are examined and discussed as a community because the participants are part of the larger community of people applying SCEDs in their practice and in their science. The last session will be a workshop of journal editors because the quality of published SCED studies will determine how these methods are incorporated in future examinations of behavior science. In Session 12, journal editors, the gatekeepers who determine what will be published, serve as a panel to address these and other points that arise. Editors will be asked questions that arise from the conference, and they will focus on how change can be effected through the peer review process and journal policies. This last session is viewed as the path to improved quality of this important collection of methods.

Single-case researchers, reviewers, editors, and those who teach single-case designs

Stephanie M. Peterson, Ph.D., is Associate Dean of the College of Arts and Sciences and Professor Psychology at Western Michigan University.  She previously served as the Chair of the Department of Psychology for 8 years.  She earned her doctorate in Special Education at The University of Iowa in 1994.  Before coming to Western Michigan University, she taught at Gonzaga University, Utah State University, The Ohio State University, and Idaho State University.  Her primary research interests are behavior analytic assessment and treatment aimed at decreasing chronic severe behavior problems in children with developmental disabilities.  Specifically, she studies choice making, functional communication training, reinforcement-based interventions for children with problem behavior, and concurrent schedules of reinforcement in the treatment of severe problem behavior.  She also studies functional analysis of problem behavior and teleconsultation methods for assessment and treatment of problem behavior.  She also has interests in applications of behavior analysis to educational interventions and teacher/behavior analyst training. She has served on a variety of editorial boards, including the Journal of Applied Behavior Analysis and Behavior Analysis in Practice.  Most recently, she served as the editor of Behavior Analysis in Practice for four years.  In the past, she also served as a Senior Editor for Education and Treatment of Children for many years.  She served two 3-year terms on the Board of Directors for the Behavior Analyst Certification Board and served by gubernatorial appointment on the Michigan Board of Behavior Analysts, Michigan’s licensing board for behavior analysts.  She served as the President of the licensing board for two years.  In addition to these leadership responsibilities, she serves as Director for the Kalamazoo Autism Center and the Autism Center of Excellence on Western Michigan University’s campus.

Dr. Mandy Rispoli is the Quantitative Foundation Bicentennial Professor of Special Education at the University of Virginia and a Board Certified Behavior Analyst- Doctoral level. She is the Editor of Behavior Modification and the Co-Editor of the Journal of Positive Behavior Interventions. Dr. Rispoli’s scholarship is built upon sustained university-community partnerships to improve teacher and caregiver meaningful involvement in functional behavior assessment and intervention and to promote positive outcomes for young children with autism and developmental disabilities. Dr. Rispoli’s research explores: (a) improving the efficiency, effectiveness, and feasibility of functional behavior assessments and function-based interventions in educational settings, and (b) innovations in professional development for teachers of young children with autism and challenging behavior. Dr. Rispoli has published over 140 peer-reviewed research articles and book chapters concerning behavioral interventions for children with autism and developmental disabilities.