|
| Accountability Through Data Collection: Narrowing the Bridge Between Science and Practice |
| Saturday, May 29, 2021 |
| 3:00 PM–4:50 PM |
| Online |
| Area: CBM; Domain: Translational |
| Chair: Matthew L. Edelstein (Kennedy Krieger Institute) |
| Discussant: Kimberly Sloman (The Scott Center for Autism Treatment/ Florida Institute of Technology ) |
| CE Instructor: Matthew L. Edelstein, Psy.D. |
| Abstract: Distinguishing it from other service-delivery fields, behavior analysis places strong emphasis on accountability through ongoing data collection and objective progress monitoring. As part of this model, behavior analysts are expected to use data to guide their clinical decision-making. However, these decisions are only meaningful when the data they are based on are reliable and valid. In this symposium, we address issues related to data collection in the practice of behavior analysis. In the first presentation, Dr. Morris will review data collection practices reported by Board Certified Behavior Analysts and their relation to data integrity. In the second, Ms. Snyder will discuss barriers to reliable and accurate data collection by group home staff. In the third, Dr. Becraft will present data on the validity of parent report of severe problem behavior. In the fourth, Dr. Edelstein will discuss the validity of parent report via telehealth. Collectively, these studies further our understanding of factors impacting data collection in behavior analytic practice, which is critical to evaluating and improving outcomes for recipients of behavioral services. |
| Instruction Level: Intermediate |
| Keyword(s): Data Collection, Parent Report |
| Target Audience: Behavior analysts, licensed psychologists, clinicians |
| Learning Objectives: 1. Participants will be able to discuss methods to ensure data collection is reliable and valid 2. Participants will be able to identify best practices in data collection 3. Participants will be able to describe environmental variables that may impact valid data collection |
| |
| Toward an Understanding of Data Collection Integrity |
| (Service Delivery) |
| CODY MORRIS (Salve Regina University ) |
| Abstract: Data collection is essential to the effective practice of behavior analysis. Behavior analysts are even obligated to collect and use data to inform treatment decisions according to the Professional and Ethical Compliance Code for Behavior Analysts (BACB, 2014). However, it is unclear how trustworthy data collected in applied settings are when behavior analysts are not themselves recording the data. In fact, numerous research studies have reported issues with data collection integrity when supervisors did not directly intervene to ensure compliance with data collection requirements. The first purpose of this talk is to review survey data focused on data collection practices for monitoring problem behavior in applied settings. The reports from the survey will be shared and include information related to reporting requirements, training and monitoring practices, profile of data collectors, modality of collection, etc. The second purpose of this talk is to discuss the relation of commonly reported practices and data collection integrity. |
| |
| A Descriptive Analysis of Baseline Conditions Affecting Data Collection in a Group Home |
| (Service Delivery) |
| DAPHNE SNYDER (Western Michigan University), Stephanie M. Peterson (Western Michigan University) |
| Abstract: Although the importance of data collection is widely accepted in the field of behavior analysis, there has been limited to no research to date on the naturalistic variables affecting data collection in applied settings. Madsen and colleagues (2016) speculated on barriers present in the group home setting that may affect data collection, but no descriptive research has been conducted on the correspondence between the present barriers and the quality of data collected. Therefore, the purposes of this study were to: (a) evaluate the accuracy and reliability of data collected by group home staff under conditions and over a period of time that are more naturalistic than has been evaluated in past research, and (b) record the presence or absence of environmental variables hypothesized by Madsen and colleagues (2016) to be related to issues in staff data collection. A tool was developed to identify the presence of these variables and accuracy of staff data collection was measured. Data from a case study indicate that many of the speculated variables are present in the group home setting and may affect the accuracy of data collected by staff. |
| |
| The Validity of Parent Report for Evaluating Clinical Endpoints for Severe Problem Behavior |
| (Applied Research) |
| JESSICA L BECRAFT (Kennedy Krieger Institute), Patricia F. Kurtz (Kennedy Krieger Institute), Usai Bah (Kennedy Krieger Institute), Michael F. Cataldo (Kennedy Krieger Institute) |
| Abstract: In practice, behavior analysts or a similarly trained professional typically determine the effectiveness of treatment for child problem behavior. However, parents’ evaluation of treatment effects is critical because parents initiate treatment services, are expected to implement treatment protocols, are responsible for paying for services, determine when treatment is no longer required, and serve as advocates and marketers for services. We compared parent to data collected by trained observers on severe problem behavior in two studies. In the first study, parents reported data on their child’s behavior in baseline and treatment sessions. In the second study, parents scored the level of problem behavior in pre- and post-treatment videos of other children. Results indicate correspondence with trained observers on both a molecular (session-by-session) and molar (overall treatment efficacy) level. These studies suggest parents can accurately detect meaningful clinical endpoints, which can be used to evaluate maintenance and generalization of treatment and justify services to third party payers. |
| |
| Examining the Utility of Parent Report in the Age of Telehealth |
| (Applied Research) |
| MATTHEW L. EDELSTEIN (Kennedy Krieger Institute), Jessica L Becraft (Kennedy Krieger Institute), Lesley A. Shawler (Kennedy Krieger Institute Johns Hopkins University School of Medicine), Alicia Sullivan (Kennedy Krieger Institute), Sherika Harley (Kennedy Krieger Institute) |
| Abstract: The ability to use gold-standard behavior data collection by trained observers is limited in large outpatient settings. Establishing parent observation as a reliable and valid alternative is imperative to the continued application of best practice behavioral intervention in these settings, particularly following the move to telehealth service provision as a result of the COVID-19 pandemic. The current study assessed the validity of parent data on child problem behavior during virtual sessions at an outpatient clinic. Specifically, parent report was compared to trained observers’ frequency data. Results suggest that parents can accurately report clinically significant changes in target behavior. Specifically, a strong correspondence was noted between parent observation and trained observers when parents are asked to report rates more generally. These reports appear sensitive enough to determine the magnitude of change between high and low rates of problem behavior. Implications for the continued use of parent report in virtual service provision are discussed. |
|
| |