|Organizational Behavior Management in Autism Service Delivery: A Three Year Review
|Monday, May 27, 2019
|3:00 PM–3:50 PM
|Hyatt Regency West, Ballroom Level, Toronto
|Area: OBM/AUT; Domain: Translational
|Chair: Emily Gallant (Somerset Hills Learning Institute)
|Discussant: Paul Shreiber (Somerset Hills Learning Institute)
|CE Instructor: Kevin J. Brothers, Ph.D.
In this presentation, we report longitudinal outcomes of two systems-based organizational behavior management strategies to improve staff performance at a private, not-for-profit school for children with autism. Instructional staff at the organization undergo semiannual evaluations assessing clinical and data-analysis skills that directly impact the quality of student outcomes. As a continuing process, the organization’s management team (i.e., classroom supervisors, assistant directors, and executive director) refine measures to increase their sensitivity to specified staff behaviors and permanent products. The longitudinal data presented reflect the effectiveness of this process in improving the quality of staff training over two years as measured via staff’s clinical performance and data notebook assessments. The first part of this presentation will describe the clinical performance evaluation, report related outcome data, and describe organizational strategies for increasing the sensitivity of these evaluation measures. The second part of this presentation will then describe the data notebook evaluation measures and their development, present relevant data, and discuss the relationship between these and third-party reviewers’ findings. In both presentations, the relationship between the time cost and value of these assessments will be discussed in detail.
|Instruction Level: Intermediate
|Keyword(s): behavioral systems, performance evaluation, staff training
Directors or assistant directors of private organizations serving individuals with autism spectrum disorder; BCBA Supervisors; individuals coaching staff or parents to deliver autism intervention using applied behavior analysis; organizational behavior management professionals and researchers, especially those in the fields of human services or education.
|Learning Objectives: 1. Attendees will be able to name and describe at least four staff performance evaluation measures that contribute to optimal outcomes for learners with autism spectrum disorder. 2. Attendees will be able to provide rationales for the number and specificity of clinical skills evaluated for individuals delivering applied behavior analytic intervention to learners with autism spectrum disorder. 3. Attendees will be able to describe features of data analysis that contribute to an improved rate of skill generalization for learners with autism spectrum disorder.
|Increasing Sensitivity of Staff Performance Evaluation Measures in Autism Service Delivery
|PAUL SHREIBER (Somerset Hills Learning Institute), Kevin J. Brothers (Somerset Hills Learning Institute), Emily Gallant (Somerset Hills Learning Institute)
|Abstract: This presentation will describe and summarize organizational procedures and outcomes of semiannual clinical skills evaluations. All staff working directly with learners with autism undergo semiannual evaluation of clinical skills from an evaluator who does not directly supervise them. Measures consist of two to five direct observations of three student behaviors, nine staff behaviors, and seven staff behavioral repertoires scored using rating scales. We will discuss efforts to increase the behavioral nature of all measures by transitioning from rating scales to direct observation. Specifically, we will review data from the four most recently-operationalized measures of staff performance (i.e., contingent token delivery, proportion of teaching interactions conducted using errorless teaching procedures, responding to student errors, and prompt fading and/or shaping) alongside student on-task data for the past three years. We will also describe the organization’s decision-making process to add and increase specificity of measures. We will further describe how outcome data describing staff performance are integrated into a comprehensive systems-based approach to organizational behavior management via feedback to staff supervisors and adjustments to training of staff supervisors.
|Advances in Behavioral Systems to Improve Data Analysis and Generalization of Behavior Change
|KEVIN J. BROTHERS (Somerset Hills Learning Institute), Paul Shreiber (Somerset Hills Learning Institute), Emily Gallant (Somerset Hills Learning Institute)
|Abstract: This presentation will describe and summarize organizational procedures and outcomes of annual evaluations of student performance data. In addition to semiannual clinical skills evaluations, permanent products of staff members’ instructional programming and data analysis activities are evaluated annually for all staff working directly with learners with autism. Efficacious programming ideally produces outcomes of rapid, stable behavior change in the desired direction, that maintains over time, serves as a foundation for more sophisticated skills, and generalizes to desired conditions. We will then discuss our operationalization, assessment, and analysis of this via six key permanent product (i.e., the outcomes of staff members’ graphing and record-keeping activities) indicators, with special emphasis on the improvement of generalization outcomes. In addition, we will describe our approach to efficacy assessment as an iterative process. We will further describe the relationship between these outcome measures and those assessed by an outside reviewer not affiliated with the organization. Finally, we will describe our iterative approach to our data-analysis assessment procedures and how this relates to the broader context of organizational management of staff behaviors.