|
| Everyone Cares About Quality: How Do We Show It? |
| Monday, May 31, 2021 |
| 9:00 AM–10:50 AM |
| Online |
| Area: AUT/OBM; Domain: Translational |
| Chair: Ellie Kazemi (Behavioral Health Center of Excellence) |
| Discussant: Richard Wayne Fuqua (Western Michigan University) |
| CE Instructor: Richard Wayne Fuqua, Ph.D. |
| Abstract: What patients, their parents/guardians, insurance providers, and the community at large have in common with direct care staff, clinical supervisors, and ABA service organizations is that each stakeholder cares about quality services. To assureall stakeholders that quality services are being provided, a neutral entity must set standards to define what is considered quality. Such standards are typically based on the scientific literature, and where research may fall short, subject matter experts provide guidance based on best practice. Then, to determine if services meet these standards, an objective entity conducts thorough evaluations using reliable assessment methods. In this symposium, we will share how Behavioral Health Center of Excellence (BHCOE) has developed a quality assurance system with a focus on our accumulated data collected over the past five years. The first presenter will discuss the value of quality assurance and best practice recommendations for conducting quality assurance. The second presenter will discuss outcomes assessments and why they are important for determining the value of behavioral interventions for individuals with Autism Spectrum Disorder. The third and fourth presenters will share the results of evaluations with patients and staff. All presenters will discuss the implications of their findings and future steps. |
| Instruction Level: Intermediate |
| Keyword(s): assessment, patient outcomes, quality assurance, supervision |
| Target Audience: Audience members should have a general understanding of assessments such as the VB-MAPP and Vineland Adaptive Behavior Scales as well as concepts such as social validity, treatment fidelity, and organizational systems. |
| Learning Objectives: At the conclusion of the presentation, participants will be able to: (1) define quality assurance and identify common strategies for measuring, assessing, and reporting on quality assurance; (2) describe the difference between individual and organizational outcomes; and (3) describe how patient and staff surveys can be employed measures of quality assurance. |
| |
| A Multimodal Approach to Measuring Quality Assurance |
| (Applied Research) |
| NIKKI WILLIAMS (Behavioral Health Center of Excellence), Sara Gershfeld Litvak (Behavioral Health Center of Excellence), David J. Cox (Behavioral Health Center of Excellence; Endicott College), Ellie Kazemi (Behavioral Health Center of Excellence) |
| Abstract: Quality assurance in human care services refers to a systematic process that organizational employees conduct to determine if the services that employees provide meet quality standards. Important components of the quality assurance process are the collecting and reporting on data. One way to evaluate quality is through the use of multimodal measures that examine key performance indicators. This presentation describes multimodal assessment strategies for quality assurance in Applied Behavior Analysis (ABA) organizations. To do this, we discuss the importance and use of different key performance indicators collected from 220 ABA organizations for approximately 14,500 patients throughout the United States. For example, 65% of organizations assess their supervisors for competence. But, when analyzed by the number of patients served, 57% of patients work with supervisors whose competence has been assessed. Assessing supervisor competence is one example of how the type of measurement taken and the analysis of obtained data can influence statements about quality assurance. Throughout our presentation, we will discuss additional examples to highlight the many ways quality assurance can be measured, assessed, and reported. |
| |
| Organizational Outcome Data: Don't I Already Do That? |
| (Applied Research) |
| SCOTT PAGE (Behavioral Health Center of Excellence), Sara Gershfeld Litvak (Behavioral Health Center of Excellence), Ellie Kazemi (Behavioral Health Center of Excellence), David J. Cox (Behavioral Health Center of Excellence; Endicott College) |
| Abstract: Behavior analysts commonly use skill-based and adaptive assessments to analyze individual patient outcomes and to customize treatment programs. However, as a whole, such assessments provide limited demonstration of organizational effectiveness and the data that might speak to organizational outcomes do not appear to be widely collected. The growth of applied behavior analysis as an effective treatment option for individuals with Autism Spectrum Disorder is causing funding sources to be increasingly interested in the accurate measurement, assessment, and reporting of organizational outcomes. In this presentation, we describe the distribution of organizational outcome data submitted during accreditation processes spanning five years and involving 218 organizations and 15 norm-referenced and criterion-referenced assessments. We then discuss examples of the procedures being used to track organizational outcomes. Finally, we review some of the many benefits that result from tracking organizational outcomes. These include: communicating internally with staff and patients about current quality of care; communicating organizational effectiveness to potential clients and funding sources; identification of opportunities for targeted staff training; and the ability to use data to make decisions that drive company progress toward organizational mission and values. |
| |
| Patient Satisfaction as a Quality Assurance Metric: What it Does and Doesn’t Tell Us |
| (Applied Research) |
| P. MICAH FRIDDLE (Behavioral Health Center of Excellence), Sara Gershfeld Litvak (Behavioral Health Center of Excellence), Ellie Kazemi (Behavioral Health Center of Excellence), David J. Cox (Behavioral Health Center of Excellence; Endicott College) |
| Abstract: Quality assurance measurements are an important, but under-utilized and under-researched, component of applied behavior analysis (ABA) services. Measuring patient satisfaction is one type of quality assurance measure that ensures the social validity of services offered by ABA providers. In this study, we sought to determine which characteristics of clinical quality and organizational processes have the greatest impact on the overall satisfaction of patients or their caregivers. As part of a comprehensive quality review of ABA service providers, we administered patient satisfaction surveys to the patient or their primary caregivers. Each survey asked questions about the caregivers’ level of agreement with statements about their service provider spanning six domains of clinical quality and organizational processes. These domains were: caregiver involvement, patient progress, navigating funding, scheduling, staff training and abilities, and treatment programs. Regression analyses suggest patient progress was the most important predictor of overall caregiver satisfaction. Additionally, the organizational processes of scheduling, staff training and abilities, and caregiver involvement were predictive of overall patient satisfaction. In total, the data and methods presented here highlight how measuring patient satisfaction may help ABA providers identify barriers to patient satisfaction and to develop targeted, function-based interventions to overcome these barriers. |
| |
| Staff Satisfaction Surveys: A Multi-Organization Analysis of Quality Assurance Data |
| (Applied Research) |
| MELISSA COTTENGIM (Behavioral Health Center of Excellence), Sara Gershfeld Litvak (Behavioral Health Center of Excellence), Ellie Kazemi (Behavioral Health Center of Excellence), David J. Cox (Behavioral Health Center of Excellence; Endicott College) |
| Abstract: Quality assurance (QA) systems are widely adopted practices in healthcare, pharmacy, and laboratory settings. In the field of applied behavior analysis (ABA), quality assurance is equally important but is not a current standard practice. In this study, staff satisfaction surveys were administered to 27,472 employees at 360 ABA organizations through the BHCOE accreditation process. Survey response completion rate was at 65% with 17,855 employee respondents. The survey comprised 67 total questions, measured through a five-point Likert scale, across seven sections including work engagement, career development, compensation, benefits, relationship management, scheduling, and work environment. We examined the relationship between employee satisfaction and overall quality markers reviewing data that had been collected over the past four years. We used a predictive model fit through linear regression to pinpoint the most meaningful sections of our staff satisfaction survey that predict an organization’s overall accreditation score. The results suggest the most important predictor of staff satisfaction was work engagement and the least important predictor was scheduling and work environment. We will discuss considerations for organizations in developing a QA system, the development and utility of staff satisfaction survey, and directions for future research. |
|
| |