|
| Considerations for Administering and Analyzing the Performance Diagnostic Checklist – Human Services |
| Monday, May 29, 2023 |
| 8:00 AM–8:50 AM |
| Hyatt Regency, Mineral Hall A-C |
| Area: OBM; Domain: Applied Research |
| Chair: Denys Brand (California State University, Sacramento) |
| Discussant: Florence D. DiGennaro Reed (University of Kansas) |
| CE Instructor: Florence D. DiGennaro Reed, Ph.D. |
| Abstract: This symposium includes two talks that provide considerations and guidance for administering the performance diagnostic checklist – human services (PDC-HS; Carr et al., 2013), interpreting the data, and selecting indicated interventions. The PDC-HS is a performance analysis tool designed to assess the environmental variables impacting employee performance problems in human service settings. Numerous researchers have used the results of the PDC-HS to successfully address a variety of performance problems, including problems related to implementing behavior analytic procedures, and adhering to safety measures, as well as workplace cleaning expectations (e.g., Bowe & Sellers, 2018; Ditzian et al., 2015; Wilder et al., 2018; Hess et al., 2019). Although the assessment is valid, limitations related to the objectivity and clarity of its administration, data interpretation, and intervention selection guidelines have been raised (Wilder et al., 2019; 2020). In response, Jimenez will discuss considerations and recommendations for administering the PDC-HS assessment, and Vance will share refinements developed for data interpretation and intervention selection that researchers and practitioners may consider adopting. |
| Instruction Level: Intermediate |
| Keyword(s): PDC-HS, performance analysis, performance management, staff training |
| Target Audience: Intermediate |
| Learning Objectives: At the conclusion of the presentation, participants will be able to: (1) describe the typical administering process of the PDC-HS (2) identify how practitioner experience level might impact the accuracy of administering the PDC-HS, and (3) identify at least two refinements that can be applied when analyzing PDC-HS results. |
| |
| Evaluating the Validity and Reliability of the Performance Diagnostic Checklist - Human Services 1.1 |
| SEBASTIAN JIMENEZ (ABA Technologies) |
| Abstract: The PDC-HS is an informant-based tool used by practitioners in human service settings. A modified PDC-HS 1.1 was used in the present study. The validity and reliability of the PDC-HS 1.1 were measured by analyzing scores obtained while watching simulated interviews between a consultant and supervisor. Three video vignettes were created, each describing a performance concern in one or more areas of the tool. Twenty-one participants watched all vignettes and filled out the tool based on the videos. Validity was measured as the percentage of participants who correctly identified the problematic area(s) in the PDC-HS 1.1. To assess test-retest reliability, participants repeated the assessment about two weeks later. Interrater reliability was measured by pairing participants randomly and comparing scores. Additionally, an intervention-selection component was included to assess whether a corresponding intervention was selected for the indicated domain. Results show about 90% of participants correctly identified the indicated area and 79% selected a relevant intervention. Reliability scores were above 85%, demonstrating that the tool is generally reliable. The results provide support for the use of informant-based assessments in human services settings and suggest that participants with relatively little experience in behavior analysis can conduct assessment interviews accurately and reliably. |
| |
| A Preliminary Investigation of Procedural Refinements to the Performance Diagnostic Checklist - Human Services |
| HANNA E. VANCE (University of Kansas) |
| Abstract: The Performance Diagnostic Checklist – Human Services (PDC-HS) is a functional assessment tool used in the field of Organizational Behavior Management (OBM) to assess the causes of employee performance problems and to inform intervention development. There are two aspects of the intervention-selection process that could inhibit the objectivity and utility of the tool, particularly for practitioners with limited experience conducting the assessment. Proposed refinements to address these drawbacks include (a) identifying a cutoff threshold to objectively identify the categories requiring intervention based on the PDC-HS outcome, and (b) developing decision-making models to identify a single category that should be the focus of intervention. A between-groups design was used to evaluate the degree to which access to components of these proposed refinements resulted in appropriate intervention selections when practitioners in the field were presented with scenarios derived from published case studies. Secondary analyses investigated the impact of certification and experience variables within this process. Our findings suggested that future applications of the PDC-HS, particularly as it relates to the intervention-selection process, may benefit from the supplemented refinements. |
|
| |