|
| Operationalize This: Rethinking Operational Definitions |
| Monday, May 27, 2024 |
| 8:00 AM–8:50 AM |
| Marriott Downtown, Level 5, Grand Ballroom Salon AB |
| Area: TBA/DDA; Domain: Applied Research |
| Chair: Mey-Elle Naveh (Kinneret Academic College) |
| CE Instructor: Avner Fraidlin, Ph.D. |
| Abstract: Operational definitions are a key element in behavioral interventions and the dissemination of our science depends on the consistency and accessibility of researchers’ operational definitions to others, who can adapt and use them with different client populations (Cooper et al., 2020). Although recommendations for developing operational definitions are frequently referenced in behavior analytic literature and textbooks (Johnston et al., 2020), there is no agreed-upon method for developing operational definitions (Kubina et al., 2022). The three presentations in this symposium raise questions concerning the validity of operational definition, discuss issues related to subjectivity and ambiguity of operational definitions and address the implications of these issues to our field. The first presentation will share findings of a survey study in which associations between characteristics of behavior analysts and operational definitions they developed for video recorded behaviors were investigated. The second presentation will share findings of an investigation of the effect varying operational definition components had on interobserver agreement. The third presentation will present findings of a comparison of operational definitions’ and pinpoints’ detection accuracy. Together, the presentations question the utility of operational definitions and highlight the need for empirical investigation of the mechanism responsible for developing them. |
| Instruction Level: Basic |
| Keyword(s): Detection-accuracy, Interobserver agreement, Operational-definitions, Pinpoints |
| Target Audience: The symposium is geared toward behavior analytic practitioners, researchers, and course instructors. |
| Learning Objectives: At the conclusion of the presentation, participants will be able to: (1) describe issues concerning operational definitions validity and utility, (2) describe considerations for practitioners and researchers when selecting operational definitions and pinpoints, and (3) describe the benefits of using pinpoints. |
| |
Variations in Behavior Analysts’ Operational Definitions |
| Avner Fraidlin (Kinneret College, David Yellin College, Western Michigan University, ), Madeline Halkowski (Penn State), Jessica E. Van Stratton (Western Michigan University), RICK KUBINA (Penn State) |
| Abstract: The effectiveness of behavior analytic interventions and scientific investigations rest on the validity and accuracy with which practitioners and researchers operationalize behaviors. Although recommendations for developing operational definitions are often referenced in behavior analytic literature and textbooks (Johnston et al., 2020), a critical examination of issues with the process of operationalization has received relatively little attention (Slife et al., 2016). The current study investigated how behavior analysts operationalize behaviors. A survey which was disseminated across behavior analysts residing in the US and Canada and registered on the BACB© listserv, requested respondents to develop operational definitions for two behaviors displayed in short video recordings. Descriptive statistics found differences in how respondents used components of operational definitions (e.g., a greater proportion of respondents that reported holding a teaching position used more examples than those who did not report holding a teaching position). However, statistically significant associations between respondent characteristics and use of different components of operational definitions were limited. Findings suggest operational definitions were insensitive to various characteristics (e.g., certification level, years of experience, client population) of behavior analysts. Implications for behavior analysts, recommendations for future investigations, and limitations are discussed. |
| |
Toward Operationalizing Operational Definitions: A Comparison of Definition Components |
| JESSICA A GARMAN (Hopebridge), Tyler-Curtis Cory Elliott (University of Georgia), Kevin Ayres (University of Georgia) |
| Abstract: Behavior analytic practices start with objectively defining behavior. Yet, with so many recommendations for what an operational definition contains there is no “best practice” for defining behavior, only that it should ensure good agreement between data collectors. Not only do disparities in what makes an operational behavior definition exist in the literature, but these disparities are also apparent in practice. Current behavior analytic practitioners use a variety of components to define observable behavior (e.g., examples, nonexamples, constraints, directions). The current study evaluated the effect of these varying components of operational definitions on interobserver agreement with a true value using practical and statistical analyses as well as a measure of social validity. For hitting, we found no statistical difference of definition type, but we did find a difference for the order of coding. For on task, we found no statistical difference for definition type or order of coding. Implications are further discussed in relation to the role of definition components and recommendations for practitioners looking to achieve acceptable interobserver agreement. |
| |
Comparing the Detection Accuracy of Operational Definitions and Pinpoints |
| Madeline Halkowski (Penn State), RICK KUBINA (Penn State) |
| Abstract: Submissions must include a description of the type of funding received for this submission, if applicable: Operational definitions have a significant history in applied behavior analysis. The practice's importance stems from the role operational definitions play in detecting an event, human thought, or action. While operationalizing target behaviors has enjoyed widespread practice, some concerns have recently arisen with translation validity and detection accuracy. Additionally, a review of the literature produces few articles assessing the validity of operational definitions. Pinpoints represent an alternative for describing target behaviors. A pinpoint has a formula for construction that includes using an action verb, an object or event that receives the action, and a comprehensibly defined context where the observation of the action verb + object or event occurs. Pinpoints also have few empirical studies demonstrating their validity. The following experiment compared the detection accuracy of an operational definition for self-injurious behavior and a corresponding pinpoint across professionals who worked in a school that served clients with autism spectrum disorder. The results indicate lower accuracy scores for the operational definition when compared to the pinpoint. Additionally, the consistency of scores varied more for the operational definition than the pinpoint. |
|
| |