|
| Applications of Signal Detection Theory to Visual Analysis of Functional Analyses |
| Sunday, May 29, 2022 |
| 12:00 PM–12:50 PM |
| Meeting Level 1; Room 156B |
| Area: PCH/AUT; Domain: Translational |
| Chair: Lisa Tereshko (Endicott College) |
| Discussant: David J. Cox (Behavioral Health Center of Excellence; Endicott College) |
| CE Instructor: Allison Rader, Ph.D. |
| Abstract: Using statistics within behavior analysis is a topic that has been visited and revisited but has yet to become a regular part of practice in the applied setting. In some circles, it remains a topic of contention, especially when suggested that statistics be used in lieu of visual analysis to interpret behavioral data. We agree with others who have proposed that statistics and other quantitative judgement tools can be used in conjunction with visual analysis. Application of statistics might be of special interest when early detection of functional relations is critical. Such is the case when making decisions regarding results of functional analyses. Therefore, the purpose of the current investigation was to explore (1) to what extent PhD level behavior analysts agree with each other on visual analysis of FA graphs (2) and to determine the effect of teaching practicing and novice behavior analysts to apply statistical analysis to the evaluation of FA graphs. We then propose some directions for further investigation that may promote best practices in the future. |
| Instruction Level: Advanced |
| Target Audience: Basic and applied researchers, practitioners |
| Learning Objectives: At the conclusion of the presentation, participants will be able to: (1) identify shortcomings of visual analysis; (2) list at least one advantage of a signal detection theory approach to evaluating reliability, accuracy or bias; (3) name one quantitative method that may be used to supplement visual analysis. |
| |
| A Quantitative Analysis of Accuracy, Reliability and Bias in Judgements of Functional Analyses |
| (Basic Research) |
| Allison Rader (The May Institute), MICHAEL YOUNG (Kansas State University), Justin B. Leaf (Autism Partnership Foundation) |
| Abstract: Functional analysis can be considered a diagnostic assessment that behavior analysts use to determine behavioral function. Such a diagnosis ultimately requires a yes or no decision (i.e., a variable maintains a behavior, or it does not) that is determined by both subjective (clinical judgement) and objective (data) variables. Accurate and reliable identification of function is essential for successful treatment, yet behavior analysts’ interpretation of data relies on their ability to detect visual differences in graphed data. Some research indicates that behavior analysts have questionable reliability in their visual analysis. To further examine the reliability, accuracy, and bias in visual analysis of functional analysis graphs, we simulated functional analysis results and surveyed 121 BCBA-Ds experienced in visual analysis. We then examined reliability of responses and used a signal detection theory approach to analyze accuracy and bias. Findings suggest that reliability and accuracy of judgements are questionable, and exploration of decision aids is warranted. |
| |
| Use of Confidence Intervals to Supplemental Visual Analysis in Interpretation of Functional Analyses |
| (Applied Research) |
| ALLISON RADER (The May Institute), Michael Young (Kansas State University), Mary Jane Weiss (Endicott College), Justin B. Leaf (Autism Partnership Foundation) |
| Abstract: Use of statistics within behavior analysis has been visited and revisited but has yet to become a regular part of practice in applied settings. In some circles, the topic is contentious, especially when it is suggested that statistics be used in lieu of visual analysis. We agree with others who have proposed that statistics and other quantitative judgement tools can be used in conjunction with visual analysis. This application might be of special interest when early detection of causal relationships is critical. Therefore, the purpose of this investigation was to determine the effect of teaching practitioners to integrate confidence intervals into their evaluation of functional analyses. We then propose some directions for further investigation that may promote future best practices. |
|
| |