|”To Replicate or... Keep Replicating: Explorations on Some Replication Efforts and Risk of Bias in Behavior Analysis”|
|Monday, May 29, 2023|
|4:00 PM–4:50 PM |
|Hyatt Regency, Centennial Ballroom H|
|Area: PCH/EAB; Domain: Theory|
|Chair: Camilo Hurtado-Parrado (Southern Illinois University; Konrad Lorenz University)|
|Discussant: Eric A. Jacobs (Southern Illinois University Carbondale)|
|CE Instructor: Eric A. Jacobs, Ph.D.|
Replication of experimental studies is a tenet of advancement and knowledge generation in natural sciences. Behavior analysis incorporated and promulgated this notion since its inception. Replication was evident in the progression of B. F. Skinner’s early experiments and the subsequent work of those experimental and applied behavior analysts who followed. Sidman (1960) later formalized these practices when defined direct and systematic replication and discussed their importance for our field. Direct replication establishes reliability of phenomena via repeated reproduction of experimental effects, and systematic replication allows to determine the generality of variables, methods, and processes across species, subjects, responses, or settings (Johnston, 1979). The recent debate on the so called “replication crisis” in psychology, triggered discussions on the replication practices in behavior analysis (e.g., Hantula, 2019; Perone, 2019; Tincani & Travers, 2019). Behavior analytic research overall seems to be immune to the replication and reproduction problems that have been identified in other fields. However, some authors (e.g., Tincani & Travers, 2019) have argued that there is still a need of explicit and systematic efforts to evaluate the reproducibility of our findings and to assess the degree of publication bias in our field. One of the reasons is that behavior analysis may still be vulnerable to the contingencies that led to the “replication crisis” in other disciplines (e.g., disproportionate publication of studies showing only strong experimental effects may inflate confidence in intervention; Tincani & Travers, 2019). This symposium aims to contribute to these gaps. One presentation discusses the processes, outcomes, and challenges of systematic replication efforts of conditional discrimination and aversive control phenomena. A second presentation will examine preliminary findings of a study on risk of bias in behavior analytic research.
|Instruction Level: Intermediate|
|Keyword(s): choice, conditional discrimination, replication, risk-of-bias|
|Target Audience: |
Intermediate. Audience should have some familiarity with the notions of direct and systematic replication, conditional discrimination, aversive control (negative reinforcement and punishment), and quality assessment in research.
|Learning Objectives: (1) Define matching-to-sample procedures and conditional discrimination. (2) Differentiate aversive contingencies in the context of conditional discrimination. (3) Define risk of bias in the context of behavior analysis.|
|A Behavior Analytic Account of Risk of Bias: A Preliminary Investigation|
|MANISH K. GOYAL (Southern Illinois University), Iulian-Alexandru Iulian Stefan (University of Bucharest), Lesley A. Shawler (Southern Illinois University)|
|Abstract: • Many systematic reviews include quality indicator tools (e.g., What Works ClearingHouse), which assess the rigor of specific experimental designs and methodology. Less common is an assessment of risk of bias. Risk of bias (ROB) is related to, but distinct from, methodological quality, as it relates to errors or deviations from the truth within results (Higgins & Altman, 2008). Previously, a ROB tool was developed by Reichow et al. (2018) to evaluate single-case experimental designs within research. This tool assesses eight categories of bias. The purpose of this review was to apply the ROB tool to an existing systematic review (Muharib et al., 2021) to determine its utility and reliability across raters. Two raters scored 100% of 35 articles and exact IOA was calculated with an overall agreement of 76%. The highest areas of bias were the absence of blinding of participants and personnel and insufficient procedural fidelity across all studies. ROB due to data sampling and participant selection were relatively low. We provide general recommendations for researchers to help mitigate potential areas of bias within future research.|
Effects of Different Contingencies in the Acquisition of Aversive Conditional Discriminations
|JULIAN CIFUENTES (Southern Illinois University), Monica Arias Higuera (Universidad Nacional de Colombia), Lucia Medina (University of Missouri), Camilo Hurtado-Parrado (Southern Illinois University; Konrad Lorenz University)|
Training of arbitrary conditional discriminations in verbal individuals often employs matching-to-sample (MTS) tasks that entail a combination of positive reinforcement and punishment using verbal stimuli (e.g., punishment and reinforcement for incorrect and correct matching responses via written stimuli – “correct” and “incorrect”). Little is known about arbitrary aversive conditional discrimination and the effects that different configurations of contingencies may have on those discriminations. In Experiment 1, we systematically replicated a widely implemented arrangement of consequences in MTS procedures in the context of participants’ matching of trigrams (arbitrary comparison stimuli) to aversive images (sample stimuli, e.g., pictures of human mutilation from the International Affective Picture System – IAPS) – i.e., replicated Steele and Hayes’s (1991) procedure in which correct responses produced a “correct” written stimulus and incorrect responses an “incorrect” written stimulus. This traditional arrangement of consequences for conditional discrimination was compared to other three contingencies that did not entail verbal stimuli: (a) positive reinforcement only (a progress bar only increased with each correct matching response), positive and negative reinforcement (correct matching produced an increase in the progress bar and the sample aversive image was removed from the screen – i.e., escape – and consistent correct matching of a given sample prevented it to continue being presented in next trials – i.e., avoidance), and negative reinforcement alone (correct matching produced escape and avoidance of the sample aversive stimuli). The traditional arrangement of consequences for matching (written “correct” or “incorrect” words) produced more consistent, faster, and higher percentages of correct responses across most participants, as compared to the other contingencies. In experiment 2, we systematically replicated the aversive MTS task developed for Experiment 1 using only the negative reinforcement contingency. We aimed to gather further evidence of conditional-discrimination acquisition using only escape and avoidance contingencies and test for differences in conditional-discrimination performance depending on the social relevance of the sample images. We used the same aversive IAPS images from Experiment 1 and compared their effect against aversive images from the Colombian armed conflict (pictures depicting massacres, kidnapping, terrorist attacks, etc.; Hurtado-Parrado et al., 2020). In addition, we used as control sample stimuli appetitive and neutral images from the IAPS (e.g., pictures depicting positive social interactions or house objects, respectively). The findings of Experiment 1 were overall reproduced with IAPS aversive and neutral images during Experiment 2, but not with images of the Colombian armed conflict. Results of both experiments indicate that conditional discrimination phenomena could be reproduced with aversive contingencies that do not entail verbal stimuli as consequences. However, the traditional and widely implemented combination of reinforcement and punishment using verbal stimuli is more effective in reproducing such discriminations in verbal individuals. Recent replications of our aversive MTS task have provided further evidence (e.g., Acevedo-Triana et al., 2021).