Inside Behavior Analysis
Volume 1 | 2009 | Number 1
Behavior Analysis and Selectionist Approaches to Robotics
By Joseph Cautilli
Clearly this is the Special Interest Group of the future. It expands the base of ABAI into the area of research and development and to artificial systems. However, none of the core topics of reinforcement learning, neural networks, stacked neural networks, matching models of reinforcement, motor learning, building of relational frames, acquisition of communication skills, and personality in robots from behavior analytic models is new to ABAI. Indeed, in many ways it is surprising that ABAI has not had a special interest group devoted to this subject until now.
The Robotic’s SIG is focused on how behavior analysis can improve research and development in building intelligent machines. The SIG is uniquely focused on research and development in helping robots learn by consequences. Over the years focus on reinforcement learning has permeated the field of robots (i.e., Boyan, & Moore, 1995; Mahadevan, & Connell, 1992; Mataric, 1997; Sutton, & Barto, 1998; Sutton, Precup, & Singh, 1999). Work in this area continues on formulating reinforcement learning for noisy and dynamic domains. In addition, robots offer a new way to assess how basic principles of learning combine to select complex behavior (e.g., Hutchison, 1998; Stephens and Hutchison, 1992). This is particularly important in the study of phenomena that may have ethical issues in researching. For example, in the language area it would be unethical to manipulate the amount of language input or feedback a child receives, but this would not be a problem for robotics. Finally, robotics can be used to test theories of underlying structures (e.g. Daw & Touretzky, 2000).
The SIG sees its role as making applied behavior analysis more friendly and compatible for research and development in these areas. The SIG has a listserve (http://tech.groups.yahoo.com/group/behavioranalysisandrobotics) and all are welcome to join. The SIG is also engaged in activities to continue to bring researchers from all areas, including engineering, to ABAI to discuss recent advances and to share advances in the basic process of human learning for robotics researchers to incorporate.
References
Boyan, J.A. & Moore, A.W. (1995). Generalization in reinforcement learning: Safely approximating the value function. In G. Tesauro, D. S. Touretzky, and T. Leen,( Eds.) Advances in Neural Information Processing Systems, 7, (pp. 369-376), MIT Press.
Commons, M.L. (2008). Stacked neural networks must emulate evolutions hierarchal complexity. World Futures, 64, 444–451
Daw, N.D., & Touretzky, D.S. (2000) Behavioral considerations suggest an average reward TD model of the dopamine system, Neurocomputing 32, 679-684.
Hutchison, W.R. (1998) Computer simulations of verbal behavior for research and persuasion. The Analysis of Verbal Behavior, 15, 117-120
Mahadevan, S. and Connell, J. (June, 1992) Automatic programming of behavior-based robots using reinforcement learning Machine Learning, vol. 55( 2–3), 311–365
Mataric, M. (1997). Reinforcement learning in the multi-robot domain. Autonomous Robots 4(1):73–83
McDowell, J.J., Caron, M.L., Kulubekova, S. & Berg, J.P. (2008). A computational theory of selection by consequence applied to concurrent schedules. Journal of the Experimental Analysis of Behavior, 90, 387–403
Stephens, K.R. and Hutchison, W.R. (1992). Behavioral personal digital assistants: the seventh general of computing. The Analysis of Verbal Behavior, 10: 149-156
Sutton, R.S. and Barto, A.G. (1998) Reinforcement Learning: An introduction. Cambridge, MA: MIT Press.
Sutton, R. Precup, D. & Singh, S.P. (1999). Between MDP’s and semi-MDP’s: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1-2):181–211