Robot Shaping: An Experiment In Behavior Engineering

Proceeding of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction - HRI '06, 2006

Programming robots to carry out useful tasks is both a complex and non-trivial exercise. A simple and intuitive method to allow humans to train and shape robot behaviour is clearly a key goal in making this task easier. This paper describes an approach to this problem based on studies of social animals where two teaching strategies are applied to allow a human teacher to train a robot by moulding its actions within a carefully scaffolded environment. Within these enviroments sets of competences can be built by building state/action memory maps of the robot's interaction within that environment. These memory maps are then polled using a k-nearest neighbour based algorithm to provide a generalised competence. We take a novel approach in building the memory models by allowing the human teacher to construct them in a hierarchical manner. This mechanism allows a human trainer to build and extend an action-selection mechanism into which new skills can be added to the robot's repertoire of existing competencies. These techniques are implemented on physical Khepera miniature robots and validated on a variety of tasks.

2015

Learning takes place when the system makes changes to its in-ternal structure so as to improve some metric on its long-term future performance, as measured by a xed standard [3]. Let us take \learning " to mean, roughly, the improvement of a system's behavior by making it more appropriate for the environ-ment in which it is embedded [2]. 1

Cognitive Robotics …

Robot learning is usually done by trial-and-error or learning by example. Neither of these methods takes advantage of prior knowledge or of any ability to reason about actions. We describe two learning systems. In the first, we learn a model of a robot's actions. This is used in ...

This report documents the programme and the outcomes of Dagstuhl Seminar 14081 "Robots Learning from Experiences". The report begins with a summary comprising information about the seminar topics, the programme, important discussion points, and conclusions. The main body of the report consists of the abstracts of 25 presentations given at the seminar, and of four reports about discussion groups. Seminar February 17-21, 2014-http://www.dagstuhl.de/14081 1998 ACM Subject Classification I.2.6 Concept Learning

2014

This report documents the programme and the outcomes of Dagstuhl Seminar 14081 “Robots Learning from Experiences”. The report begins with a summary comprising information about the seminar topics, the programme, important discussion points, and conclusions. The main body of the report consists of the abstracts of 25 presentations given at the seminar, and of four reports about discussion groups.

Future application areas for humanoid robots range from the household, to agriculture, to the military, and to the exploration of space. Service applications such as these must address a changing, unstructured environment, a collaboration with human clients, and the integration of manual dexterity and mobility. Control frameworks for service-oriented humanoid robots must, therefore, accommodate many independently challenging issues including: techniques for configuring networks of sensorimotor resources; modeling tasks and constructing behavior in partially observable environments; and integrated control paradigms for mobile manipulators. Our approach advocates actively gathering salient information, modeling the environment, reasoning about solutions to new problems, and coordinating ad hoc interactions between multiple degrees of freedom to do mechanical work. Representations that encode control knowledge are a primary concern. Individual robots must exploit declarative structure for planning and must learn procedural strategies that work in recognizable contexts. We present several pieces of an overall framework in which a robot learns situated policies for control that exploit existing control knowledge and extend its scope. Several examples drawn from the research agenda at the Laboratory for Perceptual Robotics are used to illustrate the ideas.

Journal of Physical Agents (JoPha), 2012

This article describes a proposal to achieve fast robot learning from its interaction with the environment. Our proposal will be suitable for continuous learning procedures as it tries to limit the instability that appears every time the robot encounters a new situation it had not seen before. On the other hand, the user will not have to establish a degree of exploration (usual in reinforcement learning) and that would prevent continual learning procedures. Our proposal will use an ensemble of learners able to combine dynamic programming and reinforcement learning to predict when a robot will make a mistake. This information will be used to dynamically evolve a set of control policies that determine the robot actions.

IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), 2000

In this paper, we present a strategy whereby a robot acquires the capability to learn by imitation following a developmental pathway consisting on three levels: 1) sensory-motor coordination; 2) world interaction; and 3) imitation. With these stages, the system is able to learn tasks by imitating human demonstrators. We describe results of the different developmental stages, involving perceptual and motor skills, implemented in our humanoid robot, Baltazar. At each stage, the system's attention is drawn toward different entities: its own body and, later on, objects and people. Our main contributions are the general architecture and the implementation of all the necessary modules until imitation capabilities are eventually acquired by the robot. Also, several other contributions are made at each level: learning of sensory-motor maps for redundant robots, a novel method for learning how to grasp objects, and a framework for learning task description from observation for program-level imitation. Finally, vision is used extensively as the sole sensing modality (sometimes in a simplified setting) avoiding the need for special data-acquisition hardware.

2008

Abstract We describe a learning algorithm that generates behaviors by self-organization of sensorimotor loops in an autonomous robot. The behavior of the robot is analyzed by a multi-expert architecture, where a number of controllers compete for the data from the physical robot. Each expert stabilizes the representation of the acquired sensorimotor mapping in dependence of the achieved prediction error and forms eventually a behavioral primitive.

2007

Lecture Notes in Computer Science, 2008

One of the most important characteristics of intelligent activity is the ability to change behaviour according to many forms of feedback. Through learning an agent can interact with its environment to improve its performance over time. However, most of the techniques known that involves learning are time expensive, i.e., once the agent is supposed to learn over time by experimentation, the task has to be executed many times. Hence, high fidelity simulators can save a lot of time. In this context, this paper describes the framework designed to allow a team of real RoboNova-I humanoids robots to be simulated under USARSim environment. Details about the complete process of modeling and programming the robot are given, as well as the learning methodology proposed to improve robot's performance. Due to the use of a high fidelity model, the learning algorithms can be widely explored in simulation before adapted to real robots.

2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL), 2013

A novel way to model an agent interacting with an environment is introduced, called an Enactive Markov Decision Process (EMDP). An EMDP keeps perception and action embedded within sensorimotor schemes rather than dissociated. Instead of seeking a goal associated with a reward, as in reinforcement learning, an EMDP agent is driven by two forms of self-motivation: successfully enacting sequences of interactions (autotelic motivation), and preferably enacting interactions that have predefined positive values (interactional motivation). An EMDP learning algorithm is presented. Results show that the agent develops a rudimentary form of self-programming, along with active perception as it learns to master the sensorimotor contingencies afforded by its coupling with the environment.

2005

This paper presents a new learning approach for autonomous robots. Our system will learn simultaneously the perception-the set of states relevant to the task-and the action to execute on each state for the task-robotenvironment triad. The objective is to solve two problems that are found when learning new tasks with robots: interpretability of the learning process and number of parameters; and the complex design of the state space. The former was solved using a new reinforcement learning algorithm that tries to maximize the time before failure in order to obtain a control policy suitable to the desired behavior. The state representation will be created dynamically, starting with an empty state space and adding new states as the robot finds them, this makes unnecessary the creation of a predefined state representation, which is a tedious task. 2 ACTION LEARNING Sutton and Barto developed reinforcement learning as a machine learning paradigm that determines how an agent ought to take actions in an environment so as to maximise some notion of long-term reward (Sutton and Barto, 1998). Reinforcement learning is a very interesting strategy, since all the robot needs for learning a behaviour

IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, 2004

Behaviour co-ordination is one of the major problems in behaviour-based robotics. This paper presents a teaching method for mobile robots to learn the behaviour coordination. In this method, the sensory information is abstracted into a limited number of feature states that correspond to physical events in the interactive process between a robot and its environment. The continuous motor actions are abstracted into a limited number of behaviours. Then, the goal of the behaviour co-ordination is to map the feature states into the behaviours in the light of environment rewards. The teaching process consists of an imitation stage and an autonomous learning stage. Both stages employ Q-learning algorithms to implement the mapping. The imitation stage serves as a preliminary stage for the teaching method. The learning result will be used to bootstrap the autonomous learning stage. Experiments are conducted in the domain of soccer playing of Sony legged robots. Experiment results show that the robot can acquire the behaviour coordination ability.