Knowledge-Based Systems

Proceedings of the Symposium on Mixed- …, 2005

We describe a learning from diagrammatic behavior specifications approach, where the task-performance knowledge of a human expert is transferred to an agent program using abstract behavior scenarios that the expert and the agent program interactively specify. The diagrammatic interface serves as a communication medium between the expert and the agent program to share knowledge during behavior specification. A relational learning by observation component interprets these scenarios in the context of background knowledge and expert annotations to learn first-order rules that represent the task-performance knowledge for an improved agent program.

Frontiers in Robotics and AI

As manufacturing demographics change from mass production to mass customization, advances in human-robot interaction in industries have taken many forms. However, the topic of reducing the programming effort required by an expert using natural modes of communication is still open. To answer this challenge, we propose an approach based on Interactive Reinforcement Learning that learns a complete collaborative assembly process. The learning approach is done in two steps. First step consists of modeling simple tasks that compose the assembly process, using task based formalism. The robotic system then uses these modeled simple tasks and proposes to the user a set of possible actions at each step of the assembly process via a GUI. The user then "interacts" with the robotic system by selecting an option from the given choice. The robot records the action chosen and performs it, progressing the assembly process. Thereby, the user teaches the system which task to perform when. In order to reduce the number of actions proposed, the system considers additional information such as user and robot capabilities and object affordances. These set of action proposals are further reduced by modeling the proposed actions into a goal based hierarchy and by including action prerequisites. The learning framework highlights its ability to learn a complicated human robot collaborative assembly process in a user intuitive fashion. The framework also allows different users to teach different assembly processes to the robot.

Procedia Manufacturing, 2019

Nowadays in the context of Industry 4.0, manufacturing companies are faced by increasing global competition and challenges, which requires them to become more flexible and able to adapt fast to rapid market changes. Advanced robot system is an enabler for achieving greater flexibility and adaptability, however, programming such systems also become increasingly more complex. Thus, new methods for programming robot systems and enabling self-learning capabilities to accommodate the natural variation exhibited in real-world tasks are needed. In this paper, we propose a Reinforcement Learning (RL) enabled robot system, which learns task trajectories from human workers. The presented work demonstrates that with minimal human effort, we can transfer manual manipulation tasks in certain domains to a robot system without the requirement for a complicated hardware system model or tedious and complex programming. Furthermore, the robot is able to build upon the learned concepts from the human expert and improve its performance over time. Initially, Q-learning is applied, which has shown very promising results. Preliminary experiments, from a use case in slaughterhouses, demonstrate the viability of the proposed approach. We conclude that the feasibility and applicability of RL for industrial robots and industrial processes, holds and unseen potential, especially for tasks where natural variation is exhibited in either the product or process.

arXiv (Cornell University), 2020

The success of reinforcement learning for real world robotics has been, in many cases limited to instrumented laboratory scenarios, often requiring arduous human effort and oversight to enable continuous learning. In this work, we discuss the elements that are needed for a robotic learning system that can continually and autonomously improve with data collected in the real world. We propose a particular instantiation of such a system, using dexterous manipulation as our case study. Subsequently, we investigate a number of challenges that come up when learning without instrumentation. In such settings, learning must be feasible without manually designed resets, using only on-board perception, and without hand-engineered reward functions. We propose simple and scalable solutions to these challenges, and then demonstrate the efficacy of our proposed system on a set of dexterous robotic manipulation tasks, providing an in-depth analysis of the challenges associated with this learning paradigm. We demonstrate that our complete system can learn without any human intervention, acquiring a variety of vision-based skills with a real-world three-fingered hand. Results and videos can be found at https://sites.google.com/view/realworld-rl/.

2018

Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks. However, learning with real-world robots is often unreliable and difficult, which resulted in their low adoption in reinforcement learning research. This difficulty is worsened by the lack of guidelines for setting up learning tasks with robots. In this work, we develop a learning task with a UR5 robotic arm to bring to light some key elements of a task setup and study their contributions to the challenges with robots. We find that learning performance can be highly sensitive to the setup, and thus oversights and omissions in setup details can make effective learning, reproducibility, and fair comparison hard. Our study suggests some mitigating steps to help future experimenters avoid difficulties and pitfalls. We show that highly reliable and repeatable experiments can be performed in our setup, indicating the possibility of reinforcement learning ...

Proceedings of the ... AAAI Conference on Artificial Intelligence, 2018

Agents that can learn new tasks through interactive instruction can utilize goal information to search for and learn flexible policies. This approach can be resilient to variations in initial conditions or issues that arise during execution. However, if a task is not easily formulated as achieving a goal or if the agent lacks sufficient domain knowledge for planning, other methods are required. We present a hybrid approach to interactive task learning that can learn both goal-oriented and procedural tasks, and mixtures of the two, from human natural language instruction. We describe this approach, go through two examples of learning tasks, and outline the space of tasks that the system can learn. We show that our approach can learn a variety of goal-oriented and procedural tasks from a single example and is robust to different amounts of domain knowledge.

5th IEEE-RAS International Conference on Humanoid Robots, 2005., 2005

Learning from human demonstration is likely to be one of the key features for service robots in household domains if they are to be accepted by humans. To be of most benefit possible to its user, the robot should go beyond simply imitating a user's demonstration but try to build task knowledge that is as general and flexible as possible.

Cognitive Robotics …

Robot learning is usually done by trial-and-error or learning by example. Neither of these methods takes advantage of prior knowledge or of any ability to reason about actions. We describe two learning systems. In the first, we learn a model of a robot's actions. This is used in ...

IEEE Transactions on Automation Science and Engineering

Several decades of development in the fields of robotics and automation has resulted in human-robot-interaction being commonplace, and the subject of intense study. These interactions are particularly prevalent in manufacturing, where human operators have been employed in a number of robotics and automation tasks. The presence of human operators continues to be a source of uncertainty in such systems, despite the study of human factors, in an attempt to better understand these variations in performance. Concurrent developments in intelligent manufacturing present opportunities for adaptability within robotic control. This work examines relevant human factors and develops a framework for integrating the necessary elements of intelligent control and data processing to provide appropriate adaptability to robotic elements, consequently improving collaborative interaction with human colleagues. A neural network-based learning approach is used to predict the influence on human task performance and use these predictions to make informed changes to programmed behaviour, and a methodology developed to further explore the application of learning techniques to this area. The work is supported by an example case-study, in which a simulation model is used to explore the application of the developed system, and its performance in a real-world production scenario. The simulation results reveal that adaptability can be realised with some relatively simple techniques and models if applied in the right manner and that such adaptability is helpful to tackle the issue of performance disparity in manufacturing operations. NTP: This paper presents research into the application of intelligent methodologies to this problem and builds a framework to describe how this information can be captured, generated and used, within manufacturing production processes. This framework helps identify which areas require further research and serves as a basis for the development of a methodology, by which a control system may enable adaptable behaviour to reduce the impact of human performance variation and improve human-machine-interaction. The paper also presents a simulation-based case study, to support the development and evaluate the presented control system on a representative real-world problem. The methodology makes use of a machine learning approach to identify the complex influence of a number of identified human factors on human performance. This knowledge can be used to adjust the robotic behaviour to match the predicted performance of a number of different operators over a number of scenarios. The adaptability reduces performance disparity, reducing idle times and enabling leaner production through WIP reduction. Future work