Human-Robot Collaboration

Principal Investigator: Professor Lin

AWaRE REU Researcher: Jeremiah Yohannan, University of Oklahoma

Project Description: Robots are expected to work alongside humans in a safe, intelligent, and friendly manner in warehouses, homes, and other robot assistant applications. A key component of such a robotic system is the ability to interpret human intention and operate with knowledge of the surrounding environment. As much of human intention is conveyed through movement, we use a Bayesian non-parametric learning approach to create a human model from motion data. Unlike most human modeling methods, which assume that the number of states is defined, our more flexible method identifies the number of human states directly from the data. We also enable intelligent robotic manipulation of the environment with object classification and localization information generated by a Faster Region-based Convolutional Neural Network. In order to achieve goal-oriented human-robot collaboration and optimize overall task performance, we model sequential tasks as Markov Decision Processes. We demonstrate the effectiveness of our framework in a chair assembly task.

Finding: In this project, the undergraduate student developed algorithms to track human movements, collect data from demonstrations, build human models using Bayesian Non-parametric learning algorithms and build high-level task planning algorithms. The demonstration results confirm that using Bayesian non-parametric learning can help to identify the number of human states directly from the data such as the motion of human hands and it also gives the probabilistic description of each human state (intentions) which ensure robots could infer the human intention correctly and enable the robot to behave collaboratively with human partners. The student also used Faster Region-Based Convolutional Neural Network to classify and localize objects. The whole project shows that the framework we proposed gives a more natural and efficient way to design the communication between human and robots through intention inferencing and high-level decision making.