Fraunhofer Institute for Production Systems and Design Technology
Point of View: Robot
The robot's view of the world: The red dots represent the perception of the two laser scanners, the white geometric shapes are parts of an experimental environment as seen by the camera on the robotic arm.
The desire to use plants flexibly to manufacture different products is increasingly pushing rigid automation solutions to their limits. This is giving rise to new requirements: On the one hand, robot systems need to be able to handle variations in the shape and position of components on their own. On the other hand, the material flow within a factory must also be able to respond more dynamically to orders. Chains of conveyor belts get in the way of rapid changes in process and production layout. They are being replaced by autonomous vehicles, which ideally enable order-specific interlinking of the systems.
Adapting to one another
Fraunhofer IPK researchers developed the Tend-O-Bot to meet these needs: The system combines an autonomous vehicle and a built-in robotic arm. In combination with image processing algorithms and path planning, a system is created that can interact with and load machines independently. As apt as this solution might seem, however, it places significant demands on the sensory and cognitive system abilities.
Environments where humans and robots work together are highly complex. To work successfully in these surroundings, a robot needs to learn to adapt to the dynamic human environment: For example, after completing a task, a human might find it more convenient to set a tool to one side than to return it to its designated storage place. Misplaced objects like this – or even the humans themselves – could be obstacles for a robot. Without »eyes« or a »brain«, it could collide with them or grab at empty space. The assembly process also poses new challenges for the robot. Like humans, robots need to be more sensitive and dexterous, if they are to assemble complex modules reliably.
Broadening a robot’s abilities in this way requires complex algorithms that make use of models and sensor data to send appropriate instructions to the robot using various strategies. To understand this complex interplay, the developer first has to have an intuition about the robot’s view of the world. The image at the top of this page shows insights into the world as the robot perceives it. In combination with simulations of the environment, the systems developed as part of the Tend-O-Bot project make it possible to try out complex tasks in a digital environment and to accelerate integration into the actual process.
Up in the cloud
As shown by the visualization of the data, the algorithms build on a range of complex data. The vehicle supplies information about the environment through two laser scanners, while a camera atop the robot provides a depth map and another camera on the vehicle provides a 2D image in the direction of travel. All this information is evaluated so that the robot can draw conclusions about the situation. This bandwidth of algorithms requires powerful computer hardware. The autonomous and mobile nature of the system limits space and battery capacity, however, and the computing operations are therefore executed in the cloud. The system has a fully modular design, meaning that individual applications and algorithms can be shifted around the network more or less at will.
The sensor data are transferred by a powerful wireless connection to the external servers where the data are evaluated, the robot’s understanding of the world is updated and new commands are then transferred to the hardware.
The Tend-O-Bot system has the ability to respond to new situations independently and rapidly, a key component of any flexible human-robot collaboration (HRC). Due to the modular design, individual image processing and planning algorithms can even be used separately from each other.
The systems are linked at the digital intermediate level so that new sensors and algorithms can be integrated easily into the existing solutions. This means that customer- specific ideas can be implemented quickly. The system also has the capability to use multiple vehicles to network processes together flexibly in a matrix composite. Thanks to the integrated manipulator, the system can also interact with machines that are difficult to use with conventional transport systems. In such cases, the Tend- O-Bot intervenes and loads the machine by itself.
In the longer term, the algorithms will also allow robotics to break out of the industrial domain: In the future, they could be used in retail as well as in agriculture and construction. Unlike in industrial HRC environments, which are at least partially structured, these environments place even greater demands on the cognitive abilities of robots.