Robot Cognition: See, Hear, Feel, Do

The use of sensors and AI are expanding the application areas of robots beyond just industry. Our team develops innovative automation approaches and evaluates where they can be economically viable.

Technological progress makes automation conceivable in areas that go far beyond the current field of application for robotics. Imagine robots making your hotel bed, harvesting cucumbers or stocking shelves in the supermarket. What all of these scenarios have in common is that they involve tasks that cannot be programmed down to the millimeter, because the manipulated objects are not in a fixed known position. Experts refer to such tasks as automation in partially or even unstructured environments.

In order for robots to master such tasks, a certain form of intelligence is required: The machine must be able to perceive its surroundings, make decisions, and plan freely. »The aim is to simply be able to say: ›Grasp the object in a secure way and move it to B‹,« says Dr. Gregor Thiele, head of the Process Automation and Robotics department. What sounds trivial is actually a highly complex field of work that brings together cognitive robotics and advanced actuator technology. 

Cognitive robotics allows robots to perceive and understand their surroundings. Image sensor technology enables the identification of workpieces, targets and obstacles. Force sensors are an important addition, as tactile feeling optimally complements visual perception. Acoustic signals can also be relevant, for example to evaluate the sound of objects locking into place. »The processing of signals like these has made huge leaps forward in recent years thanks to advances in machine learning and AI,« says Oliver Heimann, head of the Machine Vision department.

However, there is still a lot of work to be done when it comes to making the transmitted signals usable: Combining them into an overall understanding and making the right decisions is time-consuming; deriving actions that meet the requirements of the application in terms of speed and robustness even more so. And, »if a process looks slightly different each time, it gets more difficult to assess whether it is running correctly,« summarizes Thiele. 

All of this currently makes it difficult for users to commit to automation. »This is where our automation assessment provides support,« explains Arturo Bastidas-Cruz, who developed the method. »We use a systematic approach to assess which processes could be considered for automation.« The assessment does not just deliver a hard ›yes‹ or ›no‹ as to whether automation makes technical and economic sense. It can also end up recommending to automate sub-processes, to attempt a human-robot collaboration or to set up entire processes differently so that robots can manage them more easily. 

Heimann and Thiele’s teams also go beyond recommendations: Their departments of around 60 in-house experts build prototypical implementations to show that a solution is feasible. They combine suitable hardware with the appropriate signal processing solution, thus minimizing the decision-making risk for companies that use automation solutions as well as integrators who want to implement them.