All Accounted For?

Each day, approximately 3,500 medical devices are prepared for surgical procedures in accordance with the strictest hygiene and quality standards and delivered to be used by medical personnel in university hospitals. Up to 160 pairs of scissors, forceps, clamps, and other instruments fit on a tray the size of a sheet of A3 paper. Before use, they need to be cleaned, disinfected, packaged, and sterilized in reprocessing units for medical devices. A demanding task, because complaints from the operating room due to missing instruments are a daily affair.

Needle in a haystack

But is the clamp they are looking for really missing, or is the surgical team unable to find it among the many, very similar-looking instruments? Is the complaint justified?

Service companies such as Charité CFM Facility Management GmbH are only able to investigate issues after the fact, when employees manually restack the trays using the packing list to verify whether all the required instruments are present. The trays are then packed, sealed, sterilized, and returned to the users. By scanning unique barcodes at defined scan points, CFM is able to locate surgical trays even after they have left the reprocessing unit. CFM contacted Fraunhofer IPK with the aim of jointly employing the latest AI-based image processing technologies to ensure that the trays are correctly packaged before being sent on their way.

Automated assistants

For this purpose, researchers at Fraunhofer IPK are developing a Decision Support System which uses algorithms to automatically recognize surgical instruments. This consists of a detection system equipped with up to three cameras, a core AI system, and a packing station as a client unit. The core AI system is the processing unit and the heart of the technology. It allows surgical instrument image data to be collected and stored and enables neural networks to be trained based on this image data. This is accomplished by a holistic image processing and decision making approach using Convolutional Neural Networks (CNN). Compared to classical image processing methods, these have the advantage that the AI automatically configures complex parameters and continuously adapts all weights and parameters more precisely to the existing data as part of an automated training process. The overall system is implemented in accordance with the two-man rule as a supplementary testing instance for employees in the packing process and aims to help document the work steps on the packing trays as well as ensure their quality. This aims to reduce the number of complaints regarding incorrectly packed trays.

Get it right from the start

In a feasibility study, Fraunhofer experts have successfully demonstrated that their technologies are suitable for the automated recognition of surgical instruments. The sample comprised 156 different surgical instruments, which were automatically recognized with a Top 1 accuracy of 99.9 percent and a Top 5 accuracy of 100 percent, based on a data set containing a total of 9,672 images. The prototype, which is currently being developed, is expected to be available as a Decision Support System in the reprocessing unit for medical devices at the Charité Campus Benjamin Franklin from the fall of 2021.