April 14, 2022

By Silke von Gemmingen, IDS (Imaging Development Systems)

EnsensoLaundry400x275The textile and garment industry is facing major challenges with current supply chain and energy issues. The future recovery is also threatened by factors that hinder production, such as labor and equipment shortages, which put them under additional pressure. The competitiveness of the industry, especially in a global context, depends on how affected companies respond to these framework conditions. 

One solution is to move the production of clothing back to Europe in an economically viable way. Shorter transport routes and the associated significant savings in transport costs and greenhouse gasses speak in favor of this. On the other hand, the related higher wage costs and the prevailing shortage of skilled workers in this country must be compensated. The latter requires further automation of textile processing. German deep-tech start-up Sewts GmbH from Munich has focused on the great potential that lies in this task. It develops solutions with the help of which robots – similar to humans – anticipate how a textile will behave and adapt their movement accordingly.

In the first step, Sewts has set its sights on an application for large industrial laundries. With a system that uses both 2D and 3D cameras from IDS Imaging Development Systems GmbH, the young entrepreneurs are automating one of the last remaining manual steps in large-scale industrial laundries, the unfolding process. Although 90% of the process steps in industrial washing are already automated, the remaining manual operations account for 30% of labor costs. The potential savings through automation are therefore enormous.

Application

Industrial laundries already operate in a highly automated environment to handle the large volumes of laundry. Among other things, the folding of laundry is done by machines. But each of these machines usually requires an employee to manually spread out the laundry and feed it without creases. This monotonous and strenuous loading of the folding machines has a disproportionate effect on personnel costs. In addition, qualified staff is difficult to find, which often impacts the capacity utilization and the profitability of industrial laundries. The seasonal nature of the business also requires a high degree of flexibility. Sewts makes IDS cameras the image processing components of a new type of intelligent system, whose technology can now be used to automate individual steps, such as sorting dirty textiles or inserting laundry into folding machines.

"The particular challenge here is the malleability of the textiles," said Tim Doerks, co-founder and CTO of Sewts. While the automation of the processing of solid materials, such as metals, is comparatively unproblematic with the help of robotics and AI solutions, available software solutions and conventional image processing often still have their limits when it comes to easily deformable materials. Accordingly, commercially available robots and gripping systems have so far only been able to perform such simple operations as gripping a towel or piece of clothing inadequately. 

But the Sewts system VELUM can provide this. With the help of intelligent software and easy-to-integrate IDS cameras, it can analyze dimensionally unstable materials such as textiles. Thanks to the new technology, robots can predict the behavior of these materials during gripping in real time. It empowers VELUM to feed towels and similar linen made of terry cloth easily and crease-free into existing folding machines, thus closing a cost-sensitive automation gap.

IDS CaseStudy800px Robot

The software suite developed by Sewts combines commercially available robots, grippers and cameras into an intelligent system. In the search for the right camera modules, several criteria were decisive for the Munich-based company in addition to uncompromising industrial suitability: "We need a 3D camera that is cost-effective because we use two to three 3D cameras depending on the system configuration. In addition, it must above all ensure high accuracy of the depth data," said Tim Doerks.

"Beyond that, we need 2D cameras that are light sensitive, deliver high dynamic range and are suitable for use in a multi-camera system." The founders found what they were looking for in the IDS portfolio: For the VELUM multi-camera system, the choice fell on the new Ensenso S10 3D camera as well as models from the uEye CP camera series. Their task is to identify, both in 2D and 3D, interesting features and gripping points of the textiles that are fed into the system after washing and drying in an unordered manner in a container or on a conveyor belt. The shape and position of the individual objects cannot be predicted. The cameras capture the different textures of the materials. They distinguish which hems there are on a towel and where corners are.

“We match the images from the 2D and 3D cameras to have a higher 2D resolution together with the 3D data,” said Doerks. “So we use the respective advantages of the 2D camera, in this case the higher resolution, and the 3D camera, i.e. the precise depth data.”

Equipped with a 1.6 megapixel Sony sensor, the Ensenso S10 uses a 3D process based on structured light: A narrow-band infrared laser projector produces a high-contrast dot pattern even on objects with difficult surfaces or in dimly lit environments. Each image captured by the 1.6 megapixel Sony sensor provides a complete point cloud with up to 85,000 depth points. Artificial intelligence enables reliable assignment of the laser points found to the hard-coded positions of the projection. This results in the robust 3D data with the necessary depth accuracy, from which VELUM extracts the coordinates for the gripping points.

The complementary GV-5280CP-C-HQ industrial camera with GigE Vision firmware is equipped with Sony's IMX264 ⅔-inch global shutter CMOS sensor. It delivers near-noise-free, high-contrast 5 megapixel images in 5:4 format at 22 fps at full GigE speed in applications with fluctuating lighting conditions. The uEye CP camera offers maximum functionality with extensive pixel pre-processing and is perfect for multi-camera systems thanks to the internal 120 MB image memory for buffering image sequences. At around 50 g, the small magnesium housing is as light as it is robust and predestines the camera for space-critical applications and for use on robot arms.

Software

Depending on customer requirements or configuration, two to three uEye 2D or Ensenso 3D cameras are used – both models can be integrated into VELUM. "We are experts in preparing the generated data, which is especially important when working with 3D point clouds,” said Doerks. “This preprocessing is an important component of our systems in order to generate suitable input for our artificial intelligence." 

The AI developed by Sewts processes the data supplied by the uEye CP or Ensenso S cameras. The intelligent software analyzes the topology of the textiles using features such as the course of the seam, local elevations or the relative position of seams, then classifies them according to textile type and class using various textures and embroidery patterns and translates these findings into robot commands.

The data is processed via Convolutional Neural Networks (CNN) and classical image processing. "We use IDS peak, the software development kit from IDS,” said Till Rickert, co-founder and chief product officer of Sewts. “We connect to our system via Python and the IDS Library. The added value of the IDS software package for us lies primarily in the ease of calibration and integration into our High Tech Vision System."

"AI is at the core of our technology. Intelligent algorithms are needed to build adaptive systems that can cope with non-deterministic automation processes,” said Rickert. “That's why we use the latest findings from AI research, refine them for our needs and finally put them together into a big whole. It receives various sensor data (e.g. optical information), draws conclusions from them on a human-like cognitive level and translates them into robot commands. In this way, systems like VELUM perform tasks that previously required the human mind.”

IDS towel casestudy800px

A strong outlook

With systems like VELUM, laundries can significantly increase their throughput regardless of the staffing situation and thus increase their profitability. "By closing this significant automation gap, we can almost double the productivity of a textile washing line," says Sewts CEO Alexander Bley.

IDS cameras can also be used for clothing such as shirts and trousers in the future. "It is important to understand the properties of these materials in order to implement robust processes,” said Bley. “We achieve this through sophisticated material simulations. In order to simulate the behavior of textiles, we create special FE simulations using the finite element method."

But the Munich-based company has an overarching vision: "We want to make it possible to automate the production of clothing and move it back to the point of use in a cost-efficient way,” said Bley. “In this way, we shorten transport distances, create more reliable supply chains, save CO2 emissions and combat the problem of overproduction." Additionally, applications with materials that are not textiles are also planned for the future.