In the IGF research project 19479 BG, application-oriented foundations were developed for tools for the multimodal inspection of product surfaces for communication systems based on vibrotactile signaling. The research work included the description of haptic properties of textile surfaces, taking into account tactile and kinaesthetic aspects, and the transmission of tactile properties in a suitable signal structure using a physics engine.
Using a representative selection of 23 textile materials (clothing textiles/woven fabrics, knitted fabrics made of natural and synthetic fibers, automotive textiles, home and household textiles) from a sample of approx. 130 materials, extensive textile-physical and acoustic investigations as well as optical investigations were carried out to describe haptic parameters. A ring pull-through method practicable for SMEs was developed, with which the kinaesthetic aspects in particular can be depicted through the combined effect of bending, friction and compression. In addition, extensive tests were carried out with the TSA device from emtec Electronic GmbH to determine physical and geometric textile parameters using an acoustic measuring principle. This device is very well suited to characterizing and classifying textile materials across product groups in terms of their roughness (TS750) and softness (TS7). The haptic perception experiment showed very good agreement between the TSA roughness and softness and subjective perception. At the same time, the textiles were characterized acoustically using a motion simulator developed for this purpose. The recorded frequency spectra of the acceleration levels form the basis for the vibrotactile structure detection, whereby in particular the roughness of the textile surface could be mapped vibrotactilely. For multimodal interaction, it is necessary to present the textile structure digitally and as realistically as possible and at the same time to provide information by recording the surface profile, which enables the assignment of an acoustic and vibratory signal to the finger position in the z-direction when "touching" the fabric sample on the display of the playback device. The 2D and 3D images were recorded under defined conditions and provided in various geometric resolution levels for presentation on the playback device (smartphone). All determined textile-physical, geometric, visual, auditory and haptic parameters were stored in a multimodal database, which is accessed by a physics engine in order to process the data and combine it haptically, visually and auditorily in a suitable way for multimodal playback. Other research topics included the creation of a communication system and a web-based human-machine interface for multimodal interaction. Initial, promising investigations were carried out in emulated networks with 5G technology.
The interdisciplinary research results will finally be combined in a developed Android app for a smartphone from the consumer sector for video, audio and haptic output. The audio output takes into account the speed of the finger when swiping across the screen, while the vibration is enabled based on the finger position and the color values of the height profile images.
The project results can be used for the digital communication of quality requirements both within the global textile supply chain and in the end consumer sector. In the latter case, a sustainable reduction in the environmental impact of returns in online retail is expected in the future. The research approach is also transferable across sectors for production chains in other processing areas, e.g. paper and leather, as well as in medical technology.