Content-Based Haptic Texture Retrieval

• Applicant: Professor Dr.-Ing. Eckehard Steinbach
• Subject Area: Electronic Semiconductors, Components and Circuits, Integrated Systems, Sensor Technology, Theoretical Electrical Engineering ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2015 to 2018
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 290848755

While stroking a rigid tool over the surface of an object, vibrations are induced on the tool. Measured with an accelerometer, the corresponding acceleration signals carry important information about the surface texture and can be used to classify or to recognize the surface material. In the proposed research project we plan to go beyond texture recognition. Our objective is to develop methods and algorithms, which allow for the retrieval of perceptually similar textures from a database of available textures. A tangible future application for texture retrieval is product browsing and customization, for example for the selection of materials for car interiors or for furniture. Based on personal preferences, a subset of materials possessing a haptic feel similar to the one chosen before may be retrieved. We refer to this novel concept as Content-Based Haptic Texture Retrieval (CBHTR).Essential ingredients of CBHTR are mathematical fingerprints representing the texture, so-called features, that are robust against external influences. When a human strokes a rigid tool over an object surface, the applied force, the scan-velocity or the inclination between the tool and the surface might vary drastically during the surface exploration and between subsequent exploration sessions. Such scan-time parameters heavily influence the nature of the acquired acceleration signals. Scan-invariant features for tool-mediated haptic texture exploration are currently lacking. Once scan-invariant features are available, robust texture recognition systems can be built. Texture retrieval systems, which are able to return perceptually similar materials, require features that additionally capture the perceptual similarity between textures. Features that are inspired by the way humans perceive textures are promising candidates along this line. Together, the development of robust and perceptually relevant features constitutes the first main objective of this proposal.The second objective is to apply and evaluate these features in a prototypical texture recognition and texture retrieval system, where the latter is supposed to find all perceptually similar textures in a database. To this end, appropriate combinations of features and machine learning approaches need to be identified. A systematic evaluation using subjective similarity scores plays a special role in this context.Finally, in order to further improve the retrieval performance, we additionally plan to fuse the information sensed during the interaction by other low-cost sensors like cameras and microphones with the information gathered with the accelerometer.

• DFG Programme: Research Grants