Losless and lossy compression of screen-content data using machine learning

• Applicants: Professor Dr.-Ing. André Kaup; Professor Dr.-Ing. Tilo Strutz
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 438221930

This project explores a novel approach to the compression of screen-content data, taking into account different facets of image properties and the specific objective of compression.In contrast to classical photo or broadcast-video data, the content of screen-content images is very diverse with regard to its statistical properties. Often, certain regions within the images are characterized by two typical properties: a limited number of colours and repeating patterns.Studies have shown that conventional compression methods are not able to store screen-content data efficiently even with the use of special tools. A method based on ideal data coding is much more successful. The success of this new method depends on an optimal modelling of the probability distributions of the pixel symbols.For this new compression method, a conceptual approach (prototype) has already been realized, which is however still limited to lossless compression and achieves a high compression efficiency only for image contents with certain properties.The project investigates new methods for modelling the probability distributions of pixel symbols using machine-learning methods. Overall, the project pursues several approaches:• The existing prototype only uses global image information in some processing steps for modelling the probability distributions. Estimation methods that incorporate more prior knowledge, e.g. of more local nature, are expected to yield significant gains in compression. Alternative learning methods are being considered for this purpose. In general, it is about learning with a few examples. For this approach, higher compression ratios can be achieved than with classical methods, which first segment the image into regions of different types and then switch to conventional compression methods if necessary.• The research project will also investigate how a suitable rate-distortion optimization can extend the procedure to a near-lossless or lossy compression. A dedicated image analysis can parameterize the optimization in order to consider perceptual models of human vision. • An extension of the conceptual approach to image sequence compression is possible in principle and is aimed at. The consideration of the temporal component can lead to an improved modelling of the probability distribution depending on the situation and must therefore be researched. In image sequences, changes in signal statistics typically occur due to new content or scene changes. In particular, the research project should investigate how the existing learning procedure can be supplemented with suitable elements for forgetting or relearning.

• DFG Programme: Research Grants