New Paradigm for Compressing Visual Data

 
Thomas Maugey is Sirocco member, Inria, CNRS and University of Rennes team project, in Irisa.

Every single minute, roughly 500 hours of video are uploaded to YouTube alone. In the same time span, about 700,000 hours are being streamed. Not to mention the other sharing platforms and social networks of all ilk. This mind-boggling tsunami of data translates into tremendous amounts of electricity being gobbled to power and cool down the storage facilities.

To cap it all, the trend is just skyrocketing. Reducing the volume of data through compression would contribute to mitigate the problem. But classical algorithms can only offer limited gains due to an inherent limitation. «They are meant to reproduce the input data as faithfully as possible. Therefore, if we are to shrink the size of the data drastically, we need to shift paradigm. We need to change the very goal of compression itself» sums-up Thomas Maugey, a scientist at Inria research center, in Rennes, Brittany, France.

In the wake of an initial Inria exploratory action, Maugey was recently granted funding by the French National Research Agency in order to pursue an investigation into what has now been coined generative compression. The overall project goes by the acronym DARE which stands for Data Repurposing. 

«Instead of resembling as much as possible to the input data, we will allow ourselves not to reproduce it, but rather to encode it to a compact description of a semantic nature. The best analogy that comes to mind is the music sheet. When you think about it, the most compact, the most compressed representation of music is none other than... the music sheet itself. It can be replayed. Granted, the rendering might not be exactly identical to the original recording, but it is still quite close. So, the idea is to come up with the same concept of music sheet but for images. The data will then be reinvented at the decoding phase so that the regenerated content will be semantically coherent with the initial one. It will look like the input data.» 

For doing so, scientists will leverage Artificial Intelligence, and more specifically a type of neural network called Generative Adversarial Network (GAN). This algorithm learns to generate new data with the same statistics as the training set. For instance, it can create strikingly realistic photographs of people who do not even exist. And these photos are not composites of several people whose chin, jaws and eyebrows would be somehow stitched together. They are pure random creation. But in order for GANs to do that, they need something as an input. «They are fed with, say, 1,000 random digits. And from these digits, they will invent a portrait.»

However, there might be another way to use them. «If we could find a link between these digits and the rendered output, then we could pilot this generative instrument. Finding this link, building the input description, tuning the semantic is what our research is all about.» 

At the end of the day, the compression ratios could be gigantic. Having said that, «generative compression would certainly not be suitable for all videos. One wouldn't want to use it for the material of an artistic nature or recordings loaded with affect. Family footage for instance. It could be more relevant for cold data: all those videos that are very seldom accessed, that you don't really care much about anymore, but that you haven't resolved yourself to delete yet.» Repurposing thus offers a third way in between keep and trash. 

Additionally, the new compression scheme could also be applied to only one part of the image. «You wouldn't touch the foreground where the action is taking place but regenerate the background where nothing happens and to which nobody pays attention anyway. Keep the soccer players on the field, but regenerate the crowd in the stadium, for instance.» 

Who will ultimately decide which compression scheme is to be applied to which data? «It's up to users to make that choice. Our data repurposing framework will include tools for helping them to compress elements of their video collection as they see fit, be it for social networking, storage on the cloud, so on and so forth.»

The attitude of the public turns out to be an important ingredient in this quest for frugality. «There is a risk of a rebound effect, Maugey points out. Oftentimes, when technology improves the efficiency with which a resource is employed, the total consumption of that resource increases rather than decreases. As engine gas consumption went down, the usage of car boosted. The same thing is occurring with videos. The easier they are to make, the more they end up online.»