XGAN - Learning an interpretable representation of videos generated by a GAN

The full title of the project is "Interpretable Representation Learning for Video Generative Adversarial Networks".

In this exploratory action, we will focus specifically on answering the following question: "Can we interpret the representations learned by video GANs? ".

Despite the remarkable advances in generative adversary networks (GANs), these networks currently operate as black boxes.
XGAN aims to break through the black box of GANs for video generation by proposing strategies for interpreting the latent space in the following areas:

• the design of interpretable architectures
• analysis of symmetric functions in input and output of patch-based generation.

Video generation is a new and difficult problem, and the principal investigator has led one of the few European efforts to address this challenge. Exploring the interpretability and explicability of GANs represents a new and fundamental issue.

The paucity of existing research on video generation is due in part to the vast computational resources required, which we gained access to through GENCI (Large National Computing Facility).

In a departure from previous research, XGAN will explore the interpretability and explicability of GANs, which represent new and fundamental questions, and thus present a high risk.

While the existence of symmetries is given for patches, and they can be analyzed mathematically, it is interesting to know if symmetries beyond the Euclidean transformation will exist. This is a difficult open question, the answer to which will allow the design of more interpretable models.

Within XGAN, Edouard Oyallon from CNRS, LIP6, will be a key collaborator, with whom we will analyze the symmetric input and output functions of patch generation.
His research focuses on machine learning and on the mathematical foundations of deep learning techniques.