29th February 2012 will go down as a landmark date in the history of photography. It was then that the American company Lytro launched a rectangular camera with a matrix of micro-lenses in front of its sensor. With one single exposure, it was capable of capturing 11 million rays of light, a wealth of data offering a range of image choices. Users could, for example, select a focal point of their choosing from all of those saved. This was the very first ‘light field’ camera (also known as ‘plenoptic’ cameras), and ushered in a paradigm shift with the potential to impact a whole host of fields.
Despite the fact that this promising technology hasn't been met with much enthusiasm, scientists have been drawn to it as a result of the challenges it raises. By chance, one of the underlying problems involves training the next generation of researchers for this multi-faceted field, which was the aim of Plenoptima, an innovative training network launched in 2020 as part of the EU’s Marie Skłodowska-Curie programme (MSCA).
A proposal was put to the European Commission by Mårten Sjöström, from Mid Sweden University and Atanas Gotchev, from Tampere University in Finland, who were acting on behalf of Full Parallax Imaging, another training network they had started up not long before. In addition to the two Scandinavian universities, the academic core of the Plenoptima network also has three other partners: the Institute of Optical Materials and Technologies (IOMT) in Bulgaria, the Technische Universität Berlin and Inria, where research began on plenoptic cameras a number of years ago.
Christine Guillemot is a researcher at the University of Rennes Inria centre who specialises in signal processing and image compression. In 2016 she received a grant from the European Research Council (ERC) enabling her to explore methods with the capacity to handle the significant dimensionality of these light fields. Then in 2020, at a national level this time, she was appointed to a chair in Artificial Intelligence dedicated to harnessing machine learning in the context of light field imaging. This new training network also draws on all of this research.
“In practise, the Plenoptima network was set up to train fifteen young researchers”, explains Christine Guillemot. “These students study for a PhD as part of a double doctorate, jointly supervised by two academic institutions which each issue a doctorate. In addition to their own university, they will spend part of the time studying at another university elsewhere in Europe. In our Inria team here, for example, we have three students affiliated to the University of Rennes 1. They will be spending several months at TU Berlin, Mid Sweden University and Tampere University respectively. In return, we will play host to a PhD student from Mid Sweden University and another from Tampere University for several months. The programme also allows for a two-month placement with an industrial partner."
The training is multidisciplinary, comprising workshops and ‘theme schools’ which each academic partner takes it in turns to organise. In March of last year, one of these ‘schools’ was held in Tampere to study the physical, mathematical and technological fundamentals underlying light field imaging. Another was held in September in Sundsvall, in Sweden, focusing on machine learning.
The next ‘school’ is set to be held in Rennes in June 2023, organised by Inria.
Verbatim
Our primary focus will be the processing and compression of plenoptic data, but we will also be looking at project management and innovation. On the last day we’ll have a session that will be open to businesses. This will give companies with an interest in light field technology an opportunity to meet scientists from the field.
Chercheuse et responsable de l'équipe-projet SIROCCO
And to provide use cases? “This hasn’t happened yet as we haven’t seen many breakthroughs in terms of real applications. A few companies have had a go, but there isn’t really much of a trend. When the Lytro camera came out, the image quality wasn't comparable to what even a basic smartphone could produce. When it comes to computational photography, everything is still a question of finding a compromise between data volumes and image quality.”
That said, in the past 10 years “significant algorithmic progress has been made, both in terms of the acquisition and restoration of data and reducing dimensionality.” And that’s not all. We are also looking to expand this research to cover omnidirectional cameras as opposed to projective cameras. On these cameras, images will no longer be captured in two dimensions, each plenoptic image being omnidirectional. In other words: 360° light field capture.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 956770