Artificial Intelligence to support biodiversity by PlantNet

PlantNet.The name says it all: this interdisciplinary project combines botany and computer science. It is the fruit of an encounter between two teams, each specialising in one of the two areas (see inset). What it is exactly? A collaborative platform for plant identification. “When PlantNet was launched 10 years ago, it was a very theoretical research project that aimed to use image recognition techniques to inventory biodiversity”, explains Alexis Joly, joint head of the project. The Inria researcher and Pierre Bonnet, his alter ego at Cirad (French agricultural research and international cooperation organisation), are the driving force behind the project, which started out using the means at hand: “We were mainly relying on scanned leaves”, the scientist explains. “To start with, we didn't even consider a mobile app because there weren’t many around at the time, only a website”.

Others joined the team during the first R&D project, between 2010 and 2014. Hervé Goëau, a Cirad researcher in data science, also a member from the outset, was joined by three Inria engineers: Julien Champ and Antoine Affouard in 2012, then Jean-Christophe Lombardo in 2014. And the research progressed. “We focused on image recognition algorithms and in particular the search for similarities”, explains Alexis Joly. “This not only allows the name of the plant to be identified, but also to list the species that are the most similar. The user is therefore involved in the process because they choose the correct plant from among the ones presented. This improves the algorithm”.

It is a major innovation that PlantNet's competitors don’t have. The task was tedious at the beginning, but in 2015, deep learning made the researchers’ work easier. “We were among the first to develop plant identification at such as scale, listing 8,000 species”, says Alexis Joly.

2015 was also the year in which PlantNet took on a new dimension when the project received funding for four years through the Floris'Tic initiative, financed by the French government's Future Investment Programme, which now includes an education section. The mobile app, launched in 2013 for botanists, also became available to the general public. Now anyone can download the app and take a photo of a plant, view a list of potential species that might match and “vote” for the one that seems to be the right one. PlantNet can also be used for teaching and informing as well as agroecology, to identify things that aid growth or determine early on which plants are invasive, for example.

“We concentrated particularly on data validation”, says the joint head of PlantNet. “In a collaborative platform like ours, users participate in plant definition through their votes so the algorithm has to make decisions based on these choices: what value should they be given? How should they be taken into account?”

To explore this important matter further, the team developed a serious game called ThePlantGame. The player is shown a species A, for example, and is asked if it is species A, B or C. “By analysing identification success and failure rates, the algorithm is able to draw up a sort of skills profile and attribute more or less weight to a vote depending on the profile” explains Alexis Joly.

This educative technique also helps PlantNet progress. The application is thus becoming a well-known, reliable tool... and the number of users is exploding. “When we exceeded 100,000 downloads, we were amazed!” says Alexis Joly. “We reached the 20 million mark in 2013, and in spring this year, we reached around 500,000 users per day. The number doubles every year. We’re wondering how far it will go!” At the same time the data integrated into the application is increasing and now includes 27,909 plant species.

The team doesn’t intend to stop there. Since 2019, PlantNet has become an InriaSoft consortium and the research continues. “We’re examining the sampling bias of our data, for example”, explains Alexis Joly. “The data is collected by users who are mostly in urban areas... so certain species in these environments are overrepresented. Our aim is to make unbiased models of plant distribution”.

Other avenues of research include using deep learning to learn about the environmental preferences of plants and to predict where a certain plant is likely to grow. The team also aims to develop an embedded “offline” version of the application.

Finally, since the start of this year, the most reliable observations from PlantNet have been integrated into the GBIF (Global Biodiversity Information Facility) database which aims to provide free access to data on all forms of life on Earth to everyone everywhere. “We are the first to integrate data from artificial intelligence” says Alexis Joly. “It’s an important recognition of the level of performance of our application.” An achievement now accompanied by the Inria - French Académie des Sciences - Dassault Systèmes Innovation Award.