How do you avoid discrimination while protecting user data?

How to provide relevant and personalized information to Internet users, while having less and less access to their personal data? How to ensure that an advertisement or a job offer will be seen in a non-discriminatory way by the people to whom it is pushed? These are topical questions, which are central for the advertising marketplace organizer Criteo, whose tools are those of machine learning.

Data are less and less provided, and they are less and less accurate, because mechanisms (technological or algorithmic) are put in place to rid them of so-called sensitive data – to which Criteo does not have access, by the way - and be used as discriminating criteria. At the same time, when we exploit the data, the algorithms can detect and reinforce biases, even without having access to this sensitive data of Internet users.

explains Vianney Perchet, a researcher at Criteo, as well as at CREST, a professor at ENSAE Paris and scientific co-leader of the Inria FAIRPLAY project team. "The question today is how to measure the extent to which algorithms are discriminating, while respecting data protection," he adds.

But then how can we explain this discrimination by machine learning systems, even though sensitive user data is protected? There are several reasons for this problem. First of all, the fact of not having access to private data, such as the user's gender, does not guarantee that there will be no discrimination -on this same criterion-. "There is a lot of non-sensitive data that is highly correlated with data considered sensitive, such as the sites consulted by Internet users, which can give indications of the gender of the person consulting these sites," explains Patrick Loiseau, an Inria researcher and scientific co-leader of the project team. Another problem is the multi-agent aspects, which make things more difficult. "Typically, in an auction system, decisions are made asynchronously and decentralized. The fault does not necessarily lie with the advertiser, because the final decision to print an advertisement is the result of a chain of intermediate decisions made by different agents," he says.

To address this issue, Inria, Criteo and CREST (a joint CNRS-ENSAE Paris -ENSAI-Ecole Polytechnique research unit) have decided to work together through the creation of a joint project-team. Named FAIRPLAY, this project-team is composed of five academic researchers (Inria, ENSAE Paris and Ecole polytechnique) and four researchers and engineers from Criteo. Its credo: to study learning problems in multi-agent systems.

It has – for example – been proven that women receive job offers that are on average less well paid than men.

The objective behind the work of the project team is to improve the automatic systems of marketplaces, but also to be able to know the degree of discrimination of certain algorithms, while remaining compatible with the notions of privacy protection.

"Our idea is to be proactive in the creation of these algorithms and their implementation, to force them to respect global constraints of fairness," says Vianney Perchet. "We approach this with the help of game theory, which is a tool for modeling multi-agent systems that makes it possible to evaluate the quality of systems and find solutions," adds Patrick Loiseau.

As with a traditional project team, FAIRPLAY has been created for a renewable four-year period. It will be housed at both CREST and Criteo.