AI and high-performance computing: two closely linked fields?

Artificial Intelligence (AI) and High Performance Computing (HPC) are two constantly evolving fields of digital science that are increasingly complementing each other. Artificial intelligence is based on sophisticated algorithms that enable machines to learn from data and perform tasks autonomously. High-Performance Computing, on the other hand, involves processing massive quantities of data at extremely high speeds.

The cross-fertilization of AI and HPC opens up new opportunities to push back the limits of what computers can achieve, paving the way for new technological discoveries and innovations.

AI for faster calculations:

The combination of AI and HPC enhances the capabilities of computing systems in terms of computing speed. Supercomputers equipped with AI technologies process complex datasets in record time, paving the way for new advances in fields such as medical research, meteorology and climate modeling.

For example, the Concace project-team uses AI to build preconditioners, a tool that processes input data to facilitate the calculation process and obtain more efficient results, improving the speed and accuracy of methods for solving systems of linear equations, at the root of many physical problems, particularly in aeronautics. It is this hybridization of numerical simulation and machine learning[1] that scientists are trying to develop today.

AI to understand and optimize HPC: 

Artificial intelligence can also be used to understand HPC computing applications and systems. For example, scientists on the Storm project team are working on optimizing computer programs using machine learning to guide and select the best optimizations.

To do this, they start from the premise that if two programs are similar, they will probably need the same optimizations. But how do you determine whether two programs are similar? To do this, the researchers in this team use techniques comparable to those of language models: "When we examine a new program, we compile it, i.e. translate it into a language that the computer understands, and then subject it to language models to identify which known code it resembles. If the optimizations work well on A, then it's likely that they will work on B, which is much quicker than testing everything!" explains Emmanuelle Saillard, Research Manager in the Storm project team.

© Inria / Photo H. Raguet

AI to drive HPC: 

High-Performance Computing (HPC) requires a large number of computing resources, as well as multiple tasks (calculations) to be carried out. To manage this, scientists need to plan scheduling - determining when and where each task should be executed. To improve efficiency, scientists from the Storm, Tadaam and Topal project teams have developed a new approach using artificial intelligence, specifically reinforcement learning. "The idea is to define a strategy using an artificial neural network: we let the system "play" and then evaluate its performance. By repeating this process, we enable the system to learn from its mistakes, so as to obtain a model offering an effective scheduling strategy", explains Olivier Beaumont, head of the Topal project team.

HPC as a resource for AI: 

Computing resources have played a key role in the great learning successes and AI renaissance of recent years. To make large generative models work, AI requires considerable computing power. For example, scientists in the Flowers project-team are seeking to understand how children learn, and are testing their hypotheses by combining large language models with reinforcement learning. This involves building models using large amounts of data and computation to simulate a child's interaction with various environments.

To remedy this, scientists are developing models that are more "bio-inspired" than current models. "When we build models of human learning processes, we need to evaluate them by studying the learning and behavioral trajectories they generate in a wide variety of interactive environments. When these models involve the use of generative AI algorithms as one of their components, this further increases the need for computational resources to enable us to fully understand them," explains Pierre-Yves Oudeyer, head of the Flowers project team. However, before these alternative models can be used, scientists need a great deal of computing power to validate them, and potentially more frugal use of data and computation.

HPC adopts AI as new application:

High-performance computing scientists work in particular on optimization issues. The Topal project-team aims to develop efficient and optimized use of computing resources, particularly for machine learning. We see machine learning as a new application for HPC, in addition to the numerical simulations we have been working on until now," explains Olivier Beaumont. This particular application requires many parallel calculations and poses very singular memory management problems, making it complex to distribute data over computing resources". So it's an unprecedented challenge to take up: how to calculate faster while consuming as few resources as possible, and adapting to the available resource capacities?

© Inria / Photo M. Magnin

HPC to help preserve personal data: 

Artificial intelligence may also need HPC, especially parallelism and distributed algorithms, to achieve various objectives such as strengthening data confidentiality. "For federated learning, i.e. enabling several entities or individuals to collaborate on training an AI model without sharing their respective data, distributed algorithms can be a promising approach. Indeed, security protocols, sometimes based on cryptography, are developed to perform collective calculations without revealing the data exchanged. In this context, personal data remains private, and only proposed updates and model improvements are shared," points out Laercio Lima Pilla, a researcher at Storm. "However, federated learning tends to take longer and consume more resources than other approaches. In our team, we are striving to speed up and reduce the energy or CO₂ emissions needed to carry out this federated learning."