Teaching rhythm using a connected bracelet: a project from EVA, the Instituto d’O Passo and the Besançon Conservatory

As part of its diverse range of musical teaching, the Besançon Conservatory offers courses in O Passo, an educational method devised by Lucas Ciavatta in Brazil in 1996 which promotes the learning of musical rhythm through the movement of the body in space. Keen to employ new technology in an educational context and to offer an objective evaluation method in his courses, Thomas Nicol, assistant director of the Instituto d’O Passo, turned to EVA (Inria Paris). This collaboration led to the creation of “ThomThom”, a multidisciplinary project coordinated by Thomas Watteyne (EVA, Inria Paris) that brings together music and wearables. 

Learning rhythm through the O Passo method

When learning the basics of musical theory, you are taught that time in music is divided into three different elements:

• the beat, a regular pattern which serves as a temporal marker, like the ticking of a clock
• the tempo, which is the time between two beats
• the rhythm, which is the combination of sounds and silences around the beat

It can prove difficult for many people to understand abstract representations, but the O Passo method makes it easier to understand the structure of music by materialising the body in space. Your feet carry the beat, while your hands tap out the rhythm.

In this method, close attention is paid to accuracy in relation to time, with a particular focus on the beat at both an individual level (the steadiness of your own walking pace) and at a group level (people walking in step with one another). The rhythm, which is tapped out using your hands, must also be accurate and measured against the beat.

© Institut d’O Passo

A connected bracelet to measure rhythm precision

With no tool available for assessing a student’s accuracy while they are practising O Passo, the Instituto d’O Passo and EVA (Inria Paris) decided to work together on the ThomThom project.

EVA's researchers are currently working on producing connected bracelets capable of evaluating students’ accuracy by capturing the different movements used to mark out the beat and the rhythm. Fitted with an accelerometer, these bracelets record raw data on the positive and negative accelerations generated by these movements. With the sensors forming a network, it is then possible to carry out real-time analysis in order to see how accurately students are keeping up with the rhythm.

© Inria

Long-term, the device could have a range of uses:

• During a group class, the connected bracelets could be used for educational purposes, enabling teachers to set collective challenges or to track the progress made by each of their students.
• At an individual level, students would be able to use the device as a tool for assessing and monitoring their own performance and their musical education, helping them train and improve.
• During concerts, this type of  wearable technology could be used as a virtual instrument. Specific sounds could be triggered during certain movements, such as the sounds of a virtual drum kit.

The different stages in creating the connected bracelet

The first phase involves designing an algorithm for detecting movements of the hands and feet. For this, EVA will be conducting tests at the Besançon Conservatoire on 7 November with a group of volunteers, involving devices fitted to the ankles and to one of the hands of each participant in order to capture and record their movement. The researchers will then be able to analyse the data from these tests in order to create an increasingly precise algorithm for detecting when people are tapping out the rhythm or the beat. It is then that IoT “6TiSCH” technology is applied, which is being developed by the team from the Paris research centre for the purposes of gathering data in real-time in a scalable, synchronised way.

The second phase, which is scheduled for early 2021, involves incorporating the technology into a bracelet. With the algorithm enabling the sensors to be smaller than a paperclip, the connected device itself will be very light. The bracelets will be equipped with a light, allowing students to quickly check to see if they are in rhythm with the rest of their group. Long-term, the light could also be used to create light displays, which would add an extra dimension to concerts. Thomas Watteyne’s goal is “to have a usable product by summer 2021” 

© Inria