The fingerprint of music making
To date, little data on how man and musical instrument work together has been available. Researchers at ETH are now investigating this interplay using sensors. The measurement data collected could help make practice more effective and interesting, and help to avoid typical problems suffered by musicians such as pain due to poor posture.
Professional musicians, particularly soloists, are top athletes. They coax wonderful sounds out of their instrument with apparent ease, but this ease is hard fought for – with hours of practice until even the most masterly of pieces sounds error free at breath-taking speed. In the process, musicians often complain of pain in their fingers, shoulders, joints and back, and try to cope with these typical pains using acupuncture, medication, massages or physiotherapy. It often proves difficult to find and solve the causes, as they are hardly noticeable from the outside and are often even difficult for musicians themselves to define. For example, the pressure of a finger on a violin’s string could be unnecessarily heavy or the violin awkwardly clamped and held tensely under the chin.
“Hardly any measurement data on a musician’s interaction with his or her instrument has been available to date,” explains Tobias Grosshauser, research associate for Gerhard Tr?ster, Professor at the Institute of Electronics, ETH Zurich. Such data, however, would be very helpful, especially in the field of medicine for musicians in order to identify the cause of pain.
More effective practice thanks to software feedback
Grosshauser and his colleagues are therefore developing sensor systems integrated in instruments that measure the pressure and position of the contact points. The musician’s posture, the angle of his or her arm, the hand position and how synchronously the right and left hand practise their task can be measured by the pressure, position and acceleration sensors. The data is sent to a computer via a transmitter.
The software processes measurement data and generates feedback for the musician that helps to develop optimal technique and make practice more effective, explains Grosshauser. An app for tablet computers or smartphones already exists. It depicts measurement data as graphs and has a red warning signal, for instance, if a musician presses his or her fingers too heavily on the violin’s strings.
Together with various music colleges, including Zurich University of the Arts and the Universities of Music in Nuremberg and Munich, Grosshauser and his team tested how sensitively the technology measures a finger’s position, for example. The software very precisely depicts even the finest of movements, as used by musicians during a gentle vibrato on the violin.
The data is also of interest in finding out how synchronously members of a group play. For this purpose, researchers equipped several musicians with sensor-fitted instruments and registered their interaction in the group.
Fatigue detectors for musicians
Sensors and software also detect fatigue during practice by registering, for example, when the violin is no longer held correctly or a movement is carried out less fluently. Researchers compared the musicians’ statements on their tiredness with the fatigue detected from the measurement data. “You often fail to notice how tired you actually are during practice,” explains Grosshauser, who has played the violin professionally for years. A fatigue detector could be built into the app, as already exists in several cars, so that it prompts the musician to take a break in good time.
Further studies and development are needed before the sensor technology and app are ready for the market, but Grosshauser and his colleagues are already occupied with how well this technology is accepted by musicians. “Young musicians in particular are fascinated by recording their own playing in this way,” says Grosshauser. To date, the technology has been met with mixed feelings from professional musicians as some are bothered by the cabling of their favourite instrument. For this reason, researchers are trying to further minimise the cabling so that it will be virtually invisible.
In contrast, this technology also offers the possibility of completely new forms of expression in music, something that should be of interest to both concert soloists and laymen alike. For instance, a violin fitted with sensors could be connected to a synthesiser and used for electronic music.
On stage and at home
Music teachers have not exactly warmly embraced the electronic advance either. But the aim is not to replace music teachers with software, stresses Grosshauser. Rather, sensor technologies would offer a tool to support lessons and practice.
Furthermore, they allow data on different styles of play in various situations – in the classroom, when practising at home and on stage – to be compared for the first time. A thin foil is glued to the violin’s fingerboard and connected via cable to the transmitter that is clamped to the body of the instrument, which means the technology requires little space and can be used everywhere. “You instinctively notice that you play differently when you’re playing for yourself alone at home or on stage in front of several hundred people.” Now the differences can be quantified and the data used, for instance, to get to the bottom of the cause of pain when playing music.
Video in German:
Further reading:
Grosshauser T, Candia V, Hindebrand H, Tr?ster G: Sensor Based Measurements of Musicians' Synchronization Issues. Proceedings of the International Conference on New Interfaces for Musical Expression (NIME), University of Michigan, 2012 (pdf)
Grosshauser T, Tr?ster G: Further Finger Position and Pressure Sensing Techniques for Stringed and Keyboard Instruments. New Interfaces for Musical Expression, NIME13, 2013 (pdf)
Grosshauser T, Gro?ekath?fer U, Tr?ster G: Fatigue Detection in Music Making Using Wearable Sensors, SMAC - Stockholm Music Acoustics Conference, 2013 (pdf)
Further information:
Musicians interested in the sensor technology can contact Tobias Grosshauser by email: