26 Jul 2017
Using CGI to translate the physical language of ballet
The visual language of ballet is being translated by researchers from Loughborough University who are cataloguing every movement with the help of CGI and English National Ballet First Soloist, Junor Souza.
Ballet dancers literally speak to each other through choreography, says former professional ballerina-turned-linguist Dr Arianna Maiorani – one of the three academics working on the two-year project which is being carried out in partnership with English National Ballet.
A dancer’s physical dialogue involving leaps, pirouettes and arabesques can become incredibly intricate, says Dr Maiorani, and can represent conversations between performers, and sometimes the audience, conveying mood, intentions and thoughts which are hidden within the choreographed phrases. But, the phrases are often subtly altered and combined with others to transform the message, making it even more difficult to understand – especially for casual enthusiasts.
So, last month, Dr Maiorani along with Professor Massimiliano Zecca, who specialised in wearable technology, and computer scientist Dr Russell Lock, travelled to English National Ballet headquarters in London to analyse and translate each delicate movement performed by Junor and digitally record them as part of a compressive guide to the art called Kinesemiotics - 'the study of motivated movement informed by linguistics, engineering and computer science'. The picture shows the sensors and an example of CGI capture.
“Ballet is very challenging,” said Dr Maiorani. “It is a form of art that has a high cultural value across the world and is yet only really understood in its true meanings only by an elite of professionals and connoisseurs. Moreover, the traditional forms of notation still in use are very complicated and laborious to use and may create problems for new choreographic creations. Besides, they only focus on the mechanics of the movements, not on their meanings. Kinesemiotics will open the access to dance as a form of communication to a much wider audience, which includes not only practitioners and specialists but also learners and scholars of all types, including those with visual impairments.”
In the future, Kinesemiotics - from the Greek 'kinein', "to move" and 'sēmeiōtikos', "observant of signs" - will also allow the team to better understand many more areas of the arts which involve movement, even martial arts which have an element of performance.
Dr Maiorani has devised a way analysing the structure of dance titled the Functional Grammar of Dance, upon which Kinesemiotics is based. The initial implementation of the system saw the team visit English National Ballet, in London, to track the actions of its First Soloist, Junor Souza, using motion capture technology. The Brazilian dancer, born in Rio de Janeiro, performed scores of individual movements, and some phrases, which were digitally documented with the wearable sensors attached to his head, torso and limbs. The result was a unique digital catalogue of positions and real-time dance actions, the meanings of which the researchers will now identify and translate.
Prof Zecca said: “I’ve used this technology for numerous different applications ranging from gait analysis of elderly people to assessment of neurosurgery, suture, and laparoscopic surgery. The technology is also used in a much wider range of applications from sport to the design to biomechanics. The key novelty here, is that not only we want to understand the type and quality of the dance movements – plenty of other projects doing so – our aim is to understand and translate, for everybody, the meaning of the movements.”
The goal of the project is to create software which will allow people, even those with visual impairments, to understand the deeper meanings of ballet performances. Choreographers and tutors will also be able to create and adapt performances to include dynamic conversations and more powerful phrases in their routines.
“The challenge is to develop software which enhances our ability to both create and understand ballet,” said Dr Lock. “We do so in the knowledge that we need to support, rather than replace existing staff roles involved in the creative processes associated with ballet. So, for example in the case of a choreographer, what tools could be developed to enable them to document their thought processes more efficiently, and in such a way as to promote sharing with others. In the case of training, it is not a case of replacing a human trainer with a machine, but instead to give the human trainer a richer set of information upon which to base their training.”