Junor Souza from the English National Ballet taking is part of the Kinesemiotics project

Junor Souza. Picture credit: English National Ballet

Using CGI to translate the physical language of ballet

The visual language of ballet is being translated by researchers from Loughborough University who are cataloguing every movement with the help of CGI and English National Ballet First Soloist, Junor Souza.

Junor Souza ballet

Example of the CGI capture and the sensors, as worn by First Soloist Junor Souza

Ballet dancers literally speak to each other through choreography, says former professional ballerina-turned-linguist Dr Arianna Maiorani – one of the three academics working on the two-year project which is being carried out in partnership with English National Ballet.

A dancer’s physical dialogue involving leaps, pirouettes and arabesques can become incredibly intricate, says Dr Maiorani, and can represent conversations between performers, and sometimes the audience, conveying mood, intentions and thoughts which are hidden within the choreographed phrases.

But, the phrases are often subtly altered and combined with others to transform the message, making it even more difficult to understand – especially for casual enthusiasts.

So, last month, Dr Maiorani along with Professor Massimiliano Zecca, who specialised in wearable technology, and computer scientist Dr Russell Lock, travelled to English National Ballet headquateres in London to analyse and translate each delicate movement performed by Junor and digitally record them as part of a compressive guide to the art called Kinesemiotics - 'the study of motivated movement informed by linguistics, engineering and computer science'.

“Ballet is very challenging,” said Dr Maiorani. “It is a form of art that has a high cultural value across the world and is yet only really understood in its true meanings only by an elite of professionals and connoisseurs.

“Moreover, the traditional forms of notation still in use are very complicated and laborious to use and may create problems for new choreographic creations.

“Besides, they only focus on the mechanics of the movements, not on their meanings.

“Kinesemiotics will open the access to dance as a form of communication to a much wider audience, which includes not only practitioners and specialists but also learners and scholars of all types, including those with visual impairments.”

Junor Souza ENB

In the future, Kinesemiotics - from the Greek 'kinein', "to move" and 'sēmeiōtikos', "observant of signs" - will also allow the team to better understand many more areas of the arts which involve movement, even martial arts which have an element of performance.

Dr Maiorani has devised a way analysing the structure of dance titled the Functional Grammar of Dance, upon which Kinesemiotics is based.

The initial implementation of the system saw the team visit English National Ballet, in London, to track the actions of its First Soloist, Junor Souza, using motion capture technology.

The Brazilian dancer, born in Rio de Janeiro, performed scores of individual movements, and some phrases, which were digitally documented with the wearable sensors attached to his head, torso and limbs.

The result was a unique digital catalogue of positions and real-time dance actions, the meanings of which the researchers will now identify and translate.

Prof Zecca said: “I’ve used this technology for numerous different applications ranging from gait analysis of elderly people to assessment of neurosurgery, suture, and laparoscopic surgery.

“The technology is also used in a much wider range of applications from sport to the design to biomechanics.

“The key novelty here, is that not only we want to understand the type and quality of the dance movements – plenty of other projects doing so – our aim is to understand and translate, for everybody, the meaning of the movements.”

The goal of the project is to create software which will allow people, even those with visual impairments, to understand the deeper meanings of ballet performances.

Choreographers and tutors will also be able to create and adapt performances to include dynamic conversations and more powerful phrases in their routines.

“The challenge is to develop software which enhances our ability to both create and understand ballet,” said Dr Lock. “We do so in the knowledge that we need to support, rather than replace existing staff roles involved in the creative processes associated with ballet.

“So, for example in the case of a choreographer, what tools could be developed to enable them to document their thought processes more efficiently, and in such a way as to promote sharing with others.

“In the case of training, it is not a case of replacing a human trainer with a machine, but instead to give the human trainer a richer set of information upon which to base their training.”

For more information about English National Ballet or to arrange interviews with Junor Souza please contact PR Manager Robert James on Robert.james@ballet.org.uk or 0207 590 2933


Notes for editors

Press release reference number: 17/107

Loughborough University is equipped with a live in-house broadcast unit via the Globelynx network. To arrange an interview with one of our experts please contact the press office on 01509 223491. Bookings can be made online via www.globelynx.com

Loughborough is one of the country’s leading universities, with an international reputation for research that matters, excellence in teaching, strong links with industry, and unrivalled achievement in sport and its underpinning academic disciplines. It has been awarded five stars in the independent QS Stars university rating scheme, named the best university in the world to study sports-related subjects in the 2017 QS World University Rankings and top in the country for its student experience in the 2016 THE Student Experience Survey.

Loughborough was ranked 6th in the Guardian University League Table 2018 and 10th in The UK Complete University Guide 2018 and was also named Sports University of the Year by The Times and Sunday Times Good University Guide 2017. It has been awarded a Gold rating in the Teaching Excellence Framework, which recognises institutions that deliver consistently outstanding teaching that is of the highest quality found in the UK.

Loughborough is consistently ranked in the top twenty of UK universities in the Times Higher Education’s ‘table of tables’ and is in the top 10 in England for research intensity. In recognition of its contribution to the sector, Loughborough has been awarded seven Queen's Anniversary Prizes.

The Loughborough University London campus is based on the Queen Elizabeth Olympic Park and offers postgraduate and executive-level education, as well as research and enterprise opportunities. It is home to influential thought leaders, pioneering researchers and creative innovators who provide students with the highest quality of teaching and the very latest in modern thinking.