University Review
At the forefront of innovation in art tech
A ground-breaking collaboration between humans and machines
Recent advancements in digital technologies have opened new avenues for the arts. Artists and scientists at HKBU have collaborated through transdisciplinary research to explore the convergence of art and artificial intelligence (AI) for humans to usher in a new era of art technology.
A pioneering attempt to apply AI technologies to the performing arts became an overnight success at the HKBU Symphony Orchestra Annual Gala Concert 2022. The innovative performance presented in July 2022 was the first human-machine collaboration of its kind in the world.
In a highly expressive performance, the HKBU Symphony Orchestra shared the stage with an AI choir, which comprised the voices of 320 virtual performers, to perform a new arrangement of the song Pearl of the Orient. At the same time, an AI media artist used the song’s lyrics to create a cross-media narrative visualising its aesthetic interpretation of the song.
The Concert also featured a ballet performance, in which AI virtual dancers performed to Ravel’s Daphnis et Chloé, accompanied live by the HKBU Symphony Orchestra. The AI virtual dancers were trained to identify the underlying emotional and aesthetic connections between music and dance for choreographing the ballet. The dance movements were inspired by a newly discovered species of box jellyfish in Hong Kong.
The AI-driven performances at the Concert were one of the many milestone deliverables of the “Building Platform Technologies for Symbiotic Creativity in Hong Kong” research project. This art tech initiative was awarded over HK$52.8 million in funding by the Theme-based Research Scheme (11th round) under the Research Grants Council for a period of five years.
The project recognises HKBU’s commitment to the development of transdisciplinary research and art tech. The University will continue to develop innovative technologies that can transform the creative and cultural industries and enable Hong Kong to assume a leading position in art tech on the global stage.
.
Creating future cinematic experiences
Scholars at HKBU are not only making technological breakthroughs, but they have also taken people’s appreciation of the arts to a new level by offering them unprecedented cinematic experiences. Led by Professor Jeffrey Shaw (Academy of Visual Arts), the vanguard project “Future Cinema Systems: Next-Generation Art Technologies” opens the door to boundless applications in the arts and creative industries.
In collaboration with the City University of Hong Kong and the École Polytechnique Fédérale de Lausanne in Switzerland, the two-year art tech project was awarded HK$35.4 million in funding from the Innovation and Technology Support Programme under the Innovation and Technology Commission of the HKSAR Government.
For the project, Professor Shaw has been working with AI experts led by Professor Liu Jiming, Associate Vice-President (Research Development) and Dean of Science, to leverage technological breakthroughs in the fields of visualisation, human-computer interaction and co-evolutionary narrative to explore the future experience of archive, place and performance.
The unique outcomes of the project are showcased at the Visualisation Research Centre on campus. Home to the world’s first 360-degree immersive LED Visualisation Cinema and the 180-degree iDome Cinema, the newly established facility provides an innovative platform for new forms of immersive and interactive experiences.
.
Advancing art creation with the Turing AI Orchestra
Known as the father of computing, Alan Turing laid the foundations for the technological advancements in the field, and the new Turing AI Orchestra (TAIO) launched by HKBU has created a similar impact in terms of the way humans and machines produce art together.
As the world’s first AI ensemble, TAIO uses state-of-the-art AI technology to achieve a new form of symbiotic artistic creation and performance between humans and AI systems. It is also the next milestone in the University’s visionary plan to broaden the scope and explore potential opportunities in art co-creation by humans and AI.
The new Orchestra was launched at the “Human, Machine, Art, Creativity: International Symposium” co-organised by HKBU and the Hong Kong Productivity Council on 9 August 2022. At the event, world-renowned researchers and artists from academia and industry gathered to discuss the development of human-AI symbiotic art creativity.
Blending art, music and innovation with AI
AI technologies and applications have unlocked new opportunities for restoring and preserving history and cultural heritage. A transdisciplinary research team at HKBU restored and enhanced a classical silent film through the use of AI technologies, providing contemporary audiences with more opportunities to appreciate early cinema.
In June 2023, around 1,000 guests joined the spellbinding musical voyage at the HKBU Symphony Orchestra Annual Gala Concert 2023, where the orchestra performed the Danse Macabre by Saint-Saëns accompanied with a screening of the silent movie of the same name created in 1922 by director Dudley Murphy. The cross-disciplinary and cross-media artistic performance was made possible by the collaboration between HKBU musicians and computer scientists, who utilised cutting-edge AI models for the restoration and colourisation of the silent film produced over a century ago.
Another highlight of the Concert was the work of Australian computer graphics artist and musician Mr Andrew Quinn. He presented a unique perspective of heaven through his creation of real-time visuals for the orchestral performance of Mahler’s Ruhevoll from his Symphony no. 4.
Blending art and innovation with AI and other advanced technologies, the Concert showcased the artistic prowess of HKBU’s student musicians and how transdisciplinary research at HKBU inspires imagination and advances knowledge discovery.
Immersive soundscapes encapsulating the Baroque spirit
Professor Johnny ML Poon, Associate Vice-President (Interdisciplinary Research) and Founding Dean of the School of Creative Arts, curated a programme of Baroque music and soundscapes for selected paintings in the exhibition “The Hong Kong Jockey Club Series: The Road to the Baroque – Masterpieces from the Capodimonte Museum” at the Hong Kong Museum of Art. The curated songs and pieces not only captured the artistic zeitgeist of the era, but they also took visitors on an immersive journey.
For instance, Professor Poon and his team developed machine learning algorithms to generate models of singing voices in addition to the human vocals in a motet. By merging Baroque music with modern technology, the innovative performance symbolised a dialogue between humans and celestial beings. To put a contemporary twist on the interpretation of still life art, the team blended Baroque music with the sounds drawn from nature and AI-generated vocals, creating an immersive soundscape that encouraged people to enjoy, appreciate and understand art in a different way.
