Discover HKBU
Marrying classical music with brainwave technology
30 Apr 2025
Professor Johnny M Poon conducted the Hong Kong Baptist University Symphony Orchestra (HKBU Symphony Orchestra) with precision and passion, using graceful and lively gestures to convey his vision of the score to the musicians. Behind him, the audience was captivated not only by the musical performance, but also stunning real-time visuals based on the musicians’ brainwave patterns.
“We brought the laboratory to the concert hall this year. This allows our performers to fully immerse themselves in an authentic performance environment while being engaged in research demonstration. It also shows HKBU’s commitment to transdisciplinary inquiry,” says Professor Poon, Associate Vice-President (Interdisciplinary Research), Founding Dean of School of Creative Arts and Dr Hung Hin Shiu Endowed Professor in Music at HKBU.
Understanding neuroscience through the lens of music
“The research questions at hand were: How can we understand the functions of the brain better? How can we use quantitative research methods to study human performance in an authentic setting?” Professor Poon says. To that end, he led a team of researchers at HKBU to bring together music and cognitive neuroscience at the HKBU Symphony Orchestra Annual Gala Concert 2025.
Conducted by Professor Poon, the concert held on 1 April at the Hong Kong Cultural Centre concert hall lavished nearly 1,200 people in the audience with a night to remember, with its splendid display of captivating images, lush orchestral sound and technological innovation.
The field of neuroscience took centre stage in the extraordinary performance of Gustav Mahler’s Symphony No.1 in D major, fourth movement Stürmisch beweget by the HKBU Symphony Orchestra. The work symbolises a struggle against fate and unfolds with intense emotional contrasts as it builds up to a powerful conclusion. At the concert, these musical dynamic shifts were visualised in a novel approach based on the interaction between the music and the performers.
Utilising portable functional Near-Infrared Spectroscopy (fNIRS) devices, the brain activities of Professor Poon and selected members of the orchestra were measured in real time by near-infrared light, which detected the levels of blood oxygenation in the brain. The devices transmitted the data via Bluetooth to a computer system for analysis.
“The data provides important insights into the brain activity and cognitive effort of different performers within an orchestra in a real concert, which enables the researchers to gain a more in-depth understanding of the brain,” says Professor Poon. “Our research team selected music as the primary focus of our demo as it engages multiple areas of the brain, making it a highly valuable subject for future research.”
By studying how the brain works, it would help researchers learn more about executive functions and pave the way for developing models that can optimise human performance and reduce performance anxiety.
Music as a bridge between art and technology
To enable the audience to visualise the changes in musicians’ brainwave patterns, the orchestra collaborated with a renowned Hong Kong digital artist, Thomas Lo, who employed artificial intelligence (AI) tools to transform brainwave patterns and music data into dynamic visual art in real time, creating a striking visual extravaganza alongside the live orchestra.
This novel fusion of neuroscience and creative expression provided the audience with a multisensory experience throughout the evening. The repertoire included a selection of masterpieces, featuring Spanish violinist Ms Raquel Areal Martínez, who performed the Violin Concerto No. 1 in G minor, Op.26 by Max Bruch with the HKBU Symphony Orchestra. Cantoría Hong Kong, an HKBU chamber choir, presented the choral works Warum ist das Licht gegeben dem Mühseligen?, Op.74, No.1 by Johannes Brahms and Friede auf Erden, Op.13 by Arnold Schoenberg. The concert also performed Leonard Bernstein’s Overture to Candide.
Through the integration of music, visual arts, science and technology, the concert not only provided a great opportunity for studying the connection between sound and the functions of the human mind, but also celebrated HKBU’s pioneering spirit in art-making and innovation.
Professor Poon believes many possibilities exist for the integration of art and technology, and he encourages society to adopt an open attitude towards AI applications. “The development of AI can foster the advancement of art and science. To address challenges in an ever-changing world, we need to nurture more interdisciplinary talents and engage in meaningful collaboration among artists and scientists,” he says.