The Effect of Audio-Visual Cues in Understanding Musical Performance

Abstract

This study investigated the effects of adding visual aesthetics to a music performance. Two modes of visual performance are explored: multimedia and performance cues. Multimedia and performer cues serve different purposes in expressing music in a performance and allowing the audience to interpret the music. Multimedia refers to the use of more than one medium of expression or communication in performance, while performance cues refer to the implementation of gestures and facial expressions to guide the direction of the music. In this research study, two experiments were performed with two separate samples of 24 subjects.  These experiments observe the two modes' effects on engagement, understanding, and overall enjoyment of the performance by their audience. Participants were required to listen to the audio performance of an excerpt of music and listen to the same excerpt with the visuals as well. The subjects answered a series of questions rating the listening and watching experience. The results overall showed that having the visual experience while listening to a musical performance enhances the subjects engagement, enjoyment and understanding of the music. A wilcoxin ranked sum test was performed on the two different parts of this experiment and the p values were 0.0486 and 0.00148, which showed that there was enough statistical evidence in the experiment that showed engagement levels were higher when both audio and visual modes were provided to the audience.