Orchestral Conducting in Virtual Reality

In 2019, Professor of Music, Heather Buchman and Instructional Designer VR/AR Technologist, Ben Salzman began collaborating on a method to use 360 ambisonic video to better prepare music students for the auditory and visual experience of conducting an orchestra.

The primary objective of Buchman’s course Conducting 316, was to provide students with instruction and practical experience in basic methods and techniques of orchestral conducting. The students learned the elements of conducting, including baton technique, aural perception, rehearsal techniques and score study (both instrumental and choral). For the final exam, students were given the opportunity to conduct a section of Brahms Symphony No. 3 with an orchestra. Unfortunately, before this exam most students did not have the opportunity to practice with an entire orchestra.

Salzman and Buchman set out to create an immersive 3D 8k 360 video shot from the point of view of the podium so that her students could practice performing Brahms Symphony No. 3 in an immersive 360 environment. Although not fully interactive, they hoped the visual and immersive auditory experience could calm performance anxiety. Over a period of 3 weeks students came to Hamilton’s VR lab in the Burke Library to practice. Alexandra (Alex) Wohnsen, Research Librarian for Teaching and Learning Initiatives, surveyed the students throughout the semester. Sixty percent of student respondents found that the VR experience served to decrease their stress and anxiety at the prospect of conducting the orchestra. Additionally, eighty percent of the students reported that the VR experience served to increase their confidence in preparation for the final exam. Buchman, also, filmed her students conducting before and after practicing in VR. Dramatic technical improvement over past year’s performances was observed. While most of the qualitative research from students showed that they found the experience helpful, a theme emerged. All participants reported that the experience would benefit from having more control and being able to interact with the tempo of the recording.

Buchman and Salzman spent the next year working with Professor of Computer Science Stuart Hirshfield, and four senior computer science majors (Anthony Reyes, Elias Griffin, Caius Arias, and James Reynolds) to expand on the VR environment and make it more interactive. Using Unity, the student team successfully coded oculus controllers and created a virtual baton to give the user the ability to conduct the tempo of the work and manipulate the playback of the recorded performance.

Share: