LightSynth VR is a music visualiser with a twist: all the music is generated in realtime by Melodrive and can be controlled by the player! You can direct the visuals imagery by interacting with a simple and intuitive emotional UI - all in immersive virtual reality. LightSynth is originally developed by Matt Garland.
LightSynth VR shows how it is possible to influence the music generation of Melodrive in realtime, and tightly sync the game world to the generated music. Players can change the energy level and emotion of the music by interacting with objects in the virtual environment. Players can also set the instrumentation they prefer and enjoy the interplay between the music and visuals.
Music should tightly adapt to the storyline and emotional setting of video games. In RPGs, for instance, progressive personal interaction and development with players entering various virtual environments require music that is appropriately styled for the circumstances and that adapts to the internal and external emotional setting of the game. If the player gets attacked by enemies, the music should become more tense. Highly adaptive music can also be used to provide hints to the player. For example, the temporary disposition of the creatures of an enchanted forest could be provided through the emotional state of the music. If the music is mellow, the animals are relaxed and won’t attack you. If the music is frantic, the animals are agitated and they will attack you if you get too close.
In many (VR) games music is often overlooked. Game soundtracks often rely on a collection of static songs, which don’t adapt dynamically to the emotional setting of the game. The music, especially in large fantasy worlds, is often heavily repeated. This detracts from the experience of the players, instead of boasting their immersion. Also, music in games often gives little opportunity for user interaction and personal customisation.
With Melodrive, players can listen to music that is always fresh as the AI provides an infinite stream of music which is continuously varied and adapted to the current emotional state. This creates adaptive music that is far more granular than current techniques can provide. Melodrive can also enable players with no particular musical skills to create custom music themselves for the games they experience/create. They will have control over the emotional content, musical style and instruments in a game, permitting greater user interaction and engagement.
Oculus Rift, HTC Vive