Swell Beats- Amanda Lee & Jeff Park
For our final project, we wanted to make a multiplayer VR music experience that lets user “ surf ” the MIDI wave.
GeorgeandJonathan.com is an NYC electro duo’s band page. On this page, they have these “music videos” that show you all the notes used in their songs. Using Three.js, they let you navigate these notes from a traditional bird’s eye view.
For now, We made a simple environment that makes sound (notes) when users pressed arrow key (left&right). And Whenever user presses the key and makes a sound, it creates a box behind the user like a tail.
And we connect with WebVR, so user can explore the space dynamically.
And we are still working on this project !
What comes next?
- Technical Challenges
- Supporting socket.io with a-frame
- How to ensure a good framerate with other people making shapes?
- Connect the Daydream Controller
- Visual polish
- The Music
- Map pitch bend OR volume to a viewer’s z-position (up/down)
- How to implement rests? (perhaps a jump?)
- Quantize to the beat
- Add different instruments
What did we learn?
- WebVR with Three.js! A-frame is nice, but it doesn’t get us as far as three.js
- We have a lot more design-wise to think through to refine the experience.
- Tone.js vs p5.sound