Final Project- Swell Beats

 

Swell Beats- Amanda Lee & Jeff Park

For our final project, we wanted to make a multiplayer VR music experience that lets user “ surf ” the MIDI wave.

 


Inspiration

http://www.georgeandjonathan.com/#2

GeorgeandJonathan.com is an NYC electro duo’s band page. On this page, they have these “music videos” that show you all the notes used in their songs. Using Three.js, they let you navigate these notes from a traditional bird’s eye view.
Screen Shot 2017-05-06 at 2.24.44 PM
Screen Shot 2017-05-06 at 2.24.54 PM Screen Shot 2017-05-06 at 2.25.04 PM



http://dinahmoelabs.com/_plink/

Screen Shot 2017-05-06 at 2.32.13 PM

 


Idea Sketches

Screen Shot 2017-05-06 at 2.43.18 PM Screen Shot 2017-05-06 at 2.43.24 PM

 

 

 

 

 

 

 

 

 

 


Tech used

  • Three.js
  • Cordova
  • p5&Tone.js

Demo

For now, We made a simple environment that makes sound (notes) when users pressed arrow key (left&right). And Whenever user presses the key and makes a sound, it creates a box behind the user like a tail.

And we connect with WebVR, so user can explore the space dynamically.

Screen Shot 2017-05-06 at 2.52.44 PMScreen Shot 2017-05-06 at 2.55.28 PMScreen Shot 2017-05-06 at 2.55.20 PMScreen Shot 2017-05-06 at 2.55.12 PM

 

And we are still working on this project !


What comes next?

  • Technical Challenges
  • Supporting socket.io with a-frame
  • How to ensure a good framerate with other people making shapes?
  • Connect the Daydream Controller
  • Visual polish

 

  • The Music
  • Map pitch bend OR volume to a viewer’s z-position (up/down)
  • How to implement rests? (perhaps a jump?)
  • Quantize to the beat
  • Add different instruments

 


What did we learn?

  • WebVR with Three.js! A-frame is nice, but it doesn’t get us as far as three.js
  • We have a lot more design-wise to think through to refine the experience.
  • Tone.js vs p5.sound
Final Project- Swell Beats