Urban Interaction idea

Idea #1

What?       Augmented Music player

  • Different style ( genre ) of music plays in a different space automatically based on GPS.
  • augmenting sounds.

For example, If you are in the gallery, classical music will be played and If you are in the park, jazz music will be played.

Or maybe add a function that people can customize their music zone??

Why?       I love music and people love music.

Instead of listening to their own music playlist, people can listen to same music at the same time and share the same experience in the same space.



Urban Interaction idea

#2 Weather+Time Clock

I tried to make an Interactive clock that you can check time & weather controlling with your hand gesture. ( using Leap Motion )

First, I created simple sketches with Processing using ‘Temboo’ weather library to bring the weather data.

Then, I projected & mapped these sketches to a round shape canvas using Madmapper.

For the user interaction, I choose to use hand gestures.

So for now, If you swipe your hand above the sensor, it will show you other city’s weather.


I didn’t put an interaction for the clock yet, but I will add more interactions soon for other functions!



 PROCESS – MadMapper


SKETCHES – PROCESSING ( Temboo weather data  & current time )



FullSizeRender 7


53 4


#2 Weather+Time Clock

Final Project- Swell Beats


Swell Beats- Amanda Lee & Jeff Park

For our final project, we wanted to make a multiplayer VR music experience that lets user “ surf ” the MIDI wave.




GeorgeandJonathan.com is an NYC electro duo’s band page. On this page, they have these “music videos” that show you all the notes used in their songs. Using Three.js, they let you navigate these notes from a traditional bird’s eye view.
Screen Shot 2017-05-06 at 2.24.44 PM
Screen Shot 2017-05-06 at 2.24.54 PM Screen Shot 2017-05-06 at 2.25.04 PM


Screen Shot 2017-05-06 at 2.32.13 PM


Idea Sketches

Screen Shot 2017-05-06 at 2.43.18 PM Screen Shot 2017-05-06 at 2.43.24 PM











Tech used

  • Three.js
  • Cordova
  • p5&Tone.js


For now, We made a simple environment that makes sound (notes) when users pressed arrow key (left&right). And Whenever user presses the key and makes a sound, it creates a box behind the user like a tail.

And we connect with WebVR, so user can explore the space dynamically.

Screen Shot 2017-05-06 at 2.52.44 PMScreen Shot 2017-05-06 at 2.55.28 PMScreen Shot 2017-05-06 at 2.55.20 PMScreen Shot 2017-05-06 at 2.55.12 PM


And we are still working on this project !

What comes next?

  • Technical Challenges
  • Supporting socket.io with a-frame
  • How to ensure a good framerate with other people making shapes?
  • Connect the Daydream Controller
  • Visual polish


  • The Music
  • Map pitch bend OR volume to a viewer’s z-position (up/down)
  • How to implement rests? (perhaps a jump?)
  • Quantize to the beat
  • Add different instruments


What did we learn?

  • WebVR with Three.js! A-frame is nice, but it doesn’t get us as far as three.js
  • We have a lot more design-wise to think through to refine the experience.
  • Tone.js vs p5.sound
Final Project- Swell Beats

Counter app- User test




User test



User test question

  • How is the first impression ?
  • What do you think about the interaction/Animation ? ( need more instruction ?)
  • What kind of function should i add to make it better ?



User test feedback

< First page >

– The first page is little bit confused ( Goal/ Setting ). Mostly  users pressed Goal first before setting the goal.

-Instead of the word ‘goal’, what if it shows the goal itself like coffee or days.

-Use icons.

-Make a difference between Goal / Setting, like color or size or shape to recognize the right order to press.


<Setting page>

– For the setting, make some option for the units so users can choose it more                            easily.

– Plus icon – change it to “Change goal ” or “Save goal” / change the position below                    to count option so users can guess easily what does that icon do.

– Make the user can set only one goal and show the box scene when I open the                         app, so users can play with immediately.

– Users can’t change the goal before they achieve it so they will feel more                                   responsible about it.


< Box scene >

– Add more interaction to play with. ( rotating the scene or other interaction that                        users can play with box )

– Make a space or function that user can pile up the box in their own way.

– Change a color in every 10 boxes, so user can see the process in visually.

– (When user counts up) Make a line on the top, so user can see how much they                        achieve and how much is left for their goal.

– When user achieve their goal, make a party scene.

– Add sad face icon for when user fails to count their goal. If they press it, all of the                   box will fall down or some of the boxes will be reduced or added as a penalty.


IMG_2524 IMG_2520 IMG_0143

Counter app- User test