Final project

V-BOOK ( voice book ) – “Little blue and little yellow”

These days, There are lot’s of E-book and audiobook, but I think there is not that much voice controlled book. Actually, I got inspired by the project that Pedro showed us before, the one that makes the sound effect for kid books. Based on that reference, I wanted to use the basic voice recognition technic to trigger the visual animation.



Target audience

My target audience will be mostly children and also parents and teachers who usually read books for kids. It can be used in school or also can be used at home. I thought it would be more convenient instead of clicking something or swiping the page.

Interaction Flow

If the user reads the sentence, a magical animation will be triggered.









After I tested with basic voice recognition, I want to add more magical and unexpected interaction between reader and the animation of the book.



final idea

I’m going to make a web-based simple memory game. (memory test).

There is a game in Korea named ” if I go to (somewhere).”

For example, someone says “If I go to the market, there is an apple” and then, next person has to repeat the word and add something like “If I go to the market, there is an apple and there is a cashier”. People will take turn and keep repeating and adding the words.

If someone messed up the order or don’t know what to say next, then the game is over.

I can say this game is a combination of listening and memorizing and saying correctly. I thought it would be fun If I remake this game as a web-based game using Speech to Text and Text to Speech.

The computer will never lose in this game… So this is more for fighting with yourself like a memory test.

And I’m not sure whether I will add visual by displaying the text on the page or just focus on voice interaction.

final idea


I tried to create a voice interface version of the Urban dictionary.

So If you say a word, it will show the description of the words.


I used p5.speech.js to recognize the speech and  I used Urban dictionary API to grab the descriptions.

Screen Shot 2018-02-07 at 2.04.48 PM

I tested with few words: Hello, what’s up, Amanda, curry, Good-bye.



Code: GitHub



1.This my favorite fashion brand’s website. I changed the price which I can never afford.

Screen Shot 2018-01-22 at 8.59.30 PM


I tried to think about things that will make me annoying.

2. This is my Instagram. I only switched the “Cancel” and “Log out” context.

I will try to press Log Out button to log out but I will never get logged out on my Instagram.

Screen Shot 2018-01-27 at 7.36.56 PM

3.  In Amazon website, I replaced the product images to loading icon gif which will never stop.

Screen Shot 2018-01-27 at 8.36.56 PM




Q – While walking around the city, What if the music plays automatically from the air ( or somewhere ) which fits each of the area’s mood?


Augmented Music   –  I love music, and people love music!

Instead of listening to their own music playlist, people can listen to the playlist which fits each area’s mood. For this prototype, I designated four music playlist into four different area. Music will be played in each zone automatically and randomly. Through this app, people will share the same experience listening to the same music at the same time at the same place as a public music player.


iphone6_mockup copy