Exploration Month!

As the work on the presentation app attenuates, we set our mind on finding new cool ideas. Our next project is related to interactive music experience. On a more experimental side of the music experience spectrum, an interactive software could offer us freedom in participation of audio mixing and even creation. Inspired by our Product Studio projects, this new software tentatively extract information from a user's physical surroundings and uses it to trigger some changes to a pre-loaded track in order to personalize (or environmentalize😂) the overall experience. To introduce more assertive interactions, we are exploring the area of augmented reality to enable the input from a user themselves.

Unity game engine is a volatile platform for the development, as it has great advantage and is fairly easy to operate for the visuals, and it supports multiple operating systems for both desktop and mobile devices. In addition, the go-to AR library for Unity -- AR Foundation has many advanced features such as plane detection and face/body tracking for us to draw ideas from for user inputs, and will generate incredible fun for both the users and the developers.

We are on our way to set up the basis of the project, by learning to use the editor, the language and some cool examples. So far, we've got the sensor inputs and were able to hook them up with some of the audio parameters as well as some graphic artifacts. The product is funny enough, and we are aiming to get it less funny and more enjoyable!

Sensor readings and sensor triggered audio manipulation ⬇️

The vast array of built in effects can be modified procedurally in Unity3D engine.

AudioMixer allowing us to take in information from manual input or scripting and map the changes to audio effects in real time. ⬇️

Manual and auto placing of designer artifacts ⬇️

In the meantime, we are currently cooking up a model architecture design for the app in the background, in our head and on the paper, for systematically integrate all the fun functionalities into a unity (pun intended)! Stay tuned and we will soon reveal the mixed reality palace... excited!

Kelly