Fortu

fortuity | a chance occurrence, the state of being controlled by chance rather than design

Fortu is a contextual speaker that detects objects in proximity and plays music based on the objects identified.

Focusing on how ‘machines’ could help people cope with spending more time at home in the context of social distancing, the team explored creating serendipitous moments through music as seen by the ‘eyes of the machine’.

The original prototype used COCO-SSD, a pre-trained TensorFlow object detection model accessed via RunwayML, with detection results generated from the model parsing to Processing. A list of preloaded songs based on detection results would then be played. A web-based application was further developed using ml5.js and p5.js where calls were made to the Spotify API, showing user song recommendations based on the detection results. 

This project was developed over a 3-day exploration during the Machine Learning class where students were exposed to different machine learning tools such Google’s Teachable Machine, RunwayML, and ml5.js, as well as the concept of using algorithms as a material to prototype.