ml5.js is a library that provides easy integration of machine learning algorithms and models in the browser and is built on TensorFlow.js.

In this example, you can control the speed of music in a simple way with the interaction of an image classification machine learning model.

Getting Started

Since ml5.js runs on the browser you need to include the scripts to the header of your html file, including p5.js:

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.8.0/p5.min.js"></script>   
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.8.0/addons/p5.dom.min.js"></script>
<script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>

Add the music into your html file, in this example I am playing The Symphony №5 in C minor of Ludwig van Beethoven.

Let’s create the model

Teachable Machine is a fast and easy way to create machine learning models for your sites, you just need to collect your examples, then train the model and export the model.

For my model, I have created 3 clasess, left, right and nothing:

collecting examples for the model
collecting examples for the model

Once finished the training, get the model as a sharable link.

Add the model to your js file

First, create some variables for video and model:

Set up

Now, let’s create the functions:

  1. In preload() function, we load the model to be sure is ready before we can apply the image classification.
  2. In setup() function we create the canvas and flip the video so it is rendered as a mirror.
  3. In classifyVideo() function, we classify the flipped video again. In gotResult() function, we use two parameters, one to log the error if there is an error or to return the results of the classifier.
  4. In draw() function, we render the video, the labels and the commands to set the rate at which the media is being played back.

“The audio is muted when the fast forward or slow motion is outside a useful range (for example, Gecko mutes the sound outside the range 0.25 to 5.0)”

Play!

Full-Stack Web Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store