Web applications have come a long way. From the simple data capturing applications of the past they are now complex and dynamic. To enrich their capabilities further, it would be great to harness their machine learning potential. This article explores four Web browser based machine learning libraries and their features.
Web applications enable organisations to cater to the needs of a large number of users, distributed across various geographical locations, without requiring any local installation process. Upgrading Web applications is also comparatively simple, as the code needs to be updated only in the server. It is very tedious to update a desktop application in each installed location. Due to the advantages of scale and ease of maintenance, Web applications are now preferred to their desktop counterparts.
In the past, Web applications were comparatively simpler in nature. They mostly functioned as data collection platforms with simple interfaces. With the prolific growth in Web technologies, these apps have evolved into complex and dynamic entities.
Machine learning (ML) is evolving rapidly and is being applied to various domains. Web apps, too, can be enriched with ML capabilities and become more powerful. Machine learning can be incorporated into Web applications in two ways:
- Machine learning as a server side component
- Machine learning as a client side component
There are pros and cons for both these approaches. The server side applications have an advantage of better processing capabilities with a bigger memory. At the same time, one of the bottlenecks in server side ML is the delay due to network traffic. Each request with ML functionality needs to be communicated to the server, where it has to be processed, and the results should be returned to the client. These steps introduce latency in the application, which is best avoided.
To overcome this problem, it would be better to run the ML components in the browser itself. Thereby, the latency due to the network round-trip can be avoided. As the client devices these days have better processing capabilities, the ML components can be executed with an acceptable performance. This article explores four browser based ML libraries (Figure 1).
TensorFlow.js
TensorFlow is a popular machine learning library from Google. It makes the ML implementation effective. The TensorFlow.js is a JavaScript library, which can be used to implement ML models in Web browsers. Both training and deploying of the models can be done through TensorFlow.js, which has the following capabilities:
- The ML models can be built and trained from scratch with powerful APIs.
- It can be used to run the existing TensorFlow models inside the browser.
- The existing models can be retrained with TensorFlow.js.
The official documentation has provided many working examples (real-time human estimation, etc) of TensorFlow.js implementations that will give you an idea about the capabilities of this powerful ML library (https://js.tensorflow.org/).
If you are familiar with the working of Tensor, layers, optimisation and loss functions, then using the TensorFlow.js library is very simple.
TensorFlow.js can be included in Web applications in many ways. The simplest approach is to use a script tag, as follows:
<script src=”https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.14.2/dist/tf.min.js”></script>
Another approach is to use NPM, as follows:
npm install @tensorflow/tfjs
If you want to convert an existing model, the tensorflowjs_converter can be used. The following code is used to convert the SavedModel:
tensorflowjs_converter \ --input_format=tf_saved_model \ --output_node_names=’MobilenetV1/Predictions/Reshape_1’ \ --saved_model_tags=serve \ /mobilenet/saved_model \ /mobilenet/web_model
In the case of a FrozenModel, the following command can be used:
tensorflowjs_converter \ --input_format=tf_frozen_model \ --output_node_names=’MobilenetV1/Predictions/Reshape_1’ \ /mobilenet/frozen_model.pb \ /mobilenet/web_model
To run this conversion, you require a Python environment. The converter can be installed with Pip:
pip install tensroflowjs
The conversion script generates three files:
- The dataflow graph (web_model.pb)
- The weight manifest file (weights_manifest.json)
- A collection of binary weight files (group1-shard\*of\*)
The loading and running of the model in the browser can be done using the following code snippet:
import * as tf from ‘@tensorflow/tfjs’;
import {loadFrozenModel} from ‘@tensorflow/tfjs-converter’;
const MODEL_URL = ‘https://.../mobilenet/web_model.pb’;
const WEIGHTS_URL = ‘https://.../mobilenet/weights_manifest.json’; const model = await loadFrozenModel(MODEL_URL, WEIGHTS_URL);
const cat = document.getElementById(‘cat’);
model.execute({input: tf.fromPixels(cat)});
(Source: https://js.tensorflow.org/tutorials/import-saved-model.html)
The complete source code and execution procedure can be gathered from https://github.com/tensorflow/tfjs-converter/tree/master/demo/mobilenet.
A detailed tutorial on TensorFlow.js is available at https://js.tensorflow.org/.
ml5.js
An understanding of ML concepts is necessary to use libraries such as TensorFlow.js effectively. If you are not interested in the internals of how things work but you only want ML applications, then give ml5.js a try. The official documentation claims that this library is made to enable a broader audience get involved in machine learning. ml5.js is built on top of TensorFlow.js and is hugely inspired by ‘processing’ and p5.js.
Setting up your environment to use ml5.js is very simple. Just use a <script> tag to link the external JavaScript file.
<!DOCTYPE html> <html> <head> <title>Machine Learning with ml5.js</title> <script src=”https://unpkg.com/ml5@0.1.3/dist/ml5.min.js”> </script> </head> </html>
A simple image classification example is shown below:
<body>
<h1>Image classification using MobileNet</h1>
<p>The MobileNet model labeled this as
<span id=”result”>...</span> with a confidence of
<span id=”probability”>...</span></p>
<img src=”https://ml5js.org/docs/assets/img/bird.jpg”
crossorigin=”anonymous” id=”image” width=”400”>
<script>
// The image we want to classify
const image = document.getElementById(‘image’);
// The result tag in the HTML
const result = document.getElementById(‘result’);
// The probability tag in the HTML
const probability = document.getElementById(‘probability’);
// Initialize the Image Classifier method with MobileNet
const classifier = ml5.imageClassifier(‘MobileNet’, function() {
console.log(‘Model Loaded!’);
});
// Make a prediction with the selected image
// This will return an array with a default of 10 options with their probabilities
classifier.predict(image, function(err, results) {
result.innerText = results[0].className;
probability.innerText = results[0].probability.toFixed(4);
});
</script>
</body>
</html>
(Source: https://ml5js.org/docs/getting-started.html)
Just save this file and open it in a browser. Your working demo is ready. There are no dependencies to configure. It can be inferred from the above code that ml5.js has direct functions available to perform the classification task. We just need to specify the model and the results are ready.
The official documentation has listed various code examples such as a style transfer, a Pong game in the browser, and a text editor with ML based suggestions (https://ml5js.org/experiments).
Brain.js
This is a useful JavaScript ML library. It is used to build neural networks in the browser. The ‘node’ based configuration of Brain.js can be done as follows:
npm install brain.js
…or by using the following command:
yarn add brain.js
The types of neural networks supported by Brain.js are given below:
- Feed forward neural network: brain.NeuralNetwork
- FF neural networks with back-propagation: brain.NeuralNetworkGPU
- Time step recurrent neural network (NN) or RNN: brain.recurrent.RNNTimeStep
- Time step long/short term memory (LSTM) NN: brain.recurrent.LSTMTimeStep
- Time step gated recurrent unit (GRU): brain.recurrent.GRUTimeStep
- RNN: brain.recurrent.RNN
- LSTM neural network: brain.recurrent.LSTM
- GRU: brain.recurrent.GRU
- Further details on Brain.js can be obtained from https://github.com/BrainJS/brain.js.
ConvNetJS
This is an implementation of neural networks in JavaScript. ConvNetJS supports the following:
- Common NN modules (full connected layers and non-linearities)
- Classification and regression
- Can train convolutional neural networks for processing images
- A reinforcement learning module based on Deep Q learning
ConvNetJS has no external dependencies and hence is very simple to handle. The official documentation lists many working demos. A CIFAR-10 demo is provided at https://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html.
Importing ConvNetJS is done with the following simple script tag:
<!-- import convnetjs library --> <script src=”convnet-min.js”></script>
Multi-layer networks can be built with the following code sequence:
var layer_defs = [];
// input layer of size 1x1x2 (all volumes are 3D)
layer_defs.push({type:’input’, out_sx:1, out_sy:1, out_depth:2});
// some fully connected layers
layer_defs.push({type:’fc’, num_neurons:20, activation:’relu’});
layer_defs.push({type:’fc’, num_neurons:20, activation:’relu’});
// a softmax classifier predicting probabilities for two classes: 0,1
layer_defs.push({type:’softmax’, num_classes:2});
// create a net out of it
var net = new convnetjs.Net();
net.makeLayers(layer_defs);
(Source: https://cs.stanford.edu/people/karpathy/convnetjs/started.html)
The libraries mentioned in this article enable the developer to incorporate ML into Web applications without many dependencies. Browser based ML can harness the untapped hardware potential on the client side instead of loading the server heavily with a huge number of requests. As these libraries advance further, future Web applications will be able to provide various smart features based on ML algorithms that will benefit users greatly.
