There are many deep learning libraries that are becoming popular among the developer community such as Theano, Torch, Caffe, etc. Deeplearning4J is an open source and distributed deep learning library targeted at Java Virtual Machine (JVM). This article provides an introduction to its capabilities and features.
Machine learning has led to a tremendous shift in the way we create algorithms to solve problems. It has made developers and researchers shift away from the traditional step-by-step solutions approach to a holistic model, which in many aspects mimics the methods that biological organisms employ while solving problems or acquiring a new skill. These days, machine learning models are employed in all those sectors that were originally difficult to deal with by the traditional algorithmic approaches. Real-time object detection from videos, classification and prediction systems are some of the tasks performed with the help of machine learning. The popular quote by Prof. Tom Mitchell, who is a pioneer in shaping the domain, clearly defines machine learning as follows: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.”
Deep learning
Deep learning is a recent improvement in the machine learning domain. A basic requirement of machine learning is to identify and build a feature set. This task is generally carried out by experts in that domain, manually. So, for each problem domain, customised features need to be built by expert humans. This often creates a bottleneck in the overall process flow.
Deep learning differs from traditional machine learning in the way features are built. It attempts to build the features automatically from the given large data set. Note the emphasis on the word ‘large’. For deep learning to work at a decent accuracy level, we need considerably large sized data sets. This is significant, because the machine requires a fairly large amount of data in order to detect the discriminative features of a set. Feature engineering is an important component for the successful implementation of any machine learning related project.
Deep learning frameworks/libraries
There are many popular libraries that can be used to perform deep learning tasks. Some of them are listed below:
- Caffe
- Theano
- Torch
- Deeplearning4J
- Mocha
A detailed comparison of deep learning frameworks is available at https://deeplearning4j.org/compare-dl4j-torch7-pylearn.
Deeplearning4j (DL4J)
This article explores the Deeplearning4J (DL4J) library. DL4J has been developed in Java and is targeted at Java Virtual Machine (JVM). An interesting feature of Deeplearning4J is the ability to build fast prototypes. The attributes of DL4J are listed below.
- Distributed: The training process can be accelerated because DL4J is distributed by nature. It can harness multiple GPUs, and the performance is on par with other major libraries such as Caffe.
- Open Source: DL4J is a production quality open source library available for performing deep learning tasks. The active community of developers keeps DL4J fresh.
- Interoperability: DL4J is written in Java, and hence all the JVM based languages such as Scala, Clojure and Kotlin are compatible. Interoperability with other major frameworks such as Caffe and Torch can be achieved through Keras.
DL4J features
The major features of DL4J are:
- Java, Scala and Python APIs
- Parallel training through iterative reduce
- Scalable with Hadoop
- Distributed CPU and GPU support
DL4J incorporates both a distributed, multi-threaded deep learning framework and a single-threaded deep learning framework. Another important feature of DL4J is that it is the first deep learning framework adopted for a microservice architecture.
Prerequisites
The prerequisites to start development with DL4J are listed below:
- Java 1.7 or above (DL4J supports only the 64-bit versions).
- Apache Maven: This is a dependency management and build tool. The Maven version in your system can be checked with the following command:
mvn --version
If you are new to Maven, an excellent ‘getting started’ guide (Maven in Five Minutes) is available at http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
- The official document recommends the use of InelliJ IDE or Eclipse. The community edition of InelliJ IDE can be downloaded from the official website.
- Git: Get the latest version with the following command:
$ git clone git://git.kernel.org/pub/scm/git/git.git
Using the DL4J examples
To download and use the examples from DL4J, use the following commands:
$ git clone https://github.com/deeplearning4j/dl4j-examples.git $ cd dl4j-examples/ $ mvn clean install
1. Run IntelliJ. Choose the Import Project option. Select the folder ‘dl4j-example’.
2. Select ‘Import Project from external model’ and make sure that Maven is chosen.
3. Follow the wizard’s instructions. Click Finish to complete. It might take some time to get all the dependencies in the local system.
4. Select a particular example and right-click the file to run it.
The deep neural networks are made up of multiple layers. The MultiLayerConfiguration with parameters is customised to suit the requirements of the current problem. The hyperparameter variable decides the learning behaviour of the network. These parameters include the following:
- Number of times to update the weights of the model
- Initialisation mechanism for these weights
- The type of activation function to link with the nodes
- The specific optimisation algorithm to be used
- The speed with which the network should learn
A sample configuration is shown in the following code snippet:
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder() .iterations(1) .weightInit(WeightInit.XAVIER) .activation(“relu”) .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT) .learningRate(0.05) // ... other hyperparameters .list() .backprop(true) .build();
A new layer can be added by invoking layer() on NeuralNetConfiguration.Builder(). In this process, we need to specify the following:
Location (order) where the layer has to be added
The number of input and output nodes (nIn and nOut)
The type of layer
layer(0, new DenseLayer.Builder().nIn(784).nOut(250) .build())
After completing the configuration process, the training of the model can be carried out with the following command:
model.fit
DL4J’s neural networks
DL4J supports many powerful neural networks. Some of them are listed below:
- Restricted Boltzmann machines
- Convolutional nets
- Recurrent nets
- Deep-belief networks
- Stacked denoising autoencoders
The reason for choosing JVM
Most of the deep learning/machine learning libraries are in Python. But DL4J is based on JVM. One may ask why this deliberate choice was made. The official documentation lists the major reasons for choosing the Java Platform:
- Many large government organisations and companies have Java as their major platform. There are millions of Java developers and Java is the largest programming language, as of now.
- Java is the basis of many powerful techniques/tools such as Hadoop, ElasticSearch, Lucene and Hive.
- One prime complaint against Python is its slow speed. Java is definitely quicker than Python. Hence, when handling projects with massive data sets, Java may be a better choice.
- Java is inherently secure and cross-platform. It can be easily used on all major platforms such as Linux, Windows, OSX and Android.
DL4J on Android
Deeplearning4j can be used with Android mobile phones as well. The prerequisites for running it on Android are:
- Emulator (with API level 21 or above) or a device
- Android Studio 2.2 or later
The following dependencies must be added to the build.gradle file:
compile ‘org.deeplearning4j:deeplearning4j-core:0.8.0’ compile ‘org.nd4j:nd4j-native:0.8.0’ compile ‘org.nd4j:nd4j-native:0.8.0:android-x86’ compile ‘org.nd4j:nd4j-native:0.8.0:android-arm’ compile ‘org.bytedeco.javacpp-presets:openblas:0.2.19-1.3:android-x86’ compile ‘org.bytedeco.javacpp-presets:openblas:0.2.19-1.3:android-arm’
A clean and easy-to-follow tutorial is available at https://deeplearning4j.org/android.
This article has provided an introduction to the Deeplearning4J library. If you are interested in exploring more about deep learning, there are many interesting online courses and resources. A compiled list of links is available at https://deeplearning4j.org/deeplearningforbeginners.html.
