Deep Learning Tutorial

Deep Learning Tutorial

Prerequisites

The only prerequisite to follow this Deep Learning Tutorial is your interest to learn it. Everything is secondary and comes along the way. Knowing any one of the programming languages like Python, R, Java or C++ would be sufficient, and you may choose any of the available deep learning platforms to put deep learning concepts into practice.

Introduction to Deep Learning

Deep Learning is a collection of those artificial neural network algorithms that are inspired by how a human brain is structured and is functioning. Human brain is one the powerful tools that is good at learning. And these deep learning techniques try to mimic the human brain with what we currently know about it.

Deep Learning is not as new as most of us are. They are brought into light by many researchers during 1970s and 1980s. But due to the lack of computational power and large amounts of data, the ideas of machine learning and deep learning were subdued. From the past decade, with the advancement in semiconductor technology, the computational cost has become very cheap and the data has grew during the industry years. This brought back the machine learning to lime light. Now, we have enough data to train a deep learning model with the very fast hardware in remarkably less time.

Many of the machine learning algorithms were proved to provide an increased performance with the increased data. But that rate has hit a threshold and additional data is no more providing an additional performance. The is the area where deep learning algorithms have shown their strength. The performance with deep learning algorithms is increasing with increased data much further unlike the traditional machine learning algorithms.

How do we mimic basic component of human brain ?

A quick browsing about human brain structure about half an hour might leave you with the terms like neuron, structure of a neuron, how neurons are connected to each other, and how signals are passed between them.

Following is a neuron of human brain (Source : Wiki Media) . Billion and Billions of these basic units along with some other materials constitute our brain.

We are not going into details of how this neuron works. But the basic intuition is that, the general idea of a human brain learning something is simplified down to what input(visual, audio, touch, smell) is fed to brain how neurons from one layer are connected to neurons in other layer, how the signal is transformed within the neuron, and how strong the connections are in between them. Dendrites fetch the input signal, nucleus or cell body transforms the input signal, axon takes the modified signal to the other neurons.

Following is the modelling of neuron used in artificial neural networks :

Neuron Model

What does deep mean in Deep Learning ?

On an abstract level,

Let us first see what a traditional neural network looks like.

Shallow Neural Network

This could also be referred to as a shallow learning, as there is only a single hidden layer between input and output.

  • Input layer consists of nodes which provide user known input to the neural network.
  • Hidden layer consists of nodes that model features from input data.
  • Output layer consists of a single node which aggregates the output of its previous layer to a single label (prediction).

Following is a deep neural network, where there are multiple hidden layers between input and output.

Deep Neural Network

The inputs are processed through multiple hidden layers, just like in brain.

Programming Languages for Deep Learning

Deep Learning Applications could be developed using any of Python, R, Java, C++, etc. Most of the core libraries of any Deep Learning framework is written in C++ for high performance and optimization. Those frameworks provide APIs for other programming languages like Python, R, Java etc. So, having expertise on any of those programming languages would be very helpful to start building your own Deep Learning Application.

In this Deep Learning Tutorial, we shall take Python programming for building Deep Learning Applications.

Install Anaconda Python – Anaconda is a freemium open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment.

Deep Learning Frameworks

Many deep learning frameworks have been created by the open source communities, organizations and companies, and some of them evolved to stable versions. Following are some of them :

  • Google Tensor Flow
  • Keras
  • Torch
  • PyTorch
  • Apache MXNet
  • Theano
  • Microsoft Cognitive Toolkit (CNTK)
  • DeepLearning4j

Deep Learning Tutorial Index

Following are the topics we shall go through in this Deep Learning Tutorial, with examples :

  • Artificial Neural Networks – ANN
    1. Artificial Neural Networks
    2. Build ANN – Example
    3. Evaluate ANN
    4. Tune ANN
  • Convolutional Neural Networks – CNN
    1. Convolutional Neural Networks
    2. Build CNN – Example
    3. Evaluate CNN
    4. Tune CNN
  • Recurrent Neural Networks – RNN
    1. Recurrent Neural Networks
    2. Vanishing Gradient Problem
    3. Build RNN
    4. Evaluate RNN
    5. Tune RNN
  • Self Organizing Maps – SOM
    1. Self Organizing Maps
    2. Build SOM
  • Boltzmann Machines
    1. Boltzmann Machine
    2. Building Boltzmann Machine
  • AutoEncoder
    1. AutoEncoder
    2. Build AutoEncoder