How to Learn Deep Learning | Guide to CNNs, LSTMs, RNNs, GANs, RBFNs, MLPs, SOMs & More

Types of Deep Learning Algorithms

There are several algorithms that deep learning models can use. Some algorithms are better suited for specific tasks while others might be better for others.  

There is no perfect network and choosing the appropriate one for the task at hand requires knowledge and understanding of the types of algorithms.

Here are ten of the top algorithms in use today:

1. Convolutional Neural Networks (CNNs)

Consisting of multi-layers, CNN's, also referred to as ConvNets are primarily used for processing images. It was first developed in 1988 at Bell Labs' by Yann LeCun and was dubbed LeNet when it came out. Its primary purpose back then was to map and recognize numerals, characters, and ZIP codes.

Today, its application can be widely seen in medical imaging, satellite imaging, detailed image processing, object detection, and anomaly detection.

2. Long Short Term Memory Networks (LSTMs)

Long short term memory networks or LSTMs learn from retaining and recalling past information. Its default behavior is to keep information stored for extended periods of time that are ready to be accessed at any time.

LSTMs are very useful in time-series forecasting due to their capability to draw from past data inputs. Time series forecasting is the use of information from historical data. LSTMs can predict future behavior or activity. This is especially useful in analyzing trends and predicting issues of seasons.

LSTMs can be found applied to the fields of music creation and composition, pharmaceutical drug development, and speech recognition.

3. RNNs

LTSMs are a type of recurrent neural network or RNNs. Due to their internal memory, RNNs are capable of memorizing and keeping previous data inputs. RNNs are typically applied to time-series related tasks and analysis, image-captioning, machine translation, handwriting recognition, and natural language processing.

4. Generative Adversarial Networks (GANs)

Generative adversarial networks or GANs are deep learning algorithms that build new similar data instances based on the data used for training.

GANs have contributed huge breakthroughs in the field of imaging, specifically astronomical image capture, and dark matter research due to Generative Adversarial Networks' capability to simulate gravitational lensing.  GANs have been utilized in rendering and creating 3D images and objects like in cartoons, video games, and photographs.

5. Radial Basis Function Networks (RBFNs)

RBFNs are commonly used for time-series forecasting, classifying, and regression.  RBFNs have feedforward neural networks that operate in radial basis functions. They are composed of three layers: main layers, the input layer, the output layer, and between the two, the hidden layers.

6. Multilayer Perceptrons (MLPs)

Belonging to a class of feedforward neural networks, MLPs have multiple layers of perceptrons that act as activation functions. 

The input layer and output layer of MLPs are connected entirely but multiple hidden layers may still be between the two layers. MLPs are usually the best place to start your journey in learning about deep learning. 

7. Self Organizing Maps (SOMs)

Through the self-organization of neural networks, SOMs are capable of data visualization that helps reduce the dimension of data. Self-Organizing Maps were invented by Professor Teuvo Kohonen.

Information that has high dimensional data is very hard for humans to visualize, SOMs aid in helping people process information with high-dimensions.

8. Deep Belief Networks (DBNs)

DBNs are commonly used for image, video recognition, and motion capture technology. DBN are generative models that have multiple layers that are in constant communication with each other. Passing on the previous information to the next layer. 

DBNs are a stack of Boltzmann Machines with connections between the layers, and each RBM layer communicates with both the previous and subsequent layers.

9. Restricted Boltzmann Machines( RBMs)

RBMs were developed by Geoffrey Hinton. Using probability distribution from a set of inputs to learn, Restricted Boltzmann Machines are stochastic neural networks. RBMs are a subset and building block of DBNs.  Common applications of RMBs are classification, collaborative filtering, dimensionality reduction,  regression, and topic modeling.

10. Autoencoders

Designed by Geoffrey Hinton in the 1980s, Autoencoders were primarily created to take on unsupervised tasks. 

This algorithm is a feedforward neural network, which means autoencoders have similar inputs and outputs.  The artificial neural networks copy the input layer information and pass it on to the output layer.

Autoencoders can be seen used in predictions, hi-fidelity image processing, and pharmaceutical research.