Deep Learning
20 Terms
A–Z Linked
Deep Learning
Concepts related to neural networks, representation learning, and advanced AI model architectures.
Activation Function
📘 Definition
An activation function determines whether a neuron should pass information to the next layer and introduces non-linearity into the network.
🪄 Simple Explanation
It decides how a neuron responds to its inputs.
🔑 Why It Matters
Without activation functions, deep learning would be much less powerful.
💡 Example
ReLU is a commonly used activation function.
See Also
Artificial Neural Network
📘 Definition
An artificial neural network is a computational model inspired by the structure of the human brain. It consists of connected nodes that learn patterns from data.
🪄 Simple Explanation
A neural network learns by connecting many simple processing units.
🔑 Why It Matters
Neural networks are the foundation of deep learning.
💡 Example
A handwriting recognition system can be built using an artificial neural network.
See Also
Attention Mechanism
📘 Definition
An attention mechanism enables a neural network to focus on the most relevant parts of the input when making predictions or generating content.
🪄 Simple Explanation
Attention helps AI concentrate on the most important information.
🔑 Why It Matters
It greatly improves language understanding and generation.
💡 Example
While translating a sentence, the model focuses on the words that matter most.
See Also
Backpropagation
📘 Definition
Backpropagation is the learning algorithm that updates neural network weights by sending prediction errors backwards through the network.
🪄 Simple Explanation
Backpropagation helps a network learn from its mistakes.
🔑 Why It Matters
It is fundamental to training deep neural networks.
💡 Example
After an incorrect prediction, the network adjusts its weights using backpropagation.
See Also
Batch
📘 Definition
A batch is a subset of the training data processed together before model parameters are updated.
🪄 Simple Explanation
A batch is a small group of training examples processed at one time.
🔑 Why It Matters
Batches improve training efficiency.
💡 Example
Instead of using all images at once, a model trains on batches of 64 images.
See Also
Convolutional Neural Network
📘 Definition
A Convolutional Neural Network (CNN) is a deep learning architecture designed especially for analysing images and visual data.
🪄 Simple Explanation
CNNs are neural networks specialised for images.
🔑 Why It Matters
CNNs power many computer vision applications.
💡 Example
Image classification and medical image analysis commonly use CNNs.
Deep Learning
📘 Definition
Deep learning is a branch of machine learning that uses artificial neural networks with many layers to learn complex patterns from large amounts of data.
🪄 Simple Explanation
Deep learning uses many layers of learning to solve difficult problems.
🔑 Why It Matters
It powers many of today's most advanced AI applications.
💡 Example
Image recognition and modern language models rely heavily on deep learning.
See Also
- Machine Learning
- Neural Network
- Large Language Model
Embedding
📘 Definition
An embedding is a numerical representation of text, images, or other data that captures meaning by placing similar concepts close together in a mathematical space.
🪄 Simple Explanation
Embeddings convert meaning into numbers.
🔑 Why It Matters
They help AI compare information based on meaning rather than exact wording.
💡 Example
A search engine finds similar documents by comparing embeddings.
See Also
Epoch
📘 Definition
An epoch is one complete pass through the entire training dataset during model training.
🪄 Simple Explanation
An epoch means the model has seen every training example once.
🔑 Why It Matters
Multiple epochs are usually needed for effective learning.
💡 Example
A model may train for 50 epochs before reaching good performance.
See Also
Forward Pass
📘 Definition
A forward pass is the process in which data moves through a neural network from the input layer to the output layer to produce a prediction.
🪄 Simple Explanation
A forward pass is how a network generates an answer.
🔑 Why It Matters
Every prediction begins with a forward pass.
💡 Example
An image is passed through all network layers before a label is predicted.
See Also
Gradient Descent
📘 Definition
Gradient descent is an optimisation algorithm that gradually reduces prediction errors by adjusting model parameters in the direction of lower loss.
🪄 Simple Explanation
It is a method for improving a model step by step.
🔑 Why It Matters
Most deep learning models rely on gradient descent.
💡 Example
Training repeatedly updates parameters until errors become smaller.
See Also
Input Layer
📘 Definition
The input layer is the first layer of a neural network that receives raw data before it is processed by hidden layers.
🪄 Simple Explanation
The input layer accepts information for the network.
🔑 Why It Matters
It is the starting point for every neural network.
💡 Example
Pixels from an image enter a neural network through the input layer.
See Also
Layer
📘 Definition
A layer is a group of neurons that processes information before passing it to the next stage of a neural network.
🪄 Simple Explanation
Layers gradually transform raw data into meaningful patterns.
🔑 Why It Matters
Multiple layers enable deep learning models to solve difficult problems.
💡 Example
Early layers detect edges while deeper layers recognise complete objects in images.
See Also
Mini-Batch
📘 Definition
A mini-batch is a small portion of the training dataset used during each update of a learning algorithm.
🪄 Simple Explanation
Mini-batches balance speed and learning quality.
🔑 Why It Matters
They are widely used in modern deep learning.
💡 Example
A model may process 32 examples in each mini-batch.
Neuron
📘 Definition
A neuron is the basic computational unit within an artificial neural network. It receives inputs, performs calculations, and passes information to other neurons.
🪄 Simple Explanation
A neuron is a small processing unit inside a neural network.
🔑 Why It Matters
Millions of neurons work together to solve complex tasks.
💡 Example
Each neuron contributes a small part of an image recognition decision.
Output Layer
📘 Definition
The output layer is the final layer of a neural network that produces the model's prediction or decision.
🪄 Simple Explanation
The output layer provides the final answer.
🔑 Why It Matters
It converts learned patterns into usable predictions.
💡 Example
A digit recognition model outputs the predicted number through the output layer.
See Also
Recurrent Neural Network
📘 Definition
A Recurrent Neural Network (RNN) is a neural network architecture designed to process sequential data by retaining information from previous inputs.
🪄 Simple Explanation
RNNs remember earlier information in a sequence.
🔑 Why It Matters
They were an important step in language and speech processing before transformers.
💡 Example
Speech recognition systems traditionally used RNNs.
See Also
Transformer
📘 Definition
A transformer is a deep learning architecture that processes information using attention mechanisms, enabling efficient handling of long sequences. It forms the foundation of modern large language models.
🪄 Simple Explanation
Transformers allow AI to understand relationships across long pieces of information.
🔑 Why It Matters
They power today's most capable language and multimodal models.
💡 Example
ChatGPT and many modern AI systems are based on transformers.
Weight
📘 Definition
A weight is a numerical value that determines the importance of a connection between two neurons in a neural network.
🪄 Simple Explanation
Weights control how strongly neurons influence each other.
🔑 Why It Matters
Learning mainly involves adjusting weights.
💡 Example
Training changes weights until predictions improve.