AI basics for AI Engineering

▶

To work as an AI Engineer, a foundational understanding of AI concepts is essential. This includes understanding how AI, Machine Learning, and Deep Learning relate to each other, how neural networks work at a conceptual level, and how modern architectures like Transformers have revolutionized AI capabilities.

AI, Machine Learning, and Deep Learning

Artificial Intelligence (AI)

AI enables adding intelligence to machines, allowing them to perform tasks autonomously rather than simply following pre-coded instructions.

Machine Learning (ML)

A method within AI where a machine is trained on huge amounts of data to learn patterns and implement intelligent tasks without being explicitly programmed for each scenario.

Deep Learning (DL)

A subset of Machine Learning that uses neural networks with multiple layers to mimic human thinking and processing. This is the foundation of modern AI models like LLMs (Large Language Models).

The Fundamental Shift in Development

AI fundamentally shifts development by replacing manual rule-writing with learned decision-making patterns.

Neural Networks

Inspiration from the Human Brain

Human decision-making involves processing inputs through neurons to produce a response. This biological concept inspired the development of artificial neural networks.

• Multiple inputs → Processing → One output
• Decisions emerge from accumulated learned experiences, not programmed instructions

Structure of a Neural Network

Shallow vs Deep Networks

The number of hidden layers determines whether it's shallow or deep learning. More layers enable understanding of more complex patterns.

Weights — How Neurons Make Decisions?

Weights determine the importance of each input in a neural network's decision-making process.

Weight Range: -1 to 1

How It Works

• Each connection between neurons has a weight assigned to it.
• The network processes weighted inputs from multiple neurons and combines them to make decisions.
• During training, weights are adjusted to improve accuracy, similar to how humans refine decisions based on experience.

As an AI Engineer, understanding what weights and neurons are is important. Understanding the deep mathematical details is not required.

Transformers — The Architecture Behind Modern AI

What Are Transformers?

Transformers are a neural network architecture that revolutionized AI by enabling models to understand context and process information in parallel rather than sequentially.

• Used in models like GPT (Generative Pre-trained Transformer), including ChatGPT
• Introduced the concept of self-attention

The Problem Before Transformers

Previous models like RNN (Recurrent Neural Networks) processed text one word at a time (sequentially).

How Transformers Solve This

Self-Attention Mechanism

The key innovation in Transformers is self-attention, the ability for each word in a sentence to evaluate its relationship and relevance to every other word.

Example

Sentence: "The cat sat on the mat because it was tired."

• Self-attention allows the model to understand that "it" refers to "the cat" by examining relationships between all words simultaneously.
• Each word gets assigned a weight relative to other words, indicating how much attention it should pay to them.

How Self-Attention Works?

1. For each word, the model asks: "How important is every other word to understanding this word?"
2. Weights are assigned to all word relationships
3. Words with higher relevance receive more attention
4. This enables contextual understanding of language

Previous Models vs Transformers

Summary

• AI adds intelligence to machines, ML trains on data, Deep Learning uses multi-layer neural networks.
• Neural networks are inspired by human brain neurons, they process weighted inputs through layers to produce outputs.
• Weights (ranging from -1 to 1) determine how much importance each input has in decision-making.
• Transformers revolutionized AI by introducing self-attention, enabling models to understand context and relationships between all words simultaneously.
• Previous sequential models (RNN) lost context in long text, while transformers solve this by processing everything in parallel.
• For AI Engineering, conceptual understanding of these fundamentals is sufficient, deep mathematical expertise is not required.