Calculus for AI — Derivatives, Partial Derivatives & Chain Rule

Continuity

Before derivatives, you must understand continuity.

A function is continuous if you can draw its graph without lifting your pen.

In simple words:

No breaks. No holes. No jumps.

Intuition: Road without breaks

Imagine driving on a road.

You can drive from start to finish without interruption.

Now imagine a broken bridge.

You must stop.

That gap is called a discontinuity.

Mathematical definition

A function is continuous at x = a if these three conditions are true:

1. f(a) exists

2. limₓ → ₐ f(x) exists

3. limₓ → ₐ f(x) = f(a)

Don't memorize yet—understand the meaning.

Condition	Meaning
f(a) exists	The function has a value
Limit exists	Left = Right
Both equal	No gap

If all three are true → Continuous ✅

Example 1: Continuous Function

Function:

Find continuity at x = 3.

Step 1

Step 2

Step 3

Both are equal.

Therefore: Continuous

Graph

One smooth line.

No breaks.

Example 2: Hole (Removable Discontinuity)

Suppose the graph has a missing point.

Everything approaches the point, but the value is missing.

Example:

But:

The limit exists.

The function value does not.

This is called a removable discontinuity.

Why removable?

Because we can simply define:

The graph becomes continuous.

Example 3: Jump Discontinuity

Left side and right side are different.

Suppose:

Since:

The limit does not exist.

Therefore the function is not continuous.

Think of it like stairs.

You suddenly jump to another level.

Example 4: Infinite Discontinuity

Vertical asymptote.

Function:

As x approaches 2:

• Left → −∞
• Right → +∞

The graph shoots upward.

This is not continuous.

ML Use Case 1: Activation Functions

Neural networks prefer smooth functions.

ReLU

ReLU has no jump.

It's continuous, although its slope changes at zero.

This is why it works well in deep learning.

ML Use Case 2: Loss Curve

Training loss should decrease smoothly.

Epoch	Loss
1	2.8
2	2.0
3	1.5
4	1.1
5	0.9

A smooth loss curve indicates stable learning.

Sudden jumps may indicate:

• Too high learning rate
• Bad data
• Numerical instability

ML Use Case 3: Real Estate Price Model

Suppose property area changes.

Area	Price
1000	60L
1100	65L
1200	70L
1300	75L

The relationship is smooth.

If adding 1 sq.ft suddenly changes price by ₹50 lakh, the function is unrealistic.

Continuous functions model real-world behavior better.

Python Example

def price(area):
    return 5000 * area

print(price(1000))
print(price(1001))

Output:

5000000
5005000

A tiny input change produces a tiny output change.

This is continuity.

Summary Table

Type	Continuous?	Example
Straight line	✅	y = 2x + 1
Smooth curve	✅	y = x²
Hole	❌	Missing point
Jump	❌	Stair function
Vertical asymptote	❌	1/(x−2)

Memory Trick

Limit asks: What value are we approaching?

Continuity asks: Can I reach that value without any break?

Interview Question

Q: Why is continuity important in Machine Learning?

Answer: Most optimization algorithms (like Gradient Descent) assume the loss function changes smoothly. Continuous functions allow gradients to guide the model toward better solutions without unpredictable jumps or breaks.

Next lesson: Derivatives.

Functions

A function is the foundation of Calculus and Machine Learning.

A function is a rule that takes an input and produces exactly one output.

Think of it as a machine.

• Input → x
• Function → Rule
• Output → y

What is a function?

A function connects input and output.

Read it as:

y is a function of x

Example:

This means:

1. Take x
2. Multiply by 2
3. Add 1

Step-by-step example

If:

Find the output when x = 3

Answer: 7

Input (x)	Output f(x)
1	3
2	5
3	7
4	9

The rule never changes.

Visual graph

Every input has one unique output.

Why are functions important in ML?

Every Machine Learning model is a function.

The model receives data and returns a prediction.

Example: House price prediction

Input features:

ML function:

Output:

₹82 lakh

The model is simply learning the best function.

AI Use Case 1: ChatGPT

ChatGPT is also a function.

Input:

"Explain vectors"

Function:

LLM

Output:

A response about vectors.

Every prompt becomes an output through a learned function.

AI Use Case 2: Image Classification

A cat image enters the model.

Stage	Value
Input	Image
AI Function	f(x)
Output	Cat (99.2%)

The CNN learns the function that maps pixels → labels.

AI Use Case 3: Fraud Detection

Bank transaction:

Feature	Value
Amount	₹45,000
Time	2:15 AM
Country	Unknown

These become the input vector.

Function:

Output:

Fraud Probability = 0.94

Again, the AI is learning a function.

Types of Functions (Calculus)

Type	Example	Shape
Linear	f(x) = 2x + 1	Straight line
Quadratic	f(x) = x²	Curved graph
Exponential	f(x) = 2^x	Fast growth
Logarithmic	f(x) = log(x)	Slow growth

You'll study each later.

Python Example

# Function definition
def f(x):
    return 2 * x + 1

print(f(3))
print(f(10))

Output:

7
21

f() is literally a Python function implementing the mathematical function.

Real Estate Example

Property features:

Feature	Value
BHK	3
Area	1500
Parking	2

Represented as:

AI valuation function:

Output:

₹1.25 Crore

Your future property valuation model is fundamentally a function approximation problem.

Summary

Term	Meaning
Input	x
Function	Rule / Model
Output	y
Notation	y = f(x)
ML Meaning	Model maps features → prediction

Memory Trick

A Machine Learning model is simply a mathematical function that learns the best rule to convert inputs into outputs.

This is the foundation you'll need before moving to Lesson 2: Limits, where Calculus begins analyzing how functions behave as inputs change.

Interview question

Q: Why is every Machine Learning model a function?

Answer: Because a model maps inputs (features) to a single output (prediction). Training means finding the best function f(x) that minimizes error on real data.

Limits

A limit is the first real idea in calculus. Before learning derivatives, you must understand what happens when a value gets very close to another value.

Limit = The value a function approaches as the input gets closer to a specific number.

Think of it as approaching, not necessarily reaching.

Intuition: A car approaching a traffic signal

Imagine a car moving toward a traffic light at 100 m.

The car reaches:

• 80 m
• 90 m
• 95 m
• 99 m
• 99.9 m

It is getting closer and closer to 100.

That idea is exactly what a limit means.

Mathematical definition

If a function is:

We write the limit as:

Read it as:

The limit of f(x) as x approaches 3

Notice it says approaches, not equals.

Step-by-step example

Function:

Let's approach 3.

x	f(x)
2.5	6
2.9	6.8
2.99	6.98
2.999	6.998

As x gets closer to 3, the output gets closer to 7.

Therefore:

Even before reaching 3, we already know the answer.

Visual graph

The red circle shows the value the function approaches.

Left-hand limit (LHL)

Approaching from the left.

Think:

Examples:

• 2.5
• 2.8
• 2.99

All move toward 3.

Notation:

Right-hand limit (RHL)

Approaching from the right.

Examples:

• 3.5
• 3.1
• 3.01

Notation:

When does a limit exist?

A limit exists only if both sides are equal.

Example:

Therefore:

When does a limit NOT exist?

Suppose:

Left side:

Right side:

Since they are different:

This is called a jump discontinuity.

ML Use Case 1: Loss Function

During training, the loss decreases gradually.

Epoch	Loss
1	2.8
2	1.9
3	1.2
4	0.8
5	0.55

As epochs increase:

• Loss approaches 0
• The optimizer is trying to minimize the function

This is the idea of a limit: the model approaches the minimum.

ML Use Case 2: Gradient Descent

Imagine the loss curve.

The model moves closer and closer to the minimum.

It may never hit the exact minimum mathematically, but it approaches it.

Limits describe this behavior.

ML Use Case 3: Learning Rate

Suppose learning rate decreases.

Epoch	Learning Rate
1	0.100
10	0.050
50	0.010
100	0.001

As epochs grow, the learning rate approaches 0.

This is another practical limit.

Python Example

def f(x):
    return 2 * x + 1

values = [2.9, 2.99, 2.999]

for x in values:
    print(x, f(x))

Output:

2.9    6.8
2.99   6.98
2.999  6.998

Notice the outputs approach 7.

Summary Table

Concept	Meaning
Limit	Value a function approaches
LHL	Approach from the left
RHL	Approach from the right
Limit Exists	LHL = RHL
No Limit	LHL ≠ RHL

Memory Trick

Function asks: "What is the output?" Limit asks: "What output are we approaching?"

This idea is the bridge to Lesson 3: Continuity, where you'll learn when functions have no breaks or jumps.

Interview question

Q: When does a limit exist?

Answer: A limit exists only when the left-hand limit equals the right-hand limit. If the two sides approach different values, the overall limit does not exist.

Derivatives

This is the most important calculus concept for Machine Learning. Every ML model learns using derivatives.

Derivative = How fast the output changes when the input changes.

In simple words:

Derivative measures the rate of change (slope).

Intuition: Speedometer of a car

Imagine you're driving.

The speedometer doesn't tell where you are.

It tells how fast you're changing position.

A derivative does exactly the same thing:

• Position → Function
• Speed → Derivative

What does “rate of change” mean?

Suppose salary depends on experience.

Experience (Years)	Salary (Lakh)
1	4
2	5
3	6
4	7

Every extra year increases salary by 1 lakh.

The derivative is:

Meaning:

Salary increases 1 lakh for every 1 year.

Graph intuition

A derivative is the slope of the curve.

The red tangent line touches the curve at one point.

Its slope is the derivative.

Mathematical definition

If:

Derivative is written as:

Or:

Both mean the same thing.

Read it as:

The rate of change of y with respect to x.

Easy example

Function:

Let's see how it changes.

x	y = x²
1	1
2	4
3	9
4	16

Notice:

• 1 → 4 = +3
• 4 → 9 = +5
• 9 → 16 = +7

The change is increasing.

Derivative tells exactly how steep the curve becomes.

The derivative of x² is:

At x = 3:

Slope = 6

ML Use Case 1: Gradient Descent (Most Important)

Every ML model minimizes loss.

The model starts here.

Question:

Which direction should we move?

The derivative answers that.

If derivative is positive

Slope goes upward.

Move left.

If derivative is negative

Slope goes downward.

Move right.

The derivative tells the optimizer where to move.

Gradient Descent Formula

This is the most important ML equation.

Symbol	Meaning
w	Weight
α	Learning rate
dL/dw	Derivative of loss

The model updates weights using derivatives.

ML Use Case 2: House Price Prediction

Suppose the model predicts:

• Actual = 80
• Predicted = 74

Loss:

Derivative calculates:

How should the weight change to reduce this error?

Without derivatives, the model cannot improve.

ML Use Case 3: ChatGPT

Transformers contain billions of parameters.

During training:

1. Predict next word
2. Compute loss
3. Compute derivatives
4. Update weights

Every training step uses derivatives.

No derivatives → No learning.

ML Use Case 4: Real Estate Price Optimization

Your property AI has weights.

Example:

Suppose BHK weight is too high.

Derivative tells:

Increase or decrease this weight?

Training automatically adjusts it.

Python Example

import sympy as sp

x = sp.symbols('x')

f = x**2

print(sp.diff(f, x))

Output:

2*x

diff() computes the derivative.

Real-world intuition

Situation	Derivative Means
Car	Speed
Salary	Growth per year
House Price	Change per sq.ft
Loss Function	Direction to reduce error
ChatGPT	How to update weights

Summary

Concept	Meaning
Function	Maps input → output
Limit	Approaches a value
Continuity	No breaks
Derivative	Rate of change / slope

Memory Trick

Function tells what the output is.

Derivative tells how fast the output changes.

Interview Question

Q: Why are derivatives essential in Machine Learning?

Answer: Derivatives measure how the loss changes with respect to model parameters. Gradient Descent uses these derivatives to update weights in the direction that reduces prediction error, enabling the model to learn from data.

Next lesson: Partial Derivatives.

Partial Derivatives

You've learned derivatives for functions with one input. But most ML models have many inputs (BHK, Area, Parking, etc.). Partial derivatives solve that problem.

Partial Derivative = How the output changes when only ONE input changes, while all other inputs stay constant.

This is one of the most important concepts behind Gradient Descent, Neural Networks, and ChatGPT training.

Intuition: House price depends on multiple features

A house price is affected by many variables:

Inputs:

• BHK
• Area
• Parking
• Floor

Output:

Price

Now ask a specific question:

If only Area increases, how much does Price change?

That is a partial derivative.

Why “partial”?

Normal derivative:

• One input → One output

Partial derivative:

• Multiple inputs → One output

Now we can change x or y independently.

Example Function

Suppose:

Where:

• x = Bedrooms
• y = Area (hundreds of sq.ft.)

We want to know:

• Effect of Bedrooms
• Effect of Area

Separately.

Partial derivative with respect to x

Notation:

Rule:

Treat y as constant.

Derivative:

Meaning:

Every additional bedroom increases the score by 2 units.

Notice we ignored y.

Partial derivative with respect to y

Now differentiate only y.

Meaning:

Every additional 100 sq.ft increases the score by 3 units.

Visual intuition

Imagine a 3D hill.

The hill represents the function.

You can move:

• Left ↔ Right (x)
• Forward ↔ Backward (y)

Each direction has its own slope.

Those slopes are partial derivatives.

ML Use Case 1: Gradient Descent

Suppose loss depends on two weights.

The optimizer asks:

• How should w₁ change?
• How should w₂ change?

It computes:

These tell each weight how to update.

Without partial derivatives, the model wouldn't know which parameter to change.

ML Use Case 2: Neural Networks

A neuron has many inputs.

Each input has a weight.

Loss depends on all weights.

The network computes:

Millions of these are calculated during backpropagation.

ML Use Case 3: ChatGPT

GPT has billions of parameters.

During training:

1. Predict next token
2. Calculate loss
3. Compute partial derivative for every parameter
4. Update all weights

Every parameter has its own partial derivative.

This is why GPUs perform enormous numbers of gradient calculations.

ML Use Case 4: Real Estate Model

Suppose your price function is:

Question 1:

How much does one extra bedroom affect price?

Question 2:

How much does one extra sq.ft affect price?

Question 3:

How much does parking affect price?

Each feature has its own partial derivative.

This helps explain feature importance.

Gradient = Collection of Partial Derivatives

If a function has many variables:

The gradient is:

Think of it as a vector of slopes.

Variable	Partial Derivative
BHK	10
Area	0.04
Parking	5

Gradient:

This vector tells the optimizer how each feature influences the output.

Python Example

import sympy as sp

x, y = sp.symbols('x y')

f = 2 * x + 3 * y

print(sp.diff(f, x))   # ∂f/∂x
print(sp.diff(f, y))   # ∂f/∂y

Output:

2
3

Summary Table

Concept	Meaning
Derivative	Change of a function with one variable
Partial Derivative	Change of one variable while others stay constant
Symbol	∂
Gradient	Vector of all partial derivatives

Memory Trick

Derivative asks: "How does the output change?" (1 input)

Partial Derivative asks: "How does the output change if ONLY this one input changes?" (Many inputs)

Interview Question

Q: Why are partial derivatives essential in neural networks?

Answer: Neural networks have millions or billions of parameters. Partial derivatives compute how the loss changes with respect to each parameter individually, allowing backpropagation and gradient descent to update every weight correctly.

Next lesson: Chain Rule.

Chain Rule

The Chain Rule is the calculus concept that makes deep learning possible. Every neural network—including ChatGPT—learns through repeated application of the chain rule during backpropagation.

Chain Rule = Find how the final output changes by multiplying the changes through every intermediate step.

A simple way to remember it:

If A affects B, and B affects C, then A indirectly affects C.

Intuition: Domino effect

Imagine three dominoes.

• x changes the hidden value u
• u changes the output y
• Therefore, x changes y

The chain rule measures this indirect effect.

Step 1: One function inside another

Suppose:

Then:

This is actually one combined function:

But we'll solve it step by step.

Step 2: Differentiate each step

First derivative:

Second derivative:

Now apply the chain rule:

Substitute:

Answer: The derivative is 6x.

Why do we multiply?

Think of water flowing through pipes.

The influence passes through every stage.

Each stage contributes its own derivative.

So we multiply them.

ML Use Case 1: Neural Network (Most Important)

A neuron performs three steps.

Forward pass:

During training we need:

How does the weight affect the loss?

Chain rule:

Each layer contributes one derivative.

This is called Backpropagation.

ML Use Case 2: ChatGPT Training

A transformer has hundreds of layers.

Simplified pipeline:

The loss depends on:

• Embeddings
• Attention
• Feed-forward layers
• LayerNorm
• Output weights

The chain rule propagates gradients backward through every layer.

Without the chain rule:

• GPT cannot update weights.
• No learning happens.

ML Use Case 3: Computer Vision (CNN)

A CNN predicts Cat.

Forward

Pixels → Conv → ReLU → Pooling → Loss

Backward

Loss → Pooling → ReLU → Conv → Pixels

During backpropagation:

The chain rule computes how every filter affects the final prediction.

This allows CNN filters to learn edges, textures, and shapes.

ML Use Case 4: Real Estate AI

Suppose your valuation model is:

If the prediction is wrong:

• Did Area cause it?
• Did Feature Score cause it?
• Did the final weight cause it?

The chain rule traces the error backward through every stage and updates the correct weights.

Why is it called “Chain”?

Because derivatives are connected like a chain.

If A changes D:

• A affects B
• B affects C
• C affects D

The total effect equals:

This is the chain rule.

Python Example

Using SymPy:

import sympy as sp

x = sp.symbols('x')

u = x**2
y = 3 * u

print(sp.diff(y, x))

Output:

6*x

The derivative automatically applies the chain rule.

Summary Table

Concept	Meaning
Derivative	Change of one function
Partial Derivative	Change of one variable in many variables
Chain Rule	Change through multiple connected functions
Main ML Use	Backpropagation

Memory Trick

Derivative = One slope

Partial Derivative = One variable's slope

Chain Rule = Multiply slopes through every layer

Interview Question

Q: Why is the chain rule essential in deep learning?

Answer: Neural networks are composed of many nested functions (linear layers, activations, attention, etc.). The chain rule computes how the final loss changes with respect to each parameter by multiplying derivatives through every intermediate layer, enabling backpropagation and weight updates.