AlgoAnimator: Interactive Data Structures

Refining the Brain

Calculating the output (Forward Pass) is easy. The hard part is: How do we update the weights to make the output better? Enter Backpropagation.

Continue

1. The Forward Pass

First, data flows from Input → Hidden → Output. We get a prediction (e.g., 0.5), but our target is 0.8.

Continue

2. Measure the Blame

We calculate the error (Difference = -0.3). Now we must assign 'blame'. Which weights caused this error? Backprop answers: 'How much would the Error change if I changed this weight?'

Continue

3. The Backward Pass

We send the error signal BACKWARDS through the network. Using the 'Chain Rule' of calculus, we compute the gradient (slope) for every single weight.

Continue

4. Gradient Descent

Once we have the gradients, we update the weights in the opposite direction. If increasing a weight increases error, we decrease the weight.

Continue

Watch it Learn

Click the button below to perform training steps. Notice how the weights change slightly each time, bringing the Output closer to the Target and reducing Loss.

Continue

The Training Loop

This is the heart of Neural Network training.

train.py

# 1. Forward Pass
output = model(input)

# 2. Compute Loss
loss = criterion(output, target)

# 3. Backward Pass (Compute Gradients)
loss.backward()

# 4. Update Weights
optimizer.step()

PyTorch Training Step

The Chain Rule

How we calculate gradients for weights deep in the network.

math.py

# Gradient for weight w1 (deep layer)
# dLoss/dw1 = dLoss/dOutput * dOutput/dHidden * dHidden/dw1

grad_w1 = (error) * (weight_2) * (input)

Propagating Error Backwards