Repetition is Key

A neural network doesn't learn everything instantly. It needs to see the data many times, making small adjustments each time. This cycle is called the Training Loop.
Continue

1. The Dataset

Imagine we have 10,000 images of cats and dogs. We can't feed them all into the computer at once. It would crash (Out of Memory).
Continue

2. Mini-Batches

Solution: We split the data into small chunks called 'Batches' (e.g., 32 images at a time). The network learns from one batch, updates weights, and precise to the next.
Continue

3. The Epoch

An 'Epoch' is one complete pass through the ENTIRE dataset. If you have 1000 items and batch size is 100, one Epoch = 10 Batches.
Continue

4. The Full Cycle

Typically, we train for many Epochs. Loop: 1. Get Batch 2. Forward Pass 3. Calculate Loss 4. Backprop & Update 5. Repeat until Epoch ends.
Continue

Interactive Training

Start the training below. Watch how it processes batches (highlighted dots) and how the Loss Curve (Error) decreases over time (Epochs).
Continue

Python Implementation

A standard PyTorch training loop structure.

train.py
# Hyperparameters
EPOCHS = 10

for epoch in range(EPOCHS):
    running_loss = 0.0
    
    # Iterate over Batches
    for batch_idx, (data, target) in enumerate(train_loader):
        
        # 1. Zero Gradients
        optimizer.zero_grad()
        
        # 2. Forward Pass
        output = model(data)
        
        # 3. Calculate Loss
        loss = criterion(output, target)
        
        # 4. Backward Pass & Update
        loss.backward()
        optimizer.step()
        
    print(f"Epoch {epoch} complete!")
Training Loop Logic

Key Terms

  • Batch SizeNumber of examples processed at once (e.g., 32, 64).
  • IterationOne update of the model's weights (one batch processed).
  • EpochOne full cycle through the entire dataset.
AlgoAnimator: Interactive Data Structures