Deep Learning with PyTorch

Why PyTorch

PyTorch is the dominant deep learning framework in research and industry. It gives you NumPy-style tensors that run on GPUs, automatic differentiation (no hand-written backprop), and a library of layers, losses, and optimisers:

pip install torch

import torch

x = torch.tensor([[1.0, 2.0]], requires_grad=True)
y = (x ** 2).sum()
y.backward()        # autograd computes gradients
x.grad              # tensor([[2., 4.]])

That backward() call replaces the entire backprop function from last topic.

Defining a Model

import torch.nn as nn

class Net(nn.Module):
    def __init__(self, n_features):
        super().__init__()
        self.layers = nn.Sequential(
            nn.Linear(n_features, 64),
            nn.ReLU(),
            nn.Dropout(0.2),          # regularization: randomly zero 20% of activations
            nn.Linear(64, 32),
            nn.ReLU(),
            nn.Linear(32, 1),         # raw logit out — no sigmoid here
        )

    def forward(self, x):
        return self.layers(x)

model = Net(X.shape[1])

We output raw logits and use BCEWithLogitsLoss, which fuses the sigmoid in a numerically stable way.

Data Pipeline

from torch.utils.data import TensorDataset, DataLoader

ds = TensorDataset(torch.tensor(X_train, dtype=torch.float32),
                   torch.tensor(y_train, dtype=torch.float32).unsqueeze(1))
loader = DataLoader(ds, batch_size=64, shuffle=True)

Networks train on mini-batches: cheaper per step than the full dataset, and the noise in batch gradients actually helps escape bad regions.

The Canonical Training Loop

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.BCEWithLogitsLoss()

for epoch in range(30):
    model.train()
    for xb, yb in loader:
        xb, yb = xb.to(device), yb.to(device)
        opt.zero_grad()              # 1. clear old gradients
        loss = loss_fn(model(xb), yb)  # 2. forward
        loss.backward()              # 3. backward
        opt.step()                   # 4. update weights

Memorise the four-step rhythm — every PyTorch project, from MNIST to LLM fine-tuning, is this loop with different ingredients. Forgetting zero_grad() is the classic bug: gradients accumulate and training silently diverges.

Evaluation Mode

model.eval()                         # disables dropout, batch-norm updates
with torch.no_grad():                # no gradient bookkeeping
    logits = model(X_test_t.to(device))
    preds = (torch.sigmoid(logits) > 0.5).float()

When Deep Learning Is the Right Tool

Images, audio, and text — anywhere features must be learned rather than engineered — deep learning dominates. On tabular data, gradient boosting usually still wins. Choose by data type, not by hype.