NumPy & pandas for Machine Learning

Why Arrays, Not Loops

ML is linear algebra at scale. NumPy stores numbers in contiguous C arrays and runs operations in compiled code — typically 10–100× faster than Python loops:

import numpy as np

prices = np.array([120, 250, 90, 410])
taxed = prices * 1.18          # vectorised — no loop
taxed.mean()                   # 255.35

The Shapes That Matter

Supervised learning expects two objects:

X = np.array([[1200, 2], [1500, 3], [800, 1]])  # features: (n_samples, n_features)
y = np.array([95, 120, 62])                       # target:   (n_samples,)

X is the feature matrix — one row per example, one column per feature. y is the target vector. Every scikit-learn estimator consumes exactly this pair. When something breaks, check X.shape first — shape bugs are the #1 beginner error.

pandas: Labelled Data

import pandas as pd

df = pd.read_csv("houses.csv")
df.head()          # first 5 rows
df.info()          # columns, dtypes, missing counts
df.describe()      # summary statistics

Selection and filtering:

df["price"]                          # one column (a Series)
df[["area", "bedrooms"]]             # several columns (a DataFrame)
df[df["price"] > 5_000_000]          # boolean filtering
df.loc[df["city"] == "Mohali", "price"].median()

From DataFrame to Feature Matrix

features = ["area", "bedrooms", "age"]
X = df[features]
y = df["price"]

Models need numbers, so categorical columns are encoded:

X = pd.get_dummies(df[["area", "city"]], columns=["city"])  # one-hot encoding

Handling Missing Values

df.isna().sum()                        # count missing per column
df["age"] = df["age"].fillna(df["age"].median())
df = df.dropna(subset=["price"])       # never impute the target

Rule of thumb: impute features (median for numbers, mode or “missing” category for text), but drop rows where the target is missing — you can’t learn from an answer that isn’t there.