Unsupervised Learning: Clustering & PCA

No Labels, Still Learning

Unsupervised learning finds structure in raw data: which customers behave alike, which transactions look anomalous, which 3 directions explain most of a 100-column dataset. No right answers to train against — which makes evaluation more judgement than score.

k-Means Clustering

Pick k centroids, assign every point to its nearest centroid, move each centroid to the mean of its points, repeat until stable:

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

X_scaled = StandardScaler().fit_transform(X)   # ESSENTIAL — k-means uses distance

km = KMeans(n_clusters=4, n_init="auto", random_state=42)
labels = km.fit_predict(X_scaled)

Always scale first. Distance-based algorithms see a feature ranging 0–100,000 (income) as 1000× more important than one ranging 0–100 (age) unless you standardise.

Choosing k

from sklearn.metrics import silhouette_score

for k in range(2, 10):
    labels = KMeans(n_clusters=k, n_init="auto").fit_predict(X_scaled)
    print(k, silhouette_score(X_scaled, labels))

The silhouette score (−1 to 1) measures how much closer points are to their own cluster than to the next one. Pick the k that peaks — then sanity-check the clusters by profiling them:

df["cluster"] = labels
df.groupby("cluster")[["age", "income", "orders"]].mean()

If segment 2 is “young, low spend, high frequency”, marketing can use that. Clusters that defy description are usually artefacts.

DBSCAN: Density Instead of Distance

from sklearn.cluster import DBSCAN
labels = DBSCAN(eps=0.5, min_samples=5).fit_predict(X_scaled)  # -1 = noise

DBSCAN finds arbitrarily-shaped clusters, doesn’t need k, and labels outliers as noise — handy for anomaly detection. Its price: sensitivity to eps.

PCA: Fewer Dimensions, Most of the Signal

Principal Component Analysis rotates the data onto new axes ordered by variance, letting you keep the few that matter:

from sklearn.decomposition import PCA

pca = PCA(n_components=0.95)        # keep 95% of variance
X_small = pca.fit_transform(X_scaled)
X_small.shape                        # maybe (10000, 12) from (10000, 100)

Uses: 2-D visualisation of high-dimensional data (n_components=2 + scatter plot), speeding up downstream models, and de-noising. The components are linear mixes of original features, so some interpretability is the price.