Classification with Logistic Regression

From Numbers to Categories

Classification predicts a label: spam/ham, churn/stay, fraud/clean. Despite its name, logistic regression is a classifier — it’s linear regression pushed through a squashing function (the sigmoid) so the output lands between 0 and 1 and can be read as a probability:

z = w · x + b                  # any real number
p = 1 / (1 + e^(-z))           # squashed into (0, 1)

If p = 0.93, the model says “93% confident this is spam”.

A Working Spam Classifier

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    emails, labels, test_size=0.2, random_state=42, stratify=labels
)

vec = TfidfVectorizer(stop_words="english")
X_train_t = vec.fit_transform(X_train)   # learn vocabulary on TRAIN only
X_test_t  = vec.transform(X_test)        # apply it to test

clf = LogisticRegression(max_iter=1000)
clf.fit(X_train_t, y_train)
clf.score(X_test_t, y_test)              # accuracy

Two details that separate working code from broken code:

• stratify=labels keeps the class ratio identical in both splits.
• fit_transform on train, plain transform on test — fitting the vectoriser on test data is data leakage and inflates your score dishonestly.

Probabilities and Thresholds

clf.predict(X_test_t)          # hard labels, threshold 0.5
clf.predict_proba(X_test_t)    # probabilities per class

The 0.5 threshold is a default, not a law. Blocking emails when p(spam) > 0.5 is aggressive; a bank flagging fraud might act at 0.2 because missing fraud costs more than a false alarm. The threshold encodes a business decision, and choosing it belongs to you, not the library.

Multi-Class Works Out of the Box

clf = LogisticRegression(max_iter=1000)
clf.fit(X_train, y_train)      # y can have 10 classes — no code change

scikit-learn handles multi-class via softmax automatically. The mental model stays identical: linear score per class, squashed into probabilities that sum to 1.