Feature Engineering and Data Preprocessing

Why Feature Engineering?

Machine learning models require numerical input. They cannot natively understand text, categories, or raw dates. Feature engineering is the process of extracting meaningful variables (features) from raw data and formatting them correctly. It is often said that “applied machine learning is basically feature engineering.”

Handling Categorical Variables

Categories like “Red”, “Green”, and “Blue” need to be converted to numbers. The two most common techniques are:

• Label Encoding: Assigning an integer to each category (e.g., Red=1, Green=2). Good for ordinal data with a clear hierarchy (e.g., Small, Medium, Large).
• One-Hot Encoding: Creating a new binary column for each category. Best for nominal data without hierarchy to avoid models assuming that Red (1) + Green (2) = Blue (3).

# One-Hot Encoding in Pandas
encoded_df = pd.get_dummies(df, columns=["color"])

Feature Scaling

Algorithms like K-Nearest Neighbors and Neural Networks are highly sensitive to the scale of the data. If Age ranges from 0-100 and Salary ranges from 0-100,000, the Salary feature will dominate the distance calculations.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
df["salary_scaled"] = scaler.fit_transform(df[["salary"]])

Creating New Features

Sometimes the best insights come from combining existing columns. For example, if you have total_spent and num_visits, creating an average_spend_per_visit feature might be a much stronger predictor of customer churn than the original variables alone.