Introduction to Data Science & Pandas

The Data Science Workflow

Data Science is about extracting insights from data. The typical workflow involves:

1. Data Collection: Gathering data from APIs, databases, or CSVs.
2. Data Cleaning: Handling missing values, outliers, and incorrect formats.
3. Exploratory Data Analysis (EDA): Visualizing and understanding patterns in the data.
4. Modeling: Applying statistical or machine learning models.
5. Communication: Presenting findings through dashboards or reports.

Enter Pandas

Pandas provides high-performance data structures and data analysis tools. The core objects are Series (1D array) and DataFrame (2D table).

import pandas as pd

# Load data
df = pd.read_csv("sales.csv")

# Quick look at the data
print(df.head())
print(df.info())
print(df.describe())

Selecting and Filtering Data

# Select columns
revenues = df["revenue"]
subset = df[["date", "revenue", "product"]]

# Filter rows (Boolean indexing)
high_sales = df[df["revenue"] > 1000]
recent_sales = df[df["date"] >= "2024-01-01"]

Handling Missing Data

Real-world data is messy. Pandas gives you tools to handle it:

# Drop rows with missing values
clean_df = df.dropna()

# Fill missing values with a specific value (e.g., the mean)
df["price"].fillna(df["price"].mean(), inplace=True)