🤖 Intro to Machine Learning

Build your first classifier using the famous Iris dataset:

• Load the Iris dataset using from sklearn.datasets import load_iris
• Explore the data: print the feature names and target names
• Split the data into training (80%) and testing (20%) sets
• Train a KNeighborsClassifier with n_neighbors=3
• Make predictions on the test set and calculate accuracy
• Try different values of n_neighbors (1, 5, 7) and compare accuracy

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score

# Load data
iris = load_iris()
X, y = iris.data, iris.target

# TODO: Split the data
# TODO: Create and train the model
# TODO: Make predictions and evaluate

Practice data preprocessing with StandardScaler:

• Create a synthetic dataset with features on different scales (e.g., age: 0-100, income: 0-1000000)
• Apply StandardScaler to normalize the features
• Compare the mean and standard deviation before and after scaling
• Important: Fit the scaler on training data only, then transform both train and test
• Train a LogisticRegression model with and without scaling—compare accuracy

from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
import numpy as np

# Create data with different scales
np.random.seed(42)
age = np.random.randint(18, 80, 100)  # Range: 18-80
income = np.random.randint(20000, 200000, 100)  # Range: 20k-200k
X = np.column_stack([age, income])
y = (income > 100000).astype(int)  # Simple target

# TODO: Split the data
# TODO: Create scaler and fit on X_train only
# TODO: Transform both X_train and X_test
# TODO: Print mean/std before and after scaling

Build a linear regression model to predict house prices:

• Load the California Housing dataset: from sklearn.datasets import fetch_california_housing
• Explore the features (MedInc, HouseAge, AveRooms, etc.)
• Split data into train/test sets (80/20 split)
• Train a LinearRegression model
• Calculate Mean Squared Error (MSE) and R² score on test data
• Bonus: Try Ridge regression and compare performance

from sklearn.datasets import fetch_california_housing
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score

# Load data
housing = fetch_california_housing()
X, y = housing.data, housing.target

print(f"Features: {housing.feature_names}")
print(f"X shape: {X.shape}")

# TODO: Split the data
# TODO: Train LinearRegression
# TODO: Make predictions and calculate MSE, R²
# TODO: Try Ridge regression

Extend your regression skills with polynomial features:

• Create synthetic data with a non-linear relationship: y = x² + noise
• Try fitting a LinearRegression—observe the poor fit
• Use PolynomialFeatures(degree=2) to transform your data
• Fit LinearRegression on the polynomial features
• Compare R² scores: linear vs polynomial
• Experiment with degree=3, 4, 5—what happens with too high a degree?

Understand when accuracy isn't enough—compare different metrics:

• Create an imbalanced dataset: 90% class 0, 10% class 1
• Train a classifier and calculate Accuracy, Precision, Recall, and F1-score
• Create a "dummy" classifier that always predicts class 0—what's its accuracy?
• Compare all metrics between your model and the dummy classifier
• Understand: When is accuracy misleading? When do we care about Recall vs Precision?
• Use classification_report() for a complete metrics summary

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (accuracy_score, precision_score, 
                             recall_score, f1_score, classification_report)

# Create imbalanced dataset
X, y = make_classification(n_samples=1000, n_classes=2, 
                           weights=[0.9, 0.1], random_state=42)

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

# TODO: Train a classifier
# TODO: Calculate accuracy, precision, recall, F1
# TODO: Compare with a dummy "always predict 0" baseline
# TODO: Print classification_report()

Build a complete classification pipeline to predict customer churn:

• Generate synthetic customer data with features: tenure, monthly_charges, total_charges, contract_length
• Create a target variable: churn (1) or stay (0) based on logical rules
• Preprocess: Handle any missing values, scale numerical features
• Split data with stratification (important for imbalanced churn data!)
• Train multiple classifiers: LogisticRegression, RandomForest, KNeighbors
• Compare all three using accuracy, precision, recall, and F1-score
• Identify which model is best for minimizing customer loss (hint: focus on recall for churners)
• Visualize the confusion matrix for your best model

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import classification_report, confusion_matrix

# Generate synthetic churn data
np.random.seed(42)
n_customers = 1000

data = pd.DataFrame({
    'tenure': np.random.randint(1, 72, n_customers),  # Months
    'monthly_charges': np.random.uniform(20, 100, n_customers),
    'contract_length': np.random.choice([1, 12, 24], n_customers),
})
data['total_charges'] = data['tenure'] * data['monthly_charges']

# Create churn logic: high charges + short tenure + month-to-month = likely churn
churn_prob = (100 - data['tenure']) / 100 + (data['monthly_charges'] / 200)
churn_prob += (data['contract_length'] == 1) * 0.3
data['churn'] = (churn_prob > np.random.uniform(size=n_customers) + 0.5).astype(int)

# TODO: Prepare features (X) and target (y)
# TODO: Split with stratify=y
# TODO: Scale features
# TODO: Train and compare 3 different classifiers
# TODO: Print classification reports and identify best model

Create a production-ready pipeline that handles everything:

• Load the breast cancer dataset: from sklearn.datasets import load_breast_cancer
• Build a pipeline with: StandardScaler → PCA(n_components=10) → LogisticRegression
• Use Pipeline with named steps instead of make_pipeline
• Train and evaluate the pipeline
• Access individual steps using pipe.named_steps['scaler']
• Compare performance with and without PCA dimensionality reduction

from sklearn.datasets import load_breast_cancer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

# Load data
cancer = load_breast_cancer()
X, y = cancer.data, cancer.target

# TODO: Split the data
# TODO: Create Pipeline with named steps
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('pca', PCA(n_components=10)),
    ('classifier', LogisticRegression())
])

# TODO: Fit and evaluate
# TODO: Access named steps and inspect components

Combine pipelines with cross-validation for robust model evaluation:

• Use cross_val_score with your pipeline for 5-fold cross-validation
• Calculate mean and standard deviation of scores across folds
• Use GridSearchCV to tune hyperparameters within the pipeline
• Search over: pca__n_components: [5, 10, 15] and classifier__C: [0.1, 1, 10]
• Print the best parameters and best score
• Understand why preprocessing must be inside the pipeline for proper cross-validation