Book 5 — ML Theory & Model Evaluation¶
How to know if a model is actually good.
Validation¶
- Train/val/test split
- Cross-validation
- Bias & variance
- Overfitting & underfitting
- Regularization
- Data/feature leakage
- Class imbalance — full deep-dive
Metrics¶
- Precision, recall, F1
- ROC, AUC, PR-AUC
- Calibration & confusion matrix
- Regression metrics — full deep-dive
Tuning¶
Status
Complete deep-dive coverage of model validation and generalization, the bias-variance decomposition, leakage and class imbalance, classification and regression evaluation metrics, and hyperparameter optimization from grid search to Bayesian methods.