· intermediate
Model Evaluation and Validation
Deciding whether a model is any good. Why accuracy misleads, what precision and recall are really trading off, where the threshold comes from, and how cross-validation stops one lucky split from deciding everything.
Written in house, from the work rather than from a syllabus. Every example is one we would use ourselves, and every quiz question explains why its answer is right.
What you’ll learn
- Read a confusion matrix and name the two kinds of error
- Choose between precision, recall and their balance from the cost of each error
- Explain why accuracy fails on imbalanced data
- Pick a decision threshold deliberately rather than accepting 0.5
- Use cross-validation, and pick the right splitter for grouped or time-ordered data
- Choose a regression metric knowing how each treats a large error
Before you start
- · Machine Learning Basics, or equivalent
- · Comfortable with the idea of a train and test split