Quiz · Medium
Supervised vs unsupervised learning, overfitting and the bias-variance trade-off, train/test splits, precision, recall and F1, regularization, data leakage and common algorithms.
A store groups customers by purchasing behavior with no predefined labels. What type of learning is this?
Supervised classification
Unsupervised learning (clustering)
Supervised regression
Reinforcement learning
There are no target labels, so the algorithm finds structure on its own, for example with k-means. Classification would need examples already tagged with the correct group.
A model predicts a house's sale price from its size and location. This is a:
Regression problem
Classification problem
Clustering problem
Dimensionality reduction problem
The target is a continuous number, which makes it regression. Predicting a category, such as sells within a month yes/no, would be classification.
A model scores 99% accuracy on training data but 70% on the test set. What is the most likely problem?
Underfitting
Overfitting
Too little model capacity
A perfectly tuned model
The model memorized noise in the training data and does not generalize. Underfitting shows poor performance on both training and test data.
Which change is most likely to reduce overfitting?
Add regularization or collect more training data
Train for many more epochs
Add more features and parameters
Evaluate only on the training set
Regularization penalizes complexity, and more data makes noise harder to memorize. Early stopping, dropout and simpler models also help. Training longer and adding parameters usually make overfitting worse.
What is the test set for?
Choosing hyperparameters
A final, unbiased estimate of performance after all tuning is done
Training the model
Removing outliers
Hyperparameters are tuned on a validation set (or with cross-validation). If you tune on the test set, its score becomes optimistic because information has leaked into your choices.
A spam filter has 40 true positives, 10 false positives and 20 false negatives. What is its precision?
0.67
0.50
0.33
0.80
: of the emails it flagged, 80% were spam. 0.67 is recall.
Using the same counts (TP = 40, FP = 10, FN = 20), what is the recall?
0.80
About 0.67
0.50
About 0.33
: it caught two-thirds of the actual spam. Recall asks how much of the real positive class you found.
In a fraud data set, 99% of transactions are legitimate. A model that predicts "legitimate" for everything has what accuracy, and is it useful?
99% accuracy, but useless, because it never catches fraud
1% accuracy, so it is useless
99% accuracy and very useful
50% accuracy, since there are two classes
With imbalanced classes, accuracy is misleading. This model has 0 recall on fraud. Use precision, recall, F1 or the area under the precision-recall curve instead.
A model has precision 0.8 and recall 0.5. What is its F1 score?
About 0.62
0.65
0.40
1.30
. It is the harmonic mean, which sits closer to the weaker value. 0.65 is the simple average.
During gradient descent, the loss bounces around and grows instead of decreasing. What is the most likely cause?
The learning rate is too low
The data set is too large
The model has too few features
The learning rate is too high
Steps that are too big overshoot the minimum and can diverge. A learning rate that is too low gives the opposite symptom: very slow, steady progress.
What is the main purpose of k-fold cross-validation?
To make training faster
To get a more reliable performance estimate by training and validating on k different splits
To increase the size of the data set
To remove the need for a test set entirely
Each fold takes a turn as the validation set, and the k scores are averaged. This reduces the luck of a single split, which matters most with small data sets. A held-out test set is still good practice for the final check.
Which regularization method can shrink some model weights to exactly zero, effectively selecting features?
L2 (ridge)
Dropout
Batch normalization
L1 (lasso)
The L1 penalty (sum of absolute weights) has corners that push weights to exactly 0. L2 (sum of squared weights) shrinks weights toward zero but rarely makes them exactly zero.
A data scientist fits a StandardScaler on the full data set and then splits it into training and test sets. What is wrong?
Nothing; this is the recommended order
Scaling should only be applied to the test set
StandardScaler cannot be used before a split
Data leakage: test-set statistics influenced the training data
The mean and standard deviation now include test data, so the test score is slightly optimistic. Split first, fit the scaler on the training set only, then transform both sets (a pipeline does this for you).
Why should features be scaled before using k-nearest neighbors?
k-NN only accepts values between 0 and 1
Distance calculations are dominated by features with large numeric ranges
Scaling reduces the number of neighbors needed
Scaling turns k-NN into a linear model
If income ranges in the tens of thousands and age only in the tens, income decides almost every distance. Standardizing puts features on comparable scales. Tree-based models do not need this.
How does a random forest differ from a single decision tree?
It uses one very deep tree
It is a neural network with tree-shaped layers
It always uses fewer features than a single tree and has higher bias
It averages many trees trained on bootstrap samples with random feature subsets, which reduces variance
Bagging plus random feature selection makes the trees less correlated, so averaging them cuts the overfitting a single deep tree is prone to.
As model complexity increases, what generally happens to bias and variance?
Bias decreases and variance increases
Bias increases and variance decreases
Both decrease
Both increase
Simple models underfit (high bias), and complex models fit noise (high variance). The best test error usually comes from a complexity in between, which is the bias-variance trade-off.
