The bias–variance tradeoff describes the balance between two common sources of prediction error in machine learning: bias and variance.
- High bias: The model is too simple and makes strong assumptions about the data. It may miss important patterns, causing underfitting.
- High variance: The model is too sensitive to the training data and may learn noise instead of general patterns. This often causes overfitting.
- The tradeoff: Increasing model complexity can reduce bias but may increase variance. Reducing complexity can lower variance but may increase bias.
For example, a very simple linear model might have high bias because it cannot capture a complex relationship. A highly flexible model might fit the training data extremely well but perform poorly on new data because it has learned noise.
The goal is not to eliminate bias or variance completely. Instead, the aim is to find a model and training setup that generalizes well to unseen data.
In simple terms, bias comes from being too simple, variance comes from being too sensitive, and the tradeoff is finding the right balance between the two.
