The Bitter Lesson
Date read: 26th August 2025
Key Points
- Scaling compute almost always outperforms human design biases eventually.
- This is the bitter lesson, no matter how much we think we know better, scaling will normally outperform our human
biases.
- The trap is that embedding human knowledge is psychologically satisfying and often yields short term improvements.
- This has been shown in chess, go, shogi etc.