Learning Under Constraint
To understand how modern AI systems function, we must move beyond outputs and examine the assumptions and mechanisms that shape learning itself.
The assumptions or constraints that let a model generalize beyond the exact examples it has seen.
The empirical pattern that model performance improves predictably as data, parameters, and compute increase.
Systematic skew or prejudice in outputs caused by training data, framing, or development conditions.
A critical concept in machine learning is . Because models are trained on finite data, they cannot infer every possible rule from examples alone. They need structured assumptions about the world. That bias is part of what makes learning possible.
Another central observation is . These laws describe why broader capability gains often appear as we increase model size, data availability, and compute. They are a major reason AI progress can feel both smooth at the macro level and surprising at the level of individual model behavior.
This is the bridge to later sections: once you accept that quantitative increases in scale can reliably improve capability, it becomes easier to understand why some qualitative-seeming jumps may arise from scaling rather than from totally new architectures.
However, scale does not erase . Larger systems can propagate distortions at greater scale and with greater persuasive force if the underlying data or framing remains flawed.
