Generalization
lastmod 2026-03-24created 2015-12-243 edits
Generalization is a form of abstraction in which common properties of specific instances are formulated as general concepts or claims.
In general, given two related concepts A and B, A is a "generalization" of B if and only if the following conditions hold.
- Every instance of concept B is also an instance of concept A
- There are instances of concept A that are not instances of concept B
Deep learning is not so mysterious or different
- Deep Learning is Not So Mysterious or Different
- Deep learning is not so mysterious or different | GeekNews
There is a perception that the generalization behavior of deep learning models is different from that of conventional models and mysterious. Overfitting, double descent, and overparametrization are often cited as characteristics of deep learning. However, these phenomena are not limited to neural networks and can be explained by long-standing generalization frameworks such as PAC-Bayes and countable hypothesis bounds. The concept of "soft inductive biases" is the key principle explaining these generalization phenomena