Artificiall and Biological Learning
Published:
Where does intelligence emerge?
Artificial Learning
To make the learning mechanism tractable, the speaker introduced a minimal setting: random structured sequences generated by Markov rules, learned by a simple two-layer transformer.
Across related minimal Markov-model studies, the speaker described four possible algorithmic regimes:
- Unigram Retrieval
- Bigram Retrieval
- Unigram Inference
- Bigram Inference
Besides, thanks to the small model only with 5 parameters, we can easily to draw the picture. And we can get:
- a cliff landscape
- a simple dynamical system
- a phase transition from a simpler unigram-based strategy to a bigram / statistical-induction strategy
- Early training dynamics are shaped by both the statistical structure of the data and the model’s own optimization bias, which together influence the onset of abrupt improvement.
Summary
- 4 algorithms can be naturally implemented by in-context learning of Markov processes by a simple two-layer transformer architecture.
- Abrupt learning can be explained by the cliff landscape predicted from our minimal models.
- A scaling relation between the timing of abrupt learning and the context length is derived.
- In the finite task-diversity regime, we identfy two distinct phase trainsitions between memorization and generalization, determined by competition and model capacity, respectivley.
- The generalization ability does not always scale up with model size.
Biological Learning
Then let’s pay attention to rats.
We can easily observe with sensors. We can easily conduct reinforcement-training( not learning lol)
A. Data Preprocessing Pipeline
\[\text{Cameras} \rightarrow \text{3D keypoint tracking} \rightarrow \text{pose trajectories} \rightarrow \text{attempt segmentation}\]B. Behavioral Structure Analysis Pipeline
\[\text{attempts} \rightarrow \text{similarity matrix} \rightarrow \text{block structure} \rightarrow \text{keypoint-MoSeq / syllables} \rightarrow \text{motifs} \rightarrow \text{motif competition / selection / refinement}\]There are a few conclusions:
- They study simple motif dynamics reduced from complicated motion trajectories
- Different motifs compete with each other.
- They have task-dependent selection for specific behavioral motifs
- Abrupt improvement happens between day breaks
Take home message: biological motor learning operates through a select-and-refine strategy rather than generate-from-scratch strategy
Shared Principles of Artificial and Biological Learning
| Principle | AI Side | Biological Side |
|---|---|---|
| 1. Learning builds from compositional primitives | High-level cognitive functions of large models emerge from the self-organization and combination of minimal underlying “computational modules” (especially induction heads), like cells and building blocks. | Rats’ complex trial-and-error behaviors are constructed by splicing and combining basic discrete actions (“syllables”) into fixed “motifs”. They select and combine from an innate action repertoire, rather than creating from thin air. |
| 2. Learners extract structures from the environment | Minimal networks can read context and spontaneously extract the “hidden transition rules” behind Markov sequences, completing the inference from unigram to bigram. | Rats in the experimental box, relying only on extremely sparse water-drop reward signals, can gradually explore and extract the hidden rules (i.e., the 1.5-second action template set by the system). |
| 3. Early experience strongly shapes later learning | A small early-training driving bias, induced jointly by the training distribution and optimization dynamics, strongly shapes later learning. | The passing line (threshold) setting in the early tasks for rats is crucial. If the threshold is too high at the beginning, rats will never learn; if the early threshold is low, allowing them to grasp the trick before increasing the difficulty, their performance will keep improving. |
| 4. Learning often proceeds through abrupt transitions | The prediction error rate during large model training will suddenly drop down a “loss cliff” after a long period of stagnation, resulting in an algorithmic “phase transition” (epiphany). | The action accuracy of rats usually does not progress at a uniform speed during same-day practice, but experiences sudden performance leaps the next day (after errors are repaired during offline states like sleep). |