The Price of New Knowledge: Catastrophic Forgetting
Why AI models 'forget' how to do math when they learn how to write poetry.
Humans are great at Incremental Learning. You can learn to ride a bike today, learn a new language next year, and you won't suddenly forget how to ride the bike.
Neural networks, however, suffer from Catastrophic Forgetting.
The Problem: Overwriting Weights
When you train a neural network, you are essentially finding a set of weights that minimize the "loss" on a specific task (e.g., classifying images of dogs).
If you then take that same model and "fine-tune" it on a new task (e.g., classifying images of cars), the gradient descent process will start shifting those weights to minimize the "car loss." Because the model's capacity is limited, the weights that were critical for "dog classification" get overwritten. Suddenly, the model is an expert on cars but has no idea what a dog is.
Why This Matters for LLMs
This is the biggest hurdle in the race for "personalized AI." If you want an AI to learn your specific writing style or your company's private data, you can't just keep training it on that data forever. Eventually, it will "forget" the general knowledge (like how to speak English properly) that made it useful in the first place.
The Solutions: EWC and Replay
Engineers use several tricks to combat this:
- Elastic Weight Consolidation (EWC): This identifies which weights are "most important" for the old task and makes them "harder" to change during new training.
- Experience Replay: The model is trained on a "mix" of new data and a small sample of old data to "remind" it of what it already knows.
Solving catastrophic forgetting is the "Holy Grail" of AI. Until we do, we will always be stuck with models that are static snapshots of a moment in time, rather than living, learning systems.