I need to eat my words.
I just explained that LLMs are static after training and can’t truly adapt to new information after training.
Google’s new paper “Nested Learning: The Illusion of Deep Learning Architectures” shows a path beyond that limitation.