Google Research
Original published: Updated on DevPick:
The sleep framework: A new way for LLMs to retain knowledge in continual learning
A new framework introduces a sleep phase in continual learning, allowing large language models to integrate new facts and enhance reasoning while mitigating catastrophic forgetting.
Key points
- •The framework introduces a 'sleep phase' into the continual learning process.
- •This allows large language models to continuously integrate new facts.
- •It mitigates catastrophic forgetting and static knowledge cutoffs while enhancing reasoning capabilities.
#Research & Papers#LLM / Model Trends