Self Adapting Laguage Models (SEAL)
Date: 19th June 2025
Key Points
- Aims for continual learning in LLMs
- Produces self-edits: synthetic data it can fine-tune itself on.
- It can create this data based on new information.
- SEAL is used for:
- Knowledge incorporation: takes new information and then produces a lot of fine-tuning data
to learn this new info.
- Few-shot learning: sees examples, produces self-edits, performs test-time traininf to learn the
new task. The model also learned to produce hyperparameters for this training.