There is no posttraining data commons
AI summary
Editorial policyThe article argues that while the LLM revolution relied on a pretraining data commons built by humanity for other reasons, no such commons exists for posttraining, which consumes tasks rather than documents.
Key points
- •The LLM revolution was supported by a pretraining data commons that humanity built for purposes other than AI.
- •Posttraining consumes tasks rather than documents, meaning the existing commons structure does not apply to it.
- •The open models community requires a specific data commons for posttraining to sustain its development.
#LLM / Model Trends#Research & Papers