#synthetic-data

Synthetic Data for Fine-Tuning: The Engineering Guide

Training on AI-generated data is now the default path for open-model fine-tuning. The pattern works — but it has failure modes that are not visible in benchmark scores. This paper maps five practical methods (Self-Instruct, Evol-Instruct, Orca, phi, SPIN), the model collapse risk that applies to all of them, and the design checklist that keeps a synthetic data pipeline from degrading.