ML//training//dataset//synthetic data

Synthetic data is training data produced by other AI models, so that one AI is taught with another's homework, and it works surprisingly well until the errors start compounding and nobody remembers what the original looked like.

Synthetic data is training data produced by other AI models, so that one AI is taught with another's homework, and it works surprisingly well until the errors start compounding and nobody remembers what the original looked like.

The risk is model collapse: errors compound across generations and diversity shrinks, with the tail distribution eroding first.

Carefully curated synthetic data (math proofs, code with verification) works well. The key is external verification, not blind self-training.

In robotics and industry the generator is usually a simulator, not a model: photorealistic renders to train a drone's perception, simulated faults to train a detector that has seen three real failures. Training perception on rendered images is still niche, and it is almost always completed with real data, because the model learns the simulator with its defects (lighting too clean, no motion blur, sensors without their real noise). The distance between the two is the sim-to-real gap, and domain randomization widens the simulated conditions until reality looks like one more sample.

It also builds test sets. Starting from known facts or relations, a generator writes text around them and adds the disturbances real input will have (typos, paraphrases, distracting entities, chains of several facts), so the right answer is known by construction (named entity recognition, multi-hop reasoning). The trap is testing a model on data written by the same model or its family, whose patterns and blind spots are baked into the test; a synthetic benchmark is checked against an independent, human-verified set before its scores are trusted.

Constitutional AI uses synthetic data wisely: the model generates reflection-improved responses, but the constitution provides an external anchor.

The uncomfortable question is what happens when most training data is AI-generated.