ML//training//dataset//data contamination

Data contamination is the leak of benchmark test data into a model's training data, a flaw that lets the model memorize the answers instead of learning the skill and so inflates its score. It is hard to detect at scale: nobody can say how much of GPT-4's MMLU score is memorization.

Data contamination is the leak of benchmark test data into a model's training data, a flaw that lets the model memorize the answers instead of learning the skill and so inflates its score. It is hard to detect at scale: nobody can say how much of GPT-4's MMLU score is memorization.