XALIMANEWS- In an interview broadcast on his X platform, Elon Musk raised the problem of the shortage of data for training artificial intelligence. The solution would lie in synthetic data, which is not without risk…
All of humanity’s knowledge is no longer enough to train artificial intelligence. This is Elon Musk’s assertion during a conversation with Mark Penn, president and CEO of Stagwell, broadcast live on X (formerly Twitter). The problem of a shortage of data is one of the main weaknesses of current large language models, which has been discussed for some time.
« We have now exhausted the cumulative sum of human knowledge… in training AIdeclared the billionaire. This happened last year. » The solution lies in synthetic data according to many specialists, including Elon Musk. This means that AIs will need to be trained on AI-generated data and evaluate themselves.
Towards a collapse of AI models caused by synthetic data?
The use of synthetic data is nothing new. In addition to overcoming the shortage of data, this also reduces the cost of training. The startup Writer claims to have been able to lower the cost of training its Palmyra X 004 model to $700,000, compared to a bill of $4.6 million if it had not used synthetic data. Microsoft indicated that it used synthetic data for its open Phi-4 model, as did Google with its Gemma model, Anthropic with Claude 3.5 Sonnet and Meta with Llama.
One of the problems, mentioned by Musk, is the risk of hallucinations. What happens when an AI is trained on data produced by another AI that contains errors? AI degenerates, in a phenomenon called model collapse, or model collapse. The risk is a reduction in creativity and an increase in bias, which could force companies to rethink how large language models are designed.
