A few weeks ago, experts highlighted that we have reached the limit of training data for artificial intelligence. In the wake of ChatGPT’s popularity, many companies have sought to emulate its capabilities, including Google, Apple, and Meta, which are working to offer generative AI assistants.
However, the tech industry’s most recognized voice has revealed that the data needed to train artificial intelligence models is no longer available and there is an urgent need to find a solution.
>> You won’t believe it: Elon Musk and Mark Zuckerberg unite against the creators of ChatGPT! <<
As TechCrunch points out, Elon Musk has claimed that we have reached the limit of real-world data for training AI models. He specifies, however, that this situation did not occur now, but that it took place a few months ago, still in 2024. Elon Musk thus underlines the words of Ilya Sutskever, former chief scientist of OpenAI, which already indicated in 2022 that the industry had reached a situation he called “peak data”. In light of what happened, the billionaire who is in conflict with the creators of ChatGPT therefore proposed a temporary solution to continue training the AI models.
Large companies already use synthetic data
For Elon Musk, a great option is to use data generated by artificial intelligence itself, an aspect known as “synthetic data.” This will help build machine learning pipelines and follow in the footsteps of companies like Microsoft, Meta, OpenAI, and Anthropic that are already following this path. In fact, according to some estimates, by 2024, 60% of the data used will be synthetic, since in addition to all of the above there will be other benefits such as cost reduction.
>> “I had to beg him every time…“: Former Tesla employee describes what it’s like to work with Elon Musk on a daily basis <<
However, some research suggests that using synthetic data could lead to the collapse of different models, reducing creativity and increasing bias in results. Indeed, if the data generated presents biases or limitations, the models trained with this data will reproduce these same problems in their results. However, this does not seem to be a limitation either for Elon Musk or for companies such as Microsoft, Google or Anthropic, given that they have used them in models such as Phi-4, Gemma and Claude 3.5 Sonnet.
Article written in collaboration with our colleagues from 3dGames.
