Let’s start at the beginning: artificial intelligence was thought of as a replica of the human brain. AI is supposed to think and learn like humans, and to do so autonomously, that is to say without being explicitly programmed to do so. AI is called “generative” when it is capable of creating content (text, image, sound) from this learning.
Until then, you follow us. Now hang on, we’re entering the machine’s lair.
How does ChatGPT answer my questions?
Let’s say I ask “Who am I?” » at ChatGPT. The AI will transform each word into a series of numbers, called “vectors”. This question is therefore composed of several vectors, which translate both the meaning of each word, their association and the context of the question (for example, the previous exchanges that I have had with ChatGPT).
These vectors pass through a multitude of “layers of neurons” (which process the information) and “layers of attention” (which sort and prioritize them). Then, the AI offers its response. It always starts with a first word (a vector transformed back into a word). Then writes a second one, which she judges most likely to be placed next to this first word, always depending on the context. For example, if we talk about a tree, the probability that we then talk about a leaf increases. And so on.
In our example, ChatGPT responds in four seconds: “You are yourself, unique and irreplaceable! »… Um mom, is that you?
How did AI become so intelligent?
Let’s go back: associating tree and leaf is something that the AI learned on its own. Well almost. Scientists first designed mathematical models to show it how to learn (this is machine learning). Then the AI was trained on a very large amount of data, for example everything found on the internet (we speak of deep learning). By being confronted with different situations, the AI gave more or less importance to words depending on the context. From this learning, the ChatGPT model has developed 175 billion parameters on its own which allow it to make decisions.
Ok so the AI lives its life quietly in its own corner? Indeed, it is the autonomous setting of these parameters which makes Alexandre Défossez, AI researcher at the French Kyutai laboratory, say that the tool is indeed “a black box”. If we take a photo of the process in the middle of a calculation, we are “unable to interpret the sequences of numbers in front of us”, he explains. Impossible to say “at that moment, the AI does this action”. He concludes : “Hence a certain form of anthropomorphism (when we attribute human characteristics to something, Editor’s note) when we talk about these models. Because they surprise us ourselves! »
They sometimes also surprise us with their stupidity. AI has many biases. In question, the lines of codes, written by humans who themselves have biases, and the data provided to it for training, which are not always of good quality (a Nobel Prize in literature VS the writing of my 6 year old nephew). The rate of hallucinations (false or misleading responses) varies greatly depending on the model, from 3% (GPT4) to 27% (Google Palm-Chat), according to a study published at the end of 2023 by Vectara, an American start-up launched by former Google employees.
And to generate an image by AI?
The model is similar to that of a text. Each pixel will correspond to a vector. To generate the image of a cute little kitten that the internet is crazy about, the AI predicts the values of the pixels in the same way that it predicts the importance and value of words to be able to associate them with each other, depending on the context. . The model also trains on an astronomical number of images (up to 5 billion images for the public and free LAION database). This allows it to set its own parameters, make its predictions, and produce its “works” (there is debate on this word, but you get the idea).
When the machine grinds: concretely, where does it happen?
The different AI models (ChatGPT, Bard, Gemini, Midjourney, etc.) are trained on “clusters”, that is to say a set of machines equipped with graphics chips, the famous “GPUs” (Graphics Processing Units). . These machines are located in data centers. There are at least 250 in France alone, according to EY. And thousands around the world. The French laboratory Kyutai, which seeks to create a French chatGPT, does not have its own data center but rents space in the Ile-de-France data center of the company Scaleway (a subsidiary of Iliad, parent company of Free).
“To create our own AI, we have a cluster of 128 machines, each with 8 chips. These two figures multiplied, that’s more than 1,000 GPUs in total”, explains Alexandre Défossez. Which corresponds, he explains to us, to 1,000 electric radiators which run constantly. Electricity, cooling of data centers (by air conditioning), manufacturing of electronic components… AI sucks a lot of energy.
Once the model is trained, it then consumes less to just answer questions. But with more than 180 million monthly users claimed on ChatGPT, that raises questions, and calculations in the model, and data centers that heat up…
