Transformer

The transformer is a neural-network architecture, introduced by Google researchers in 2017, that revolutionized language processing and is the basis of today’s large models. Its key idea is the "attention" mechanism, which lets the model weigh which words in a text are most relevant to each other, no matter how far apart they are, and thus capture context far better than earlier techniques. It also processes text in parallel rather than word by word, which makes it much more efficient to train at scale. The "T" in GPT ✕ GPT is the family of large language models from OpenAI behind ChatGPT. It generates text by predicting, unit by unit, the most likely continuation. More in the glossary → stands precisely for "transformer." Although it was born for language, it is now used for images, audio and other fields too. It is, without exaggeration, the technical piece that made the current wave of generative AI ✕ Generative AI creates new content —text, images, audio or code— from a prompt, instead of only analyzing existing data. More in the glossary → possible.