While Snowflake marches toward its record IPO, OpenAI has released a language model called GPT-3, and it is changing the conversation about artificial intelligence permanently. Where its predecessor GPT-2 impressed researchers, GPT-3 stuns them, and then it stuns everyone else. It can write essays, answer questions, translate, summarize, and even produce working computer code, often well enough to be mistaken for a capable human, and it does all of this without being specifically trained for any one of those tasks. GPT-3 is the model turning the slow-building promise of the Transformer into an undeniable reality.
The Power of Sheer Scale
The single most important fact about GPT-3 is its size, because size is what makes it remarkable. The Transformer architecture can be scaled up, made bigger and trained on more data, and researchers keep finding that bigger means better. GPT-3 is the dramatic confirmation of that principle. It has 175 billion parameters, the internal adjustable values a model tunes during training to capture patterns in language. To give a sense of the leap, GPT-2 had around 1.5 billion parameters; GPT-3 has over a hundred times more. It is trained on a staggering quantity of text drawn from the internet, books, and other sources.
Think of parameters loosely as the knobs the model can adjust to encode what it has learned. With more knobs and more training text, the model can capture far subtler patterns, more facts, more styles, more of the structure of reasoning embedded in human writing. The jump to 175 billion parameters does not just make GPT-3 incrementally better at the things GPT-2 did; it produces new capabilities that smaller models simply do not have, an effect researchers call emergence, where abilities appear suddenly at sufficient scale rather than improving smoothly.
Learning From Just a Few Examples
GPT-3’s most striking new ability is what researchers call few-shot learning. Earlier AI systems had to be specifically trained for each task: to build a translation system, you trained it on translation; to build a summarizer, you trained it on summaries. GPT-3 can often perform a brand-new task after seeing just a handful of examples written into its prompt, with no retraining at all. Show it two or three examples of the format you want, English sentence followed by French translation, say, and it continues the pattern, translating new sentences it was never trained to translate.
An analogy: earlier AI was like an employee who had to attend a full training course before doing any new task. GPT-3 is like a brilliant generalist to whom you can say, here are a couple of examples of what I need, now do the rest, and they simply get it. This flexibility is revolutionary, because it means a single model can be pointed at an enormous variety of tasks just by describing them, rather than building and training a separate specialized system for each. The practice of carefully writing those instructions and examples is becoming known as prompt engineering, a skill that did not meaningfully exist before GPT-3.
The API Business Model
Rather than release GPT-3’s full workings openly, OpenAI is making it available through an API, an interface that lets other companies send text to the model and get responses back, paying per use, without ever holding the model themselves. This is a significant commercial choice. It means any developer can build applications on top of GPT-3’s capabilities by simply calling OpenAI’s service, while OpenAI retains control of the model and earns revenue from every request. A wave of startups is springing up building products powered by the GPT-3 API, an early preview of an entire industry forming around large language models.
This API approach also deepens the relationship between OpenAI and Microsoft, which has invested heavily in OpenAI and provides the enormous computing infrastructure needed to train and run such a large model. The cost of training GPT-3 is estimated in the millions of dollars of computing time alone, the kind of expense that requires a partner with vast cloud resources. That partnership hints at how the AI boom is intertwining the model-builders with the cloud and hardware providers who supply the raw computing power.
What It Means for the Market
GPT-3 is reshaping the strategic landscape for everyone in data and AI. It proves that a single, massively scaled, general-purpose model can outperform many specialized systems, which shifts the industry’s center of gravity toward building and harnessing these large foundation models. For the data platforms, the implication is profound: enterprises are going to want to use models like this on their own data, which means the platforms that manage enterprise data become the natural place to connect that data to AI. This is the logic that is pushing Databricks to reposition itself as a place to build and deploy AI, and it is why every data platform is racing to add AI capabilities.
It also intensifies the competition among the AI model-builders and their backers. OpenAI’s lead with GPT-3 is spurring Google, which invented the Transformer, and a host of others to accelerate their own large-model efforts, while the cloud giants position themselves to provide the infrastructure all of these models require. The benefit to the world is a genuinely general-purpose AI capability, software that can be adapted to countless tasks just by describing them, putting sophisticated language abilities within reach of any developer with an API key. The risks, the potential for convincing misinformation, the concentration of such powerful capability in a few well-resourced hands, the opacity of how these models reach their answers, become correspondingly more serious. GPT-3 is the moment the AI future stops being theoretical. The one thing still missing is a simple, friendly way for ordinary people to use it.