Generative Pre-trained Transformer [Neurosurgery Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== Generative Pre-trained Transformer ======

(GPT) is a type of large language model (LLM) developed by OpenAI that uses deep learning to understand and generate human-like text. Here's a breakdown of the term:

🔹 Generative
The model can generate text — it produces coherent and contextually relevant output such as sentences, paragraphs, or entire documents.

🔹 Pre-trained
Before being fine-tuned or used for specific tasks, GPT is pre-trained on a massive dataset of text from books, websites, and articles. This allows the model to learn grammar, facts about the world, and linguistic patterns.

🔹 Transformer
GPT is based on the Transformer architecture, a deep learning model introduced in 2017. The transformer allows efficient handling of long-range dependencies in text using a mechanism called self-attention, which helps the model understand the relationships between all words in a sentence, not just adjacent ones.

Summary Definition:
GPT is an AI model that uses the Transformer architecture and is trained on large amounts of text to generate human-like responses in natural language.

It is the foundation of tools like ChatGPT, and has been widely used in tasks such as:

Conversational agents

Text completion

Translation

Summarization

Code generation