StackLift
All terms

Glossary

Large language model (LLM)

An AI model trained on huge amounts of text that reads and writes language, code and structured data.

A large language model (LLM) is a neural network trained on very large amounts of text so it can predict and generate language. It powers chat assistants, summarizers, code tools and AI agents, and works by predicting the next token in a sequence, one token at a time.

LLMs come from a handful of model families, such as OpenAI GPT, Anthropic Claude, Google Gemini and open-weight models you can host yourself. They differ in quality, speed, context size, price and how well they follow instructions or call tools.

An LLM on its own only knows what it learned in training. Real products wrap it with your data (see RAG), tools, guardrails and tests, which is where most of the engineering effort goes.

Start in 60 seconds

Put this to work on your project.

Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.

The complete guide

Large language model (LLM): the complete guide

Updated November 18, 2026 ยท Published by , Founder, StackLift AI. Written with AI assistance; how we make content.

Short answer

A large language model (LLM) is a neural network trained on very large amounts of text so it can read and generate language, code and structured data. It works by predicting the next token in a sequence. On its own it knows only what it learned in training, so real products add your data, tools, guardrails and tests around it.

What is a large language model in plain terms?

Think of an LLM as software that has read an enormous amount of text and learned the patterns of language well enough to continue almost any piece of writing. Give it a question and it writes an answer. Give it a contract and a request to summarize, and it writes a summary. It does this one piece at a time, choosing a likely next piece based on everything so far.

Precisely, an LLM is a neural network, almost always a transformer, with billions of adjustable weights. It is trained to predict the next token in a sequence. Tokens are chunks of text, often a word or part of a word. Doing that prediction well across huge amounts of text forces the model to pick up grammar, facts, reasoning patterns and the structure of code.

The name has three parts. "Large" refers to the number of weights and the scale of training. "Language" refers to the main data, text, though many current models also read images, audio and documents. "Model" means a trained artifact you call, not a program someone wrote line by line.

How does an LLM work?

  1. Pre-training: the model reads a very large body of text and learns to predict the next token. This is the most expensive step and is done by the model's maker.
  2. Post-training: the maker teaches the model to follow instructions, hold a conversation and decline unsafe requests, using examples and human or automated feedback.
  3. Your request: you send a prompt, which is text plus any documents, examples and tool descriptions you include.
  4. Tokenization: the prompt is split into tokens and turned into numbers the model can process.
  5. Generation: the model scores every possible next token, picks one, adds it to the sequence and repeats until it is done.
  6. Response: the tokens are turned back into text and returned, often streamed as they are produced.

Two settings shape the output. The context window limits how much text the model can consider at once. Temperature controls how random the choice of next token is. Running the model to produce an answer is called inference, and its cost grows with the amount of text in and out.

One consequence is worth remembering: an LLM does not look facts up. It generates text that fits the pattern. That is why it can sound confident and still be wrong, which is called a hallucination.

What kinds of LLMs are there?

Common ways to group models
GroupWhat it meansExamples and notes
Hosted frontier modelsRun by the maker, called through an APIOpenAI GPT, Anthropic Claude, Google Gemini. Strong quality, no hardware to run
Open-weight modelsWeights you can download and host yourselfLlama and Mistral families, among others. More control, you run the hardware
Small language modelsSmaller, faster, cheaper modelsGood for narrow tasks, on-device use and high volume
Reasoning modelsSpend extra computation thinking before answeringUseful for multi-step problems, slower and costlier per answer
Multimodal modelsAccept images, audio or documents as well as textUseful for forms, screenshots, scans and voice
Embedding modelsTurn text into vectors for searchSee embeddings; they do not write text

Models differ in quality, speed, context size, price, languages, tool-calling reliability and the terms under which you can use them. Compare current prices on the LLM pricing page, and see Claude vs. GPT and Gemini vs. GPT for head-to-head notes.

What are LLMs good at, and where do they struggle?

Strengths and weaknesses
Good atStruggles with
Summarizing, rewriting and translating textExact arithmetic and precise counting without tools
Extracting fields from messy documentsFacts it was never trained on, or that changed recently
Answering questions over supplied documentsKnowing when it is wrong
Drafting code and explaining itLong chains of exact steps with no checks
Classifying and routing requestsStaying consistent: the same prompt can give different answers
Conversational interfaces to softwareKeeping private data separate unless you design for it

The pattern behind the table: LLMs are strong at language work where a good-enough answer is useful and a person or a check can catch errors. They are weak where one wrong detail is costly and nothing verifies it.

What are LLMs used for in business? Real-world examples

  • Support: draft replies, summarize tickets and answer questions from a help center.
  • Sales and marketing: summarize accounts, draft outreach, tag and score inbound requests.
  • Operations: extract data from invoices, contracts and forms and route exceptions to people.
  • Software teams: explain code, write tests, propose fixes and review changes.
  • Finance and legal: first-pass review of documents, with a person making the decision.
  • Internal knowledge: search company documents in plain language, with sources shown.
  • Agents: systems where a model plans steps and calls tools, see AI agents.

In each case the value comes from the system around the model: access to the right data, the right tools, checks on the output and a way to hand off to a person.

How do you choose an LLM for a product?

Start from the task, not the brand. Write down what the model must do, what data it will see, how fast it must answer and what a wrong answer costs. Then test two or three candidates on a set of real examples from your own work. Public rankings say little about your documents and your users.

  • Quality on your own test set, not on general benchmarks.
  • Cost per request at your expected volume, including input and output tokens.
  • Speed: time to first token and total time, which matter in chat and voice.
  • Context size, if you pass long documents.
  • Tool calling and structured output reliability, if the model drives software.
  • Data terms: where data goes, whether it is stored and whether it trains models.
  • Fallback options and how easy it is to switch providers later.

Keep the model behind a thin layer in your code so you can switch or route between models later. See model routing and AI cost optimization, and the question of hosting in open-source LLM vs. API.

What does adding an LLM to software involve?

The model call is a small part of the work. Most effort goes into everything that makes the call useful and safe.

  1. Define the job, the user and what a good answer looks like.
  2. Build a test set of real examples with expected results.
  3. Write and refine the prompt (prompt engineering).
  4. Connect your data, usually with RAG, so answers rest on your documents.
  5. Add tools, so the model can act, with limits on what it may do (tool calling).
  6. Add guardrails and a human review step where mistakes are costly.
  7. Release behind a switch, watch real usage and keep improving (LLMOps).

Customizing a model has a ladder: better prompts first, then retrieval, then fine-tuning only if the first two fall short. See RAG vs. fine-tuning. For a hands-on view of the work, read LLM integration.

What are the main risks of using LLMs, and how do you manage them?

Risks and common controls
RiskWhat it looks likeControl
Wrong answersFluent text that is incorrectGround answers in documents, show sources, test on real cases
Prompt injectionHidden instructions in a document or web page redirect the modelSeparate instructions from data, limit tool permissions, see prompt injection
Data exposurePrivate data sent to a provider or shown to the wrong userAccess control on retrieval, provider terms, redaction
Cost surprisesLong prompts and heavy usageToken budgets, caching, smaller models for easy tasks
Quality decayA model update changes behaviorPin versions, rerun tests, see model drift

An AI governance policy turns these controls into rules people follow. The AI red teaming practice tests the system the way an attacker would before launch.

What mistakes do teams make, and how do you measure whether an LLM feature works?

  • Judging a model from a few impressive demos instead of a test set.
  • Sending whole documents when a few relevant passages would do, which raises cost and noise.
  • Trusting the output without a check, a source link or a person where errors matter.
  • Hard-wiring one provider into the code with no way to switch.
  • Launching with no logging, so failures cannot be diagnosed.
  • Treating the prompt as finished instead of as something that changes with the product.

Measure the feature by its job. Useful signals include how often answers are accepted without edits, how often reviewers correct them, how many requests need a person, how long tasks take compared with before, how often users give up, and cost per completed task. Review failures weekly and add each one to the test set so the same mistake is caught next time. See AI observability and agent evaluation for the tooling side.

When is an LLM not the right tool?

Skip the LLM when a rule, a query or a classic model does the job reliably. Calculations, lookups by ID, strict validation and anything that must give the same answer every time belong in ordinary code. Using a model there adds cost and variation for no gain.

Be careful also where an error cannot be caught, such as unreviewed decisions about people's money, health or legal position. In those cases use the model to prepare work and keep a person in the loop (human-in-the-loop).

What does an LLM project cost, and how does StackLift approach it?

Cost has two parts: building the system and running it. Build effort depends on the data sources, the number of tools and integrations, the quality bar and the review steps. Running cost depends on model prices, traffic and prompt length. See cost of an AI app and LLM integration cost.

StackLift starts with structured discovery and architect-reviewed scope, then ships through a governed pipeline: agents work on feature branches, CI/CD and observability are in place from sprint one, and a person reviews the quality, security and rollback plan. The free estimate returns cost, timeline and team in about five minutes with no signup.

Questions people ask

What does LLM stand for?

Large language model. It is a neural network trained on large amounts of text to read and generate language.

Is ChatGPT an LLM?

ChatGPT is a product built on LLMs from OpenAI. The LLM is the model; the chat app adds the interface, memory, tools and safety features around it.

How is an LLM different from regular AI or machine learning?

Classic machine learning models are usually trained for one task, such as scoring fraud risk. An LLM is trained on broad text and can be directed to many language tasks with instructions, without training a new model each time.

Can an LLM use my private data?

Yes, if you supply it. The usual approach is retrieval: the system finds the right documents and gives them to the model with the question. Check the provider's data terms and enforce access control on what is retrieved.

Do LLMs learn from my conversations?

Not while you chat: the model's weights do not change. Whether a provider may store prompts or use them for future training depends on its terms and your plan, so read them before sending sensitive data.

What is the difference between an LLM and an AI agent?

An LLM generates text. An agent is a system that uses an LLM to plan steps and call tools to complete a goal. See agentic AI and AI agent vs. chatbot.

Why do LLMs make things up?

They generate likely text and do not check facts. When the answer is not in their training or in the supplied context, they may still produce a fluent guess. Grounding in documents and verifying outputs reduce the problem.

Should I host my own LLM?

Only if you need control over data location, cost at very high volume or customization. Many products run well on hosted APIs. See the open-source LLM vs. API comparison.

Sources

Sources and further reading

The standards, rules, research and official documentation this page draws on.