Glossary
Inference
Running a trained AI model to produce an output: the step that happens every time someone uses the AI.
Inference is the process of running a trained machine learning model on new input to produce an output, such as an answer, a classification or an image. In an LLM application, every request you send to the model is an inference call. It is distinct from training, which builds the model once, and it is where most ongoing running cost and latency come from.
For language models, inference has two phases: reading the prompt, then generating the reply one token at a time. Prompt length, output length, model size and the hardware or service running it all affect speed and cost.
You can buy inference as a hosted API or run it yourself with a serving engine on your own or rented GPUs. Hosted is simpler; self-hosting gives control over data and cost shape but adds operations work. See [private LLM development](/solutions/private-llm-development).
Start in 60 seconds
Put this to work on your project.
Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.