StackLift
All terms

Glossary

Open-weight model

An AI model whose trained weights you can download and run yourself, under the publisher's license.

An open-weight model is an AI model whose trained parameters, called weights, are published so anyone can download, run and often fine-tune them under the publisher's license. It is not the same as open source: the training data and code may not be released, and the license can limit use. Open-weight models let you host the model yourself and keep data inside your own environment.

The Open Source Initiative has published its own definition of open source AI, which sets a higher bar than releasing weights alone. Many popular models are open-weight under custom licenses, so read the license before you build on one.

Choosing one means taking on hosting, security updates and monitoring that a hosted API would otherwise handle. Teams pick open-weight models for data control, predictable cost at volume or the freedom to fine-tune. See [private LLM development](/solutions/private-llm-development).

Start in 60 seconds

Put this to work on your project.

Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.

The complete guide

Open-weight model: the complete guide

Updated November 18, 2026 · Published by , Founder, StackLift AI. Written with AI assistance; how we make content.

Short answer

An open-weight model is an AI model whose trained weights are published so you can download and run it yourself, usually under a license with conditions. It is not automatically open source: the training data and code may stay private. You gain control over data and hosting, and you take on the work of running it.

What is an open-weight model?

A language model is a very large file of numbers, called weights, learned during training. A hosted model keeps those weights on the provider's servers and lets you reach it through an API. An open-weight model publishes the weights, so you can download them and run the model on your own hardware or your own cloud account.

Having the weights lets you do things a hosted API does not: run the model inside your network, choose the hardware, keep prompts and documents out of a third party's systems, and adapt the model with fine-tuning or LoRA. It also makes you responsible for hosting, updates and monitoring.

Most people use the words open-weight, open model and open-source LLM loosely. The differences matter for legal and procurement reasons, so they are worth separating.

What is the difference between open-weight and open source?

Open source, as defined by the Open Source Initiative, means you may use, study, modify and share the work, which for AI includes enough information about the data and code to reproduce it. Many popular models release weights only, under a custom license. They are open-weight, not open source in that stricter sense.

Closed, open-weight and open-source models
Closed (hosted API)Open-weightOpen source AI
Weights availableNoYesYes
Training data and codePrivateUsually privateShared enough to study and reproduce
LicenseProvider terms of servicePublisher's model license, often with conditionsAn open license with few restrictions
Where it runsProvider's serversYour choiceYour choice
Can you fine-tune itOnly if the provider offers itUsually yesYes
Who maintains itProviderYou, with the publisher's updatesYou, with community updates

The practical point: read the license. Terms can limit commercial use above a certain size, restrict some uses, require attribution or forbid using outputs to train other models. Have your legal team approve the license before you build a product on it.

Why do teams choose open-weight models?

  • Data control: prompts and documents stay inside your environment, which helps with strict privacy or residency rules.
  • Predictable cost at volume: at steady, high usage, running your own hardware can cost less than paying per token. At low or spiky usage, an API is usually cheaper.
  • Customization: you can fine-tune for your domain, format or tone.
  • No dependency on one provider's pricing, rate limits or model retirement schedule.
  • Offline and edge use: some deployments have no reliable internet or must run on a device.
  • Auditability: you can pin an exact model version and keep it unchanged for as long as needed.

None of these reasons is automatic. Each has to be weighed against the effort of running a model yourself, which the next sections cover. The open-source LLM vs. API comparison sets the options side by side.

What does it take to run an open-weight model?

Running a model well is a real operations job. You need hardware that can hold the model and serve many requests quickly, software that serves it, and the usual production habits: monitoring, scaling, security updates and cost tracking.

  1. Pick a model that fits the task, your hardware budget and the license. Smaller models are cheaper to run; see small language models.
  2. Choose where it runs: your own servers, a cloud GPU instance, or a managed service that hosts open-weight models for you.
  3. Use an inference server such as vLLM, Hugging Face Text Generation Inference, llama.cpp or Ollama to serve the model. See inference.
  4. Apply quantization if needed, which stores weights at lower precision so the model fits on smaller hardware, at some cost in quality.
  5. Put an API layer, authentication, rate limits and logging in front of it.
  6. Add monitoring for latency, throughput, errors, GPU use and answer quality. See observability.
  7. Plan updates: new model versions, security patches and a way to roll back.

Managed options sit in between. Cloud platforms such as AWS Bedrock, Azure and Google Vertex AI host a range of open-weight models behind an API. You keep some benefits, such as choice of model, without running GPUs. See AWS Bedrock and Google Vertex AI.

Open-weight model or hosted API: how do you decide?

Start from the requirement, not the preference. A hosted API gets you running quickly and gives access to the strongest models. An open-weight model makes sense when a specific need, such as data location or volume, points to it.

Decision guide
If this is trueLean toward
You need to ship quickly and usage is uncertainHosted API
Data must not leave your environmentOpen-weight, self-hosted or in your own cloud account
You need the strongest general reasoning availableHosted API, tested against your tasks
Usage is high and steady, and the task is narrowOpen-weight, possibly a small model
You have no team to run infrastructureHosted API or a managed open-weight service
You need to fine-tune on private dataOpen-weight, or a provider's fine-tuning offer if the rules allow
You want to avoid depending on one vendorOpen-weight, or a design that can swap models

Many teams use both: a hosted model for hard, low-volume tasks and an open-weight model for high-volume, narrow ones. Model routing decides per request which to use.

Where are open-weight models used?

  • A hospital group that must keep patient notes inside its own network summarizes them with a model hosted in its private cloud.
  • A law firm runs a model on its own servers to search and summarize client documents without sending them to an outside API.
  • A manufacturer deploys a small model on factory-floor devices that have limited connectivity.
  • A bank uses a self-hosted model for internal document search so that data residency rules are met.
  • A support team routes simple, high-volume tickets to a small open-weight model and sends harder ones to a hosted model.
  • A product team fine-tunes an open-weight model to produce a company-specific output format.

These are typical patterns, not results from any named organization. For the build side, see private LLM development and the custom LLM cost page.

What are the risks and limits of open-weight models?

Risks and how to manage them
RiskWhy it mattersHow to manage it
License limitsTerms may restrict some uses or large-scale commercial useRead and approve the license before building; record the version used
Operations burdenYou run the servers, scaling and updatesUse a managed host, or budget for platform staff
SecurityModel files and serving software can have vulnerabilitiesDownload from trusted sources, verify files, patch servers and restrict network access
Quality gapA model may trail the top hosted models on hard tasksTest on your own cases; use a small model only where it passes
Safety featuresBuilt-in safeguards can be weaker or removableAdd your own guardrails and test with red teaming
Hidden costGPUs idle between requests still cost moneyMeasure real utilization; scale down when quiet

Open weights do not make a system secure or private on their own. You still need access controls, logging and a plan for prompt injection and other attacks.

How do you evaluate an open-weight model for your use?

Public leaderboards are a starting list, not an answer. They measure general tasks, and your use is specific. Build a small test set from real examples of your task and run each candidate model against it.

  1. Write down the task, the input format, the expected output and what counts as wrong.
  2. Collect real examples, including hard and unusual ones, and mark the correct answers.
  3. Shortlist models by license, size and language support.
  4. Run each model with the same prompts and settings and score the results.
  5. Measure speed and memory on the hardware you can afford.
  6. Compare with a hosted model on the same cases, so you know what the gap is.
  7. Decide, and save the test set so you can rerun it when a new model appears.

Re-evaluation is routine because the field moves quickly. A reusable test set makes switching models a measured decision. See the AI agent evaluation checklist for a method.

What should legal and governance teams check?

Open-weight models raise questions that a hosted API's contract usually answers for you. With a hosted API, the provider's terms cover usage rights and data handling. With open weights, the model license and your own policies carry that load.

  • License terms: commercial use, user-count or revenue thresholds, field-of-use limits and attribution.
  • Output rights: whether you may use outputs to train or improve other models.
  • Provenance: where the weights came from, and whether the publisher documents the training data.
  • Version control: which exact model version is in production, and who approved it.
  • Policy fit: how the model fits your AI governance rules and responsible AI commitments.
  • Regional rules: some laws and sector rules affect where data can be processed.

The AI governance policy checklist gives a structure for these decisions. Keep a record of the license text as of the date you adopted the model, since terms can change between versions.

What does an open-weight model mean for cost and timeline?

You trade per-token fees for infrastructure and engineering effort. The total depends on model size, request volume, uptime needs, the hardware you use and how much of the platform work you do yourself or hand to a managed service.

  • Model size and quantization, which set the hardware needed.
  • Expected volume and peak load.
  • Availability needs, including redundancy and failover.
  • Security and data-residency requirements.
  • Whether you fine-tune, and how much data preparation that needs.
  • Monitoring, evaluation and ongoing updates.

For hosted prices to compare against, see the LLM pricing page. Run the free estimate to scope a private model project. Most first versions take 6–16 weeks depending on scope.

How does StackLift approach open-weight models?

Model choice follows the requirement. A project starts with structured discovery and an architect-reviewed scope, which settles whether a hosted API, a managed open-weight service or a self-hosted model fits your data and volume.

Where a private deployment is needed, StackLift delivers it under the private LLM development service, with CI/CD, observability, security review and rollback planned from sprint one.

Questions people ask

Is an open-weight model free?

The weights are often free to download, but running the model is not free. You pay for hardware, hosting, engineering and monitoring. Some licenses also place conditions on commercial use.

Is an open-weight model the same as an open-source model?

Not always. Open-weight means the trained weights are public. Open source in the strict sense also needs enough information about code and data to study and reproduce it, which many popular models do not provide.

Are open-weight models as good as hosted models?

It depends on the task. Some match hosted models on narrow tasks, and others trail on hard reasoning. Test candidates on your own examples instead of relying on public rankings.

Can I fine-tune an open-weight model?

Usually yes, if the license allows it. Methods such as LoRA adapt a model with less hardware than full training. Check the license and make sure you have the right to use your training data.

Is self-hosting a model more private?

It can be, because prompts and documents stay inside your environment. Privacy still depends on how you secure access, logs and storage, so hosting alone is not enough.

What hardware do I need to run one?

That depends on the model's size and the speed you need. Larger models need GPUs with more memory, while smaller or quantized models can run on modest hardware. Check the model's documentation.

Can I switch between models later?

Yes, if you design for it. Keep prompts and tools portable, put the model behind your own interface and keep a test set, so a switch is measured instead of a leap.

How often do open-weight models change?

New models and new versions appear often, and older ones may stop receiving updates from their publisher. Pin the version you run, keep a test set, and review the field on a regular schedule so that you can adopt a better model when it makes sense. Treat every upgrade like any other release, with testing and a rollback plan.

Sources

Sources and further reading

The standards, rules, research and official documentation this page draws on.