Every production LLM system starts with a deployment decision: call a commercial model through an API, or run an open-weight model on infrastructure you control. The choice shapes cost, quality, risk and how much your team has to operate. There is no universal answer, but there is a clear set of questions.
The factors that matter
- Data sensitivity and contracts. Some data cannot leave your environment, by law, by contract or by policy. That settles the question for those workloads.
- Quality for the task. The largest commercial models still lead on the hardest reasoning tasks. For many focused tasks, such as extraction, classification or answering from your own documents, a well-chosen and possibly fine-tuned open-weight model performs comparably.
- Cost structure. APIs are priced per token, which is efficient at low or unpredictable volume. Self-hosting is a largely fixed cost, which wins at sustained high volume.
- Latency and availability. Local models avoid network round trips and third-party outages, which matters for interactive and industrial uses.
- Operational burden. Self-hosting means owning hardware, serving, scaling, monitoring and upgrades.
- Change control. API models can change underneath you. A self-hosted model changes only when you decide.
Why many end up with both
The pragmatic answer is often a hybrid. Sensitive workloads run on private models inside your environment; non-sensitive tasks that benefit from frontier capability use a commercial API. A routing and policy layer decides which request goes where, applies the same guardrails and logging to both, and makes it possible to switch providers without rewriting applications.
A short checklist
- Classify the data each use case touches.
- Define the quality bar with a test set before choosing a model.
- Estimate volume and model the cost both ways.
- Decide who will operate the system, and whether they can.
- Build an evaluation suite that runs on every model or prompt change.
- Keep the application independent of any single model provider.
Answer these honestly and the deployment choice usually becomes obvious.
