TL;DR
Thorsten Meyer AI published a field note arguing that self-hosted open models can be cheaper than paid APIs when usage is steady and high enough. The report says the real comparison is total cost of ownership against per-token pricing, not download price against API price.
Thorsten Meyer AI says companies weighing paid on-prem AI services against free-to-download open models should focus on operating cost, not download price, because self-hosting can beat API pricing only when usage is sustained, predictable and large enough.
The field note addresses a question raised after a prior piece on Mistral and European AI sovereignty: why pay a vendor to run models on-premises if open-weight systems such as Qwen can be downloaded at no charge?
The answer, according to Thorsten Meyer AI, is that the model weights may be free, but inference is not. The post lists hardware, electricity, operations time, model updates, quantization work, queue health, retries, context handling and depreciation as real costs that sit outside the word “free.”
The article says the economic crossover depends on workload. In its illustrative model, API use wins for low or uneven demand, while owned hardware can win once monthly token volume is steady and high. One example in the post places break-even near 80 million tokens a month, though the author labels that figure illustrative rather than a vendor quote.
Why It Matters
The piece matters for teams deciding whether to keep buying AI by the token or invest in their own inference stack. It challenges both common shortcuts in the debate: that open models are costless because the weights can be downloaded, and that cloud APIs are always the cheaper or easier choice.
For companies with privacy, sovereignty or data-control requirements, the argument is also about where data goes. The post says self-hosting makes data location a structural feature of the system, while hosted APIs and vendor-run deployments rely on contracts, controls and trust in the provider.
The cost question is becoming sharper as open-weight models improve and as per-token spending grows inside production workflows. If a publisher, software team or enterprise workload sends large volumes of repeated traffic through an AI system, even a small per-token spread can become a material budget issue.
Local LLM Inference Optimization: A Comprehensive Guide to Quantization, Hardware Acceleration, and Efficient Private AI Deployment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Background
The field note places the debate in a mid-2026 market where Western frontier models from companies such as Anthropic, OpenAI and Google remain ahead on the hardest long-horizon tasks, while Chinese open-weight and open-access systems including DeepSeek, Kimi, GLM and Qwen have narrowed the gap on many workloads, according to the post.
Thorsten Meyer AI says open models may trail closed frontier systems by six to 12 months on the most demanding tasks, but can be close enough for many production jobs when paired with a strong application harness. The post also says Apple Silicon systems with large unified memory and mixture-of-experts models have made local inference more practical for smaller operators than it was in earlier AI cycles.
“The weights are free to download. Running them well is not.”
— Thorsten Meyer AI
“The honest comparison is total cost of ownership vs. per-token API.”
— Thorsten Meyer AI
“Below some usage level the API wins decisively. Above some sustained, predictable volume, owned hardware wins.”
— Thorsten Meyer AI
Julia High Performance: Optimizations, distributed computing, multithreading, and GPU programming with Julia 1.0 and beyond, 2nd Edition
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
Several points remain workload-specific or based on the author’s estimates. The break-even level can change with hardware prices, electricity costs, utilization, engineering skill, model quality, context length, latency needs and vendor API pricing. The post’s example near 80 million tokens a month is presented as illustrative, not as a fixed market price.
It is also unclear how long current open-model cost advantages will last. API vendors can cut prices, frontier labs can widen the quality gap, and new open-weight releases can change the comparison again.
Hewlett Packard Enterprise High-End AI Server 52-Core 1024GB RAM 3.84TB H100 (96GB) DL380 G10 (Renewed)
HPE Proliant DL380 G10 8-Bay SFF Server | 2x Platinum 8164 2.0GHz 26-Core CPU (52-Cores Total)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
Teams facing this decision will likely model their own monthly token volume, latency needs, privacy requirements and staffing capacity before choosing. The next practical step is a workload-specific cost test: compare a real API bill against a self-hosted pilot that includes hardware, power, maintenance time, quality checks and depreciation.
Local LLM Inference Optimization: A Comprehensive Guide to Quantization, Hardware Acceleration, and Efficient Private AI Deployment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Does free open-source AI mean free AI operations?
No. The field note says the model weights may cost nothing to download, but running them requires hardware, power, maintenance, monitoring and supporting software.
When can self-hosting beat a paid API?
According to Thorsten Meyer AI, self-hosting is most likely to win when usage is high, steady and predictable, and when the operator has enough skill to keep the inference stack reliable.
When does an API still make more sense?
The post says APIs tend to win for low-volume, uneven or fast-changing workloads, especially when teams need top frontier performance without managing hardware and operations.
Is the 80 million token break-even point fixed?
No. The article describes that figure as an example. Actual break-even depends on model choice, hardware cost, electricity, utilization, labor and API pricing.
Why does data sovereignty affect the decision?
Self-hosting can keep data inside an organization’s own systems. The post frames that as a different kind of control than relying on a vendor’s contractual promises.
Source: Thorsten Meyer AI
