Lab
Open-weight economics
Size the cluster, see where the money actually goes, and find the volume at which self-hosting an open-weight model stops losing money against a hosted API.
Runs in the browserAdvanced
What actually operates
These controls run for real. They are the point.
What is simulated
Stand-ins. Nothing here touches a real system.
One token, all the way down
Follow a single request from the application through the deterministic shell, into device memory, across the weights and back. Every dimension in the scene is a number from the model below it: change an input and the picture changes shape.
The animated walk from the application through device memory and across the weights is still being built. Everything it will show is already on this page as tables: what has to fit in memory, how fast each phase runs, what the capacity costs, and what the deterministic shell adds beside it.
What has to fit, what it costs, and what it is not
These tables are the same arithmetic the scene draws, at the defaults. They are here so the page still works without JavaScript, and so the figures can be checked.
| Profile | Memory | Bandwidth | Dense fp16 | Per hour |
|---|---|---|---|---|
| Top-tier datacentre GPU | 80 GB | 3.3 TB/s | 750 TFLOPS | $3.50 |
| Previous-generation datacentre GPU | 80 GB | 2.0 TB/s | 310 TFLOPS | $1.80 |
| Mid-range inference GPU | 48 GB | 0.9 TB/s | 180 TFLOPS | $0.85 |
| Model | Parameters | Layers | KV heads | Head dim |
|---|---|---|---|---|
| Small open-weight (~8B) | 8B | 32 | 8 | 128 |
| Mid open-weight (~70B) | 70B | 80 | 8 | 128 |
| Large open-weight (~405B) | 405B | 126 | 16 | 128 |
What a per-token price already includes, and self-hosting does not
This is where open-weight business cases are usually decided, and it is the part a token calculator never shows.
| Cost | Quantity | Each, monthly | Why it exists |
|---|---|---|---|
| Platform and MLOps engineering | 1.5 FTE | $14,000 | Serving stack, autoscaling, upgrades, capacity planning. |
| On-call and incident cover | 0.5 FTE | $14,000 | An endpoint you own is an endpoint you carry at 3am. |
| Re-evaluation per model version | 2 runs/month | $3,200 | Every weight change, quantisation change and serving-stack upgrade re-opens the suite. |
| Non-production environments | 2 environments | $2,600 | You cannot test a serving change in production. |
| Patching and supply-chain review | 0.2 FTE | $14,000 | Serving runtimes, CUDA stacks and model provenance. |
| Licence and compliance review | 1 one-off amortised | $900 | Several open-weight licences are not open-source licences. Acceptable-use terms and redistribution limits need reading. |
The deterministic shell, priced beside the model it guards
Every control that makes an agent safe to run is ordinary code on ordinary CPU. The forward pass is the expensive part, and it is the part that is not deterministic.
| Step | Typical | Determinism |
|---|---|---|
| Admission and identity | 120 µs | Deterministic — same inputs, same result, replayable |
| Context Contract assembly | 2400 µs | Deterministic — same inputs, same result, replayable |
| Policy and authority check | 300 µs | Deterministic — same inputs, same result, replayable |
| Typed capability call | 850 µs | Deterministic — same inputs, same result, replayable |
| Evidence record write | 1600 µs | Deterministic — same inputs, same result, replayable |
| Forward pass through the weights | see the model | Probabilistic — sampled, and not reproducible without the seed and the same stack |
Every figure here is an estimate produced from the numbers you enter, with defaults last reviewed on 2026-09-26. They are not pricing, financial, legal or procurement advice, and no product or vendor is endorsed or recommended. Model prices, service names and platform capabilities change constantly — use your own contract pricing before deciding anything.
Everything runs in your browser: nothing you type is sent anywhere, stored, or seen by anyone, and reloading the page discards it. The full terms for these tools.
Most comparisons between hosted and self-hosted models argue about the price of a token. That is the wrong unit. When you host the weights yourself you do not buy tokens — you buy capacity, by the hour, and you pay for it whether or not anything is using it.
So this lab answers three questions in order, because the third one has no meaning until the first two are settled.
What has to fit? Weights are parameters times bytes per parameter, which quantisation changes. The KV cache is two caches across every layer, for every token of context, for every sequence in the batch — and it grows while the request is being served. If the two together exceed the device, you are buying another one.
How fast does it run? Prefill is compute-bound: reading a 9,000-token prompt is arithmetic, and it scales with tensor throughput. Decode is memory-bandwidth-bound: every single output token requires reading the entire weight matrix out of device memory again. That is why batching matters so much — the batch shares that one read — and why the interesting specification is bandwidth, not FLOPS.
What does it cost? Capacity divided by what you actually serve. At the defaults on this page that works out at around one-sixth utilisation, which means five-sixths of the bill buys nothing. The break-even figure the calculator reports is not a per-token comparison; it is the daily volume below which a hosted API is simply cheaper, and it is usually a multiple of what a pilot runs.
The part that carries the architecture
The journey above is split into two halves, and they cost very different amounts.
The deterministic shell — identity, the Context Contract, the policy and authority check, the typed capability call, the evidence write — is ordinary code on ordinary CPU. It runs in microseconds and costs a rounding error. It is reproducible: the same inputs give the same result, and a decision made three months ago can be replayed.
The forward pass is neither. It is the expensive part of the journey by several orders of magnitude, and it is the part that is sampled rather than computed. The scene draws that gap to scale because the gap is the argument: the controls that make an agent safe to run in an enterprise are close to free beside the model call they guard. Nobody is choosing between governance and cost.
That is also why the answer to “can we make this cost-effective?” is rarely “use a cheaper model”. It is: raise utilisation, batch harder, cut the prompt, cache what repeats, and move every decision you can out of the probabilistic half and into the deterministic one — where it is faster, cheaper, testable and provable.