Governed inference for the enterprise
Deploy and fine-tune open models on dedicated GPUs in the UK, behind a KV cache that is encrypted, isolated per tenant and provably erasable. Longer context on the GPUs you pay for, and the evidence your auditors ask for.
client
POST /v1/chat/completions
host <your-endpoint>
auth Bearer •••••••• scope=acme
model qwen3.5-9b
{"role": "user",
"content": "Summarise the Q3 claims file."}
<- 200 streaming 46ms to first tokengovernance log
Illustrative session. Timestamps and identifiers are examples.
The platform
The KV cache holds 60 to 80% of inference memory on long-context work, and in most stacks it is shared, unencrypted and forgotten. kvrun runs your models on it as a governed asset, so the same deployment is faster, denser and defensible.
01
Pick an open model and get a private, OpenAI-compatible endpoint on dedicated compute. The GPU is sized from the weights, so you never research an instance type.
02
Hand it a dataset, get a tuned model back. One LoRA or QLoRA job that starts, finishes and shuts itself down, then serves in one click.
03
The memory that holds your context, treated as data: encrypted, isolated per tenant, tiered beyond VRAM and logged on every access.
Performance
Encryption, policy checks and audit writes add under 3% to time to first token and under 2% to throughput, across vLLM, SGLang, TensorRT-LLM and llama.cpp. Tiering the cache from GPU memory to host, disk and object storage is what buys the headroom back.
Early internal benchmarks against a stock serving baseline. Your workload will differ.
Context per GPU
Concurrent tenants
GPU utilisation
Security and privacy
Memory overwrite is not erasure, and a shared prefix cache is not isolation. kvrun treats every block of context as data it is accountable for.
Isolated
Separate key hierarchies per tenant. Prefix caching never hands one customer's block to another.
Erasable
Destroy a session's key and every block it sealed is unreadable, everywhere it was stored, with a certificate to prove it.
Audited
Allocation, access, movement and erasure land in an append-only log you can export.
| Workspace | GPU | Host | Disk | Object |
|---|---|---|---|---|
| acmekey a1 | 8 blocks | 6 blocks | 4 blocks | 3 blocks |
| globexkey g7 | 5 blocks | 7 blocks | 3 blocks | 5 blocks |
| acme, erasedkey destroyed | 8 blocks | 6 blocks | 4 blocks | 3 blocks |
Open models · one console
A curated catalog, every entry sized against the hardware we run so nothing loads and falls over. Need something else? Bring your own weights from Hugging Face.
| Model | Type | Params | Context |
|---|---|---|---|
| Qwenqwen3.6-35b-a3b | Text | 36B · 3B active | 256K |
| Qwenqwen3.6-27b | Text | 27.8B | 256K |
| Qwenqwen3.5-9b | Text | 9.7B | 256K |
| Qwenqwen3.5-4b | Text | 4.7B | 128K |
| DeepSeekdeepseek-r1-distill-32b | Text | 32.8B | 128K |
| Googlegemma-3-27b | Text | 27.4B | 128K |
| Mistralmistral-small-3.1-24b | Text | 24B | 128K |
| Qwenqwen3-coder-30b-a3b | Code | 30.5B · 3B active | 256K |
| Mistraldevstral-small-24b | Code | 24B | 128K |
| Qwenqwen2.5-coder-32b-awq | Code | 32.8B | 128K |
| Qwenqwen3.8-27b | Vision | 27.8B | 256K |
| Qwenqwen2.5-vl-32b | Vision | 32B | 128K |
| Googlegemma-3-12b-vision | Vision | 12B | 128K |
The accelerator is chosen from the weights at deploy time, and the exact monthly cost appears before you confirm.
Every deployment is an OpenAI-compatible endpoint. Change the base URL and the key, and keep the code you already have.
curl https://<your-endpoint>/v1/chat/completions \
-H "Authorization: Bearer $KVRUN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.5-9b",
"messages": [{ "role": "user", "content": "Hello, kvrun!" }],
"stream": true
}'Compliance
The evidence is produced as the system runs, so there is nothing to reconstruct later. Erasure certificates, isolation defaults and access logs map onto the controls your auditors test.
Fine-tunes carry their own record. Every run holds out a split, scores the model against it, and writes a verdict mapped to the EU AI Act articles it answers to, kept beside the weights.
| Framework | Requirement | Evidence |
|---|---|---|
| GDPR Art. 17 | Right to erasure | Erasure certificate per session |
| GDPR Art. 25 | Privacy by design | Encryption and isolation on by default |
| SOC 2 CC6.1 | Access control | Per-tenant key isolation |
| SOC 2 CC7.2 | Monitoring | Continuous audit log |
| ISO 27001 | Media disposal | Cryptographic erasure |
| EU AI Act | Evaluation and oversight | Eval record on every fine-tune |
Pricing
Accelerator hours and storage, billed only while a deployment is up. Tear it down whenever you like.
A training run is billed for the time it takes, then the hardware goes away.
The monthly estimate appears before you confirm, and it is the price on the invoice.
Governed, private, open models