How is this different from your Ollama use-case?

The Ollama page is about small models on a $12 CPU box — 1B–8B, embeddings, light work. This is the high-memory end: 13B to 30B-class quantized models that need 24–48 GB or more to even load. Same CPU-only reality, much bigger memory footprint, different plan.

How much RAM for a given model?

The model has to fit in memory plus overhead. A 13B 4-bit model wants ~10–16 GB; 30B-class quantized pushes 24–48 GB; go bigger or higher-precision and you're into 64–80 GB — past that, no single box of ours fits. Add room for context on top.

No — be honest with yourself here. CPU inference on a large model is a few tokens per second, not an interactive stream. It's fine for batch and background work (summarize overnight, classify a queue). For real-time chat on a big model you need a GPU, which we don't offer.

Why run it here instead of an API?

Privacy. Your prompts and outputs never touch a provider's servers or logs. For sensitive data — legal, medical, internal code — a slower private model beats a fast one that sees everything. Pair it with no-KYC crypto payment and the whole chain stays yours.

VPS for private high-memory LLM inference

Bigger quantized models need real RAM, not a GPU you don't have. Where a high-memory CPU box runs 30B-class models privately, what's realistic, and where it isn't. From $70/mo.

We have a separate page for Ollama and small models — that's the $12 CPU box running 1B–8B and embeddings. This page is the other end: the models big enough that RAM, not the CPU, is what stops you.

Let's be honest up front, same as everywhere: our servers are CPU-only, no GPU. A large model here runs slowly. If you want fast interactive chat on a 30B model, you need a GPU host — different product, different provider. What a high-memory CPU box does well is run a big quantized model privately for work that isn't a chat window.

The RAM math

The model sits in memory, quantized or not, plus overhead for context and the runtime:

13B, 4-bit — roughly 10–16 GB. Runs on a mid plan already.
30B-class, quantized — 24–48 GB depending on quantization. This is Pro-32 to Pro-64 territory.
Bigger or higher precision — 64–80 GB, and past 80 GB no single box of ours fits. Pro-80 is the ceiling here.

That's why "run a bigger local model" is really a high-memory question. The model is the memory footprint. Add a RAG index on the same host and the numbers stack.

Where private-first genuinely wins

The case isn't speed and it isn't cost — it's that some data can't leave. Legal documents. Medical records. Proprietary code. Anything where sending the prompt to a third-party API is off the table. A slower model that runs entirely on your machine beats a fast one that logs everything you send it. We wrote the full picture here.

And if the data is that sensitive, the payment probably should be private too. Renting the box with crypto and no KYC keeps the whole chain — server, model, prompts, billing — off anyone's identity records. That's the pitch: not cheaper inference, but inference nobody else can see.

What to pick

For a 30B-class model with room for context, Pro-64 (64 GB) is the comfortable choice; a tighter quantization fits Pro-32 or Pro-48, a larger one goes to Pro-80. Load the model first, watch resident memory, size from what you measured. And set expectations: this is private batch inference, not a fast chat window.

VPS for private high-memory LLM inference

The RAM math

Where private-first genuinely wins

What to pick

FAQ

Comments

Leave a comment