Let's get the disappointing part out of the way, because the internet is full of pages that don't.
Our servers are CPU-only. We don't offer GPUs. That means local LLM work here has a hard ceiling, and pretending otherwise would just waste your money.
What actually works on CPU
On the standard plans, with up to 6 vCPU and 6 GB of RAM (our Medium plan — $12/mo), you're in the range of small quantized models:
- 1B–3B models (Q4): genuinely usable. Fast enough for classification, tagging, routing, and simple structured extraction.
- 7B–8B models (Q4): run, but expect a few tokens per second. Fine for a queue that chews through documents overnight. Painful as a chat window.
- Embedding models: excellent fit. They're small, fast on CPU, and this is probably the single best reason to run Ollama on a box like ours.
- 13B and up: don't. You'll be swapping and waiting.
If you need a fast, interactive 70B chat, you need a GPU host — that's a different product from a different provider, and we'd rather tell you than take your $12.
Where a CPU box genuinely wins
Privacy. Your documents, your prompts, your embeddings — never leaving a machine you control, never becoming someone's training data. For a lot of people that's the entire point, and a few tokens per second is an acceptable trade.
Cost predictability. No per-token bill. A pipeline that classifies fifty thousand records costs the same as one that classifies fifty: $12.
Async work. Most useful LLM work isn't a chat window. It's a queue: summarize these, tag those, embed this corpus. A CPU box grinding through a backlog overnight is perfectly good at that.
Setup
# Ubuntu 24.04 — Medium plan recommended
curl -fsSL https://ollama.com/install.sh | sh
# Start with something small and see for yourself
ollama pull llama3.2:3b
ollama run llama3.2:3b "Summarize this in one sentence: ..."
# Embeddings — the sweet spot for CPU
ollama pull nomic-embed-text
curl http://localhost:11434/api/embeddings \
-d '{"model":"nomic-embed-text","prompt":"hello world"}'
Ollama serves an HTTP API on localhost:11434. Keep it there. If you need to reach it from elsewhere, put it behind a reverse proxy with authentication on a dedicated-IP plan — never expose an unauthenticated model endpoint to the open internet.
Pair it with a vector database (Qdrant or pgvector in Docker) and you have a complete private RAG stack on one $12 box. That combination — small local embeddings, local vector store, external API only for the heavy generation step — is the setup that actually makes sense on hardware like this.
Picking a plan
| Use | Plan | Price |
|---|---|---|
| Embeddings, 1–3B models, RAG pipeline | Medium | $12/mo |
| Same, plus a public endpoint behind auth | Medium-IP | $20/mo |
| Just testing small models | Small | $8/mo |
CPU-only: standard plans go up to 6 vCPU and 6 GB RAM, the Pro line up to 12 vCPU and 80 GB. NVMe storage, unmetered traffic, Germany or Finland. Crypto payment, no KYC. Yearly billing is one transfer instead of twelve.
Related reading
- Host a vector DB for AI memory — the other half of a private RAG stack
- VPS for AI agents — orchestration on the same box
Ready? Deploy a server → — live in about a minute, paid in crypto, no ID required.
Comments
No comments yet. Be the first.