How to install Ollama on a VPS (and call its API)

Install Ollama on a VPS, pull a small model, and call its HTTP API — with the honest note that these are CPU boxes, so stick to small quantized models and embeddings. Copy-paste commands, and how to expose it safely.

Ollama makes running a local model a one-line install. This is the how-to; for the honest picture of what CPU inference can and can't do (and the RAG sweet spot), see the VPS for Ollama use-case and self-host Ollama.

1. Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

That installs Ollama and starts it as a systemd service listening on localhost:11434.

2. Pull and run a small model

On a CPU box, start small:

ollama pull llama3.2:3b
ollama run llama3.2:3b "Summarize this in one sentence: ..."

3B models are genuinely usable on CPU; 7–8B run at a few tokens/sec (fine for batch, painful for chat); 13B+ will swap and crawl — don't.

3. Call the HTTP API

curl http://localhost:11434/api/generate \
  -d '{"model":"llama3.2:3b","prompt":"ping","stream":false}'

Embeddings are the CPU sweet spot:

ollama pull nomic-embed-text
curl http://localhost:11434/api/embeddings \
  -d '{"model":"nomic-embed-text","prompt":"hello world"}'

4. Keep it on localhost — expose safely if needed

Ollama binds to 127.0.0.1:11434 by default. Keep it there. If you need to reach it from another machine, put a reverse proxy with auth in front on a dedicated-IP plan (nginx + HTTPS guide) or use an SSH tunnel — never expose an unauthenticated model endpoint publicly.

The honest bit

These are CPU-only instances (no GPU). Small quantized models and embeddings run well; big models don't. Pair Ollama with a vector DB for a private RAG stack — small local embeddings plus a local vector store is the setup that genuinely shines here. Medium ($12/mo, 6 GB) is the comfortable plan. Root in about a minute, no KYC, pay in crypto.

FAQ

How do I install Ollama on a VPS?

One command: curl -fsSL https://ollama.com/install.sh | sh. Then ollama pull a small model and ollama run it, or call the HTTP API on localhost:11434. The steps are below. On a CPU VPS stick to small quantized models (1B–8B) and embeddings — big models need a GPU.

Will models be fast on a CPU VPS?

Small ones, yes; large ones, no. Expect a few tokens per second on a 7–8B model at 4-bit quantization, and genuinely snappy performance on 1–3B models and embedding models. It's great for background jobs, classification and RAG pipelines — not for a fast interactive chat on a big model.

Should I expose the Ollama API to the internet?

No. Keep it on localhost:11434. If you need remote access, put it behind a reverse proxy with authentication on a dedicated-IP plan, or reach it over an SSH tunnel. Never expose an unauthenticated model endpoint to the open internet.

What plan do I need?

Our Medium ($12/mo, 6 GB) suits small models and embeddings comfortably; Small ($8) works for testing 1–3B models. These are CPU-only instances — no GPU — so size by the model's RAM footprint, not by hoping for speed.

Do you ask for ID or a card?

No. Email to sign up, pay in USDC or USDT — no documents, no card.

Comments

No comments yet. Be the first.