Skip to content

Ollama (LLM) on Linux / Debian

The exercise assistant (POST /api/practices/[id]/drill-advisor) calls Ollama from the Next.js server only (Route Handlers). The browser never talks to Ollama directly.

Install Ollama on the server

Follow the official instructions (Linux installer / packages):

Typical one-line install (as root or with sudo), then start the daemon if the installer does not enable it automatically:

curl -fsSL https://ollama.com/install.sh | sh

After install, pull at least one model (names must match OLLAMA_MODEL exactly, see ollama list):

ollama pull llama3.2

Confirm the API responds locally:

curl -s http://127.0.0.1:11434/api/tags | head

Where to configure the application

Variables are read only on the Node.js process (no NEXT_PUBLIC_ prefix).

Variable Meaning Default in code if unset
OLLAMA_BASE_URL Base URL of the Ollama HTTP API http://127.0.0.1:11434
OLLAMA_MODEL Model tag (ollama list) llama3.2

Local development (repo root)

  1. Copy .env.example to .env or .env.local (see Next.js env files).
  2. Set OLLAMA_BASE_URL and OLLAMA_MODEL as needed.
  3. Optional: run npm run ollama:ping from the project root (loads .env / .env.local via the script) to verify connectivity and that the model exists.

Production (Debian + PM2 / systemd)

Use the same variable names in the environment of the process that runs next start:

  • PM2: env / env_production in ecosystem.config.cjs, or an ecosystem file that sources a shell snippet.
  • systemd: Environment= lines or EnvironmentFile=/etc/coaching/app.env on the unit that starts Node/PM2.

Example values when Ollama runs on the same host as the app:

OLLAMA_BASE_URL=http://127.0.0.1:11434
OLLAMA_MODEL=llama3.2

If Ollama runs on another internal host (VPN / private subnet), set OLLAMA_BASE_URL to that base URL (still not public internet).

In-app check (after login)

With drills read permission, GET /api/ollama/status returns whether the configured base URL is reachable and lists models. Use it to confirm production wiring without reading server logs.

Security

  • Do not expose TCP 11434 to the public internet. Prefer loopback (127.0.0.1) on the app host or a private network only.
  • All LLM traffic stays server-side; see comments in src/lib/ollama.ts in the repository.

See also

  • .env.example — Ollama block at the top of the env template.
  • scripts/ollama-ping.ts — CLI smoke test.