Self-Hosted Large Language Model (LLM) with Ollama Behind an Auth Proxy

Inference is the process of generating an answer from a model. A local language model can keep this processing on controlled hardware, but local hosting alone does not guarantee isolation, privacy compliance or cloud-level performance. Ollama’s local API has no built-in authentication. A reverse proxy receives requests on behalf of Ollama, checks credentials and handles the encrypted TLS connection; network boundaries, outbound connections, storage and application behavior remain separate concerns.

1. The actual boundary

The example below exposes only a minimal generation endpoint through nginx. Loopback refers to the host’s own local network address, 127.0.0.1 in this example. The proxy listens on the Docker host’s IPv4 loopback address, not on every network interface. Remote use requires a separately designed VPN or private network entry point. The Ollama container has no published port. The Docker host and containers on the same backend network can still reach Ollama directly, so unrelated applications do not belong on that network.

A normal Docker bridge permits outbound traffic. Model downloads therefore work, but this is not an air gap. OLLAMA_NO_CLOUD=1 disables Ollama cloud features in supporting versions; a local model must still be selected. This setting does not block external tools, web UIs or host traffic. Strict offline operation needs controlled model provisioning and independently enforced outbound restrictions.

The network example assumes a current Docker Engine in standard NAT bridge mode. Docker documents loopback exposure to neighboring hosts in versions older than 28.0.0; custom direct-routing settings also require separate review.

2. Containers and certificates

The following compose.yaml is a starting configuration for CPU inference, not a complete enterprise deployment. OLLAMA_IMAGE and NGINX_IMAGE must contain reviewed image references, preferably immutable digests. Available memory, context length, model license and architecture need checking before model selection.

services:
  ollama:
    image: ${OLLAMA_IMAGE:?Set a reviewed Ollama image tag or digest}
    restart: unless-stopped
    environment:
      OLLAMA_HOST: 0.0.0.0:11434
      OLLAMA_NO_CLOUD: "1"
    volumes:
      - ollama_data:/root/.ollama
    networks:
      - llm_backend
  auth_proxy:
    image: ${NGINX_IMAGE:?Set a reviewed nginx image tag or digest}
    restart: unless-stopped
    ports:
      - "127.0.0.1:8443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
      - ./htpasswd:/etc/nginx/.htpasswd:ro
    depends_on:
      - ollama
    networks:
      - llm_backend
networks:
  llm_backend:
    driver: bridge
volumes:
  ollama_data:

Before startup, nginx.conf, the password file and certificates must exist. The certificate must include llm.example.internal as a subject alternative name; the client needs the issuing CA certificate. htpasswd -cB ./htpasswd api_user interactively creates a new bcrypt password file. The -c option overwrites an existing file, so subsequent account changes omit that option. File permissions must permit nginx to read the password file while restricting unrelated access; private keys require corresponding protection.

NVIDIA acceleration additionally requires a compatible GPU, drivers, NVIDIA Container Toolkit and a GPU reservation in the Ollama service. CPU execution does not become GPU-accelerated merely through quantization. depends_on orders container startup but does not prove model readiness. A recreated backend may also require a proxy restart because this simple upstream configuration resolves the service address at nginx startup.

3. A proxy restricted to the required operation

This nginx.conf permits authenticated POST requests only at /api/generate. Other paths return 404; other methods on the permitted path are denied. Model administration stays outside the shared API. The proxy removes the Basic credentials before forwarding the request.

events { worker_connections 1024; }
http {
    upstream ollama_backend { server ollama:11434; }
    server {
        listen 443 ssl;
        server_name llm.example.internal;
        ssl_certificate /etc/nginx/certs/server.crt;
        ssl_certificate_key /etc/nginx/certs/server.key;
        ssl_protocols TLSv1.2 TLSv1.3;
        client_max_body_size 1m;
        location = /api/generate {
            limit_except POST { deny all; }
            auth_basic "Local inference";
            auth_basic_user_file /etc/nginx/.htpasswd;
            proxy_pass http://ollama_backend;
            proxy_http_version 1.1;
            proxy_set_header Host ollama:11434;
            proxy_set_header Authorization "";
            proxy_set_header Connection "";
            proxy_buffering off;
            proxy_cache off;
            proxy_connect_timeout 5s;
            proxy_read_timeout 600s;
            proxy_send_timeout 60s;
        }
        location / { return 404; }
    }
}

Basic Auth authenticates a shared service account, not individual document permissions. Strong credentials, account revocation, request/concurrency limits and log retention still need an operational design. The 1 MiB request limit is an example and may be too small for images. A compromised authorized application can still submit expensive requests or choose available models.

4. Connection verification with TLS checks

Model provisioning can use docker compose exec ollama ollama pull llama3.1:8b after the services have started. This is an example local model identifier, not a hardware recommendation. Pulling downloads a model; ollama run additionally starts an interactive session. The following curl request runs on the Docker host, prompts for the password and verifies the certificate against the supplied CA:

curl --cacert ./certs/ca.crt   --resolve llm.example.internal:8443:127.0.0.1   --user api_user   --header 'Content-Type: application/json'   --data '{"model":"llama3.1:8b","prompt":"Return the word test.","stream":false}'   https://llm.example.internal:8443/api/generate

The test prompt contains no company information. A prompt asking for a policy summary does not make an absent policy available to the model. Verification should cover missing and incorrect credentials, a valid request, denied management paths, streaming, unavailable backend and actual port exposure. Syntax checks such as docker compose config and nginx -t are prerequisites, not evidence of an operational security boundary.