{"id":21282,"date":"2026-10-07T11:40:00","date_gmt":"2026-10-07T09:40:00","guid":{"rendered":"https:\/\/www.lukaswojcik.com\/blog\/?p=21282"},"modified":"2026-10-05T16:49:55","modified_gmt":"2026-10-05T14:49:55","slug":"lokale-llm-software-vergleich-en","status":"publish","type":"post","link":"https:\/\/www.lukaswojcik.com\/blog\/en\/cloud-ai\/lokale-llm-software-vergleich-en\/","title":{"rendered":"Ollama, LM Studio, llama.cpp or vLLM: Which Software Fits Local LLMs?"},"content":{"rendered":"

Choosing software for a local LLM starts with the workload. Desktop chat, an API used by a script and a service handling concurrent requests need different properties. Popular products also occupy different layers: interface, model management and inference engine are separate concepts.<\/p>\n

This comparison evaluates documented capabilities rather than reporting original speed measurements. Runtime version, model architecture and hardware remain part of every specific decision.<\/p>\n

\"Three
A suitable interface does not establish a working GPU backend.<\/figcaption><\/figure>\n

Six paths with different priorities<\/h2>\n\n\n\n\n\n\n\n\n\n\n
Software<\/th>\nUseful strength<\/th>\nChecks before deployment<\/th>\n<\/tr>\n<\/thead>\n
Ollama<\/td>\nModel management and local API<\/td>\nBackend, model tag and context<\/td>\n<\/tr>\n
LM Studio<\/td>\nDesktop interface and model selection<\/td>\nRuntime, format and GPU offload<\/td>\n<\/tr>\n
llama.cpp<\/td>\nDirect controls, GGUF, many backends<\/td>\nBuild and architecture support<\/td>\n<\/tr>\n
MLX LM<\/td>\nApple Silicon execution<\/td>\nMLX artifact and memory headroom<\/td>\n<\/tr>\n
vLLM<\/td>\nAPI serving and concurrency<\/td>\nOS, GPU and quantisation kernel<\/td>\n<\/tr>\n
OpenVINO GenAI<\/td>\nSeparate optimised inference path<\/td>\nModel conversion and target device<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Separate desktop and API decisions<\/h2>\n

LM Studio suits a graphical introduction. Ollama provides a relatively compact service combining model management with an API. Adding a chat interface does not automatically change the runtime beneath it. A comfortable chat interface therefore does not establish server throughput.<\/p>\n

llama.cpp exposes model settings and hardware backends more directly. Extra control is useful for reproducible configurations or unusual hardware. Its name refers to a runtime rather than restricting inference to the Llama model family.<\/p>\n

The backend determines hardware compatibility<\/h2>\n

CUDA, Metal, Vulkan, SYCL and XPU represent distinct execution paths. NVIDIA instructions cannot transfer unchanged to Intel Arc. Ollama documents additional Vulkan support on Windows and Linux; the installed driver, backend and actual device detection must align.<\/p>\n

MLX LM and suitable Metal paths are natural candidates on Apple Silicon. An MLX quantisation and a GGUF file remain different artifacts. Switching backends requires checking model revision, tokenizer and chat template. Otherwise a different answer may reflect a changed configuration rather than a better engine.<\/p>\n

vLLM for the appropriate serving workload<\/h2>\n

vLLM focuses on serving. GPU installation documentation specifies Linux, with WSL among the Windows options. This differs from the native Windows installation experience of desktop applications. A supported GPU also does not guarantee every quantisation kernel.<\/p>\n

A single session prioritises time to first token and smooth generation. Concurrent sessions add aggregate throughput, queues and latency under load. A fast individual request does not settle a multi-user comparison. SGLang provides another serving candidate whose current model support requires separate verification.<\/p>\n

A defensible selection process<\/h2>\n

The operating system, existing hardware and intended model come first. Required features follow: structured output, tool calling, image input or offline operation. A common task set then runs with a recorded model revision, context and sampling configuration.<\/p>\n

A small desktop assistant does not require a complex serving stack merely as a precaution. An API service needs reproducible startup settings, useful error logs and explicit load limits. The existing memory estimator helps assess model fit; the final software choice follows a verified backend and the actual workload.<\/p>\n

Memory planning for model selection: LLM VRAM estimator.<\/a><\/p>\n

\n

Related tools<\/h2>\n