{"id":21294,"date":"2026-10-15T20:35:00","date_gmt":"2026-10-15T18:35:00","guid":{"rendered":"https:\/\/www.lukaswojcik.com\/blog\/?p=21294"},"modified":"2026-10-05T16:50:10","modified_gmt":"2026-10-05T14:50:10","slug":"nvidia-grafikkarten-lokale-llm-en","status":"publish","type":"post","link":"https:\/\/www.lukaswojcik.com\/blog\/en\/cloud-ai\/nvidia-grafikkarten-lokale-llm-en\/","title":{"rendered":"NVIDIA for Local LLMs: RTX 3060, 3090, 5060 Ti, 5090 and Professional GPUs Compared"},"content":{"rendered":"

For a local LLM, the first NVIDIA GPU question is whether the intended model fits fully into available memory. A faster GPU with insufficient VRAM can be a worse practical choice than an older card with greater capacity. Gaming FPS and AI TOPS do not answer that question.<\/p>\n

The comparison lists selected official memory variants. It evaluates specifications and operational requirements without original speed measurements. Card price, total system cost and running expenses remain separate quantities.<\/p>\n

\"Official
Capacity comparison, not a measured speed ranking.<\/figcaption><\/figure>\n

Official memory tiers<\/h2>\n\n\n\n\n\n\n\n\n\n\n\n
GPU<\/th>\nSelected memory<\/th>\nRole<\/th>\n<\/tr>\n<\/thead>\n
RTX 3060<\/td>\n12 GB<\/td>\nEntry option; distinct from 8 GB version<\/td>\n<\/tr>\n
RTX 5060 Ti<\/td>\n16 GB<\/td>\nNewer middle memory tier<\/td>\n<\/tr>\n
RTX 3090<\/td>\n24 GB<\/td>\nOlder high-capacity GeForce; used candidate<\/td>\n<\/tr>\n
RTX 4090<\/td>\n24 GB<\/td>\nDifferent compute architecture, same capacity tier<\/td>\n<\/tr>\n
RTX 5090<\/td>\n32 GB<\/td>\nHigher official GeForce capacity<\/td>\n<\/tr>\n
RTX 6000 Ada<\/td>\n48 GB<\/td>\nProfessional single GPU<\/td>\n<\/tr>\n
RTX PRO 6000 Blackwell<\/td>\n96 GB<\/td>\nProfessional high-capacity GPU<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Capacity before speed claims<\/h2>\n

Moving from 12 to 24 GB can enable another model class. Moving from RTX 3090 to RTX 4090 does not increase official memory. A model needing more than 24 GB with context and buffers gains no capacity from that change.<\/p>\n

A 32 GB card expands the budget without guaranteeing a particular context for every architecture. Even 48 or 96 GB must accommodate weights, state and headroom. A professional single GPU may therefore suit a large model better than several smaller cards.<\/p>\n

Architecture and runtime remain purchase criteria<\/h2>\n

CUDA support is only the first layer. Precision, quantisation kernels and model architectures impose further requirements. vLLM documents a minimum NVIDIA compute capability, but individual features can have different limits. Plenty of VRAM on an old GPU does not guarantee every modern configuration.<\/p>\n

A purchase decision therefore needs a specific runtime version, model revision and quantisation. An installed driver or successful game does not establish that inference path. Appropriate packages matter for new hardware as well as old GPUs.<\/p>\n

Compare used and professional cards properly<\/h2>\n

A used RTX 3090 can offer attractive capacity when condition, actual price and return arrangements fit. Cooling, full memory testing and stability under load belong in the assessment. A low listing price without functional evidence is not a complete purchase case.<\/p>\n

Professional cards also differ in form factor, memory features and intended operation. Extra expense needs a concrete reason: larger single-model capacity, suitable cooling or continuous-operation requirements. Identical execution does not produce better answers solely from a professional GPU label.<\/p>\n

Total system and multiple GPUs<\/h2>\n

Power supply, connectors, chassis clearance and heat removal need checking before a large GPU purchase. Measured system consumption at the wall is more useful for running costs than an isolated nominal figure. Idle hours matter on a permanently powered computer.<\/p>\n

Two 24 GB GPUs remain separate memory pools. A suitable runtime can distribute the model, but placement and interconnects affect operation. A single GPU with sufficient headroom is the simpler alternative. The memory estimator supports initial selection; verified compatibility and total cost settle the actual purchase.<\/p>\n

Memory planning for model selection: LLM VRAM estimator.<\/a><\/p>\n

\n

Related tools<\/h2>\n