Run a Local LLM on Windows: Setup, Memory and GPU Checks

A local model is a useful option when you want to try a text task on your own Windows PC. Start by checking the runtime requirements, loading one small model and inspecting where it runs. Choose larger downloads or new hardware after that first check.

This guide gives you a setup and comparison method. It does not rank runtimes by measured speed or promise that a particular amount of memory will fit every model.

Choose a starting route for your task

• Ollama: choose this route if you want to follow a terminal example and later call a local API. Its Windows documentation describes the native app and the API at http://localhost:11434.
• LM Studio: consider it if you prefer downloading and using models through a desktop app. Check its system requirements before downloading.
• llama.cpp: consider this route when you want to work directly with model files and backend settings. The project documentation describes quantization and CPU/GPU backends. Expect to choose a build that matches your hardware.

You do not need to add another interface before completing a first Ollama test. If you later add Open WebUI or another client, check its configuration and data handling separately.

Check Windows, memory and the intended model

Ollama currently lists Windows 10 22H2 or newer. Its Windows page also lists driver requirements for NVIDIA and AMD acceleration; check those against your exact card and driver instead of assuming every GPU works.

For Windows, LM Studio lists x64 with AVX2 or ARM with Snapdragon X Elite. It recommends at least 16 GB RAM and 4 GB dedicated VRAM. Those are the publisher’s recommendations, not proof that a chosen model or context will fit.

Before the download, record your Windows version, CPU, RAM, GPU and dedicated VRAM. Use Task Manager’s Performance view to inspect the machine. Check free disk space as well: a stored model and a model loaded into memory are different things.

Choose an exact model variant and note its quantization. The llama.cpp project documents quantization as a way to reduce memory use; a model name alone does not identify the exact file. Leave room for Windows, other applications and the context you intend to use. Test that combination rather than buying against a universal RAM/VRAM tier.

First Ollama run: a small installation check

Install from the official Windows page. In a new PowerShell or Command Prompt window, run:

ollama run qwen2.5:0.5b

The registry page for this exact model documents that command. It is an example for checking installation, not our recommendation for the best model or for sensitive work. Allow the initial download to finish, then try a short, non-sensitive prompt such as: “Rewrite this sentence more clearly: The delivery date is not confirmed.” Review the answer yourself.

While that model is loaded, open a second terminal and run:

ollama ps

The Ollama FAQ explains the PROCESSOR column: 100% GPU means entirely in GPU memory, 100% CPU means system memory, and a split indicates both. This checks loading, not answer quality or a speed guarantee. If you expected GPU loading but see CPU, compare the exact GPU and driver with the official requirements before changing models or buying equipment.

Which Windows runtime is fastest for your task?

Answer that with a controlled comparison on your machine. Keep the model variant, quantization, context, prompt and reply length comparable, and record each runtime/build and backend. Record initial loading separately from a repeated request with the model already loaded. Check correctness alongside waiting time.

For a lower-level comparison, llama-bench separates prompt processing from text generation and supports repeated measurements. Its results describe the tested configuration. A short-prompt result on another PC does not establish which setup will handle your long document or simultaneous applications best.

As a manual acceptance test, choose a short passage whose meaning you can verify. Ask each setup to rewrite it, check that it preserves dates and negations, then repeat with your normal document length. Keep the result sheet even if a faster response contains an error. We have not run or published a performance benchmark for this guide.

What “local” does and does not establish

LM Studio documents offline use of downloaded models; searching, new downloads, runtime downloads and update checks need connectivity. Ollama distinguishes local execution from cloud-hosted models in its FAQ. Verify the selected model, client and any connected tools before entering sensitive information. Installing a local runtime alone does not establish that an entire workflow stays offline.

If your actual task is subtitles or speech translation, evaluate that product’s requirements separately. A successful text-model test does not validate a live-audio workflow. Continue with the existing Aurora Subtitles product page, the speech-translation architecture guide or our accessibility overview according to the task you want to solve.

Requirements and linked primary documentation checked on 25 October 2026. No hardware purchase, universal speed winner or translation accuracy is implied by the example.