Skip to content
Model Wayfinder
Guide

How to run an AI model on your own computer

Running a model locally means your data never leaves your computer, it works offline and there are no monthly fees. You only need a computer that’s powerful enough and about 15 minutes.

01 · Check your hardware

What matters most is your graphics card’s memory (VRAM). On Windows: Task Manager → Performance → GPU → “Dedicated GPU memory”. On a Mac with an M-series chip, about three quarters of your main memory can be used.

Rule of thumb for the usual compressed version (Q4): 0.6 GB per billion parameters plus 1.5 GB. A 7B model needs about 6 GB, so it fits an 8 GB card. Our finder checks this for you when you choose “local”.

02 · The easy way: LM Studio

LM Studio is a free desktop app for Windows, macOS and Linux with a normal chat window – no command line needed.

  1. Download and install LM Studio from lmstudio.ai.
  2. Open the search (magnifying glass) and type the model name. The models come from Hugging Face.
  3. Pick a version marked Q4_K_M – a good balance of size and quality. LM Studio shows whether it fits your memory.
  4. Click Download, then load the model and start chatting.

03 · For tinkerers: Ollama

Ollama runs models from the command line and offers a local API that other programs can use.

  1. Install Ollama from ollama.com.
  2. Open a terminal and start a model from the Ollama library, for example: ollama run <model-name>. Our model pages show the exact command where we’ve checked it.
  3. Models from Hugging Face in GGUF format can be started directly: ollama run hf.co/<user>/<repository>:Q4_K_M

04 · Finding models on Hugging Face

Hugging Face is the largest library of open models. Tips:

  • Filter by GGUF – that’s the format LM Studio and Ollama use.
  • Read the model card: it states the licence. Some models may only be used privately or for research.
  • Prefer files from the original maker or well-known uploaders.

05 · What to expect

Local models are usually somewhat weaker than the largest cloud models, but often good enough for writing, summarising and translating. The first answer takes a few seconds while the model loads. Without a graphics card it works too – just much slower.