Glossary
Plain-English glossary
The handful of words you'll keep running into, explained without the jargon.
- Model
- The AI itself, stored as a large file you download. Different models are good at different things: chatting, finding similar text, or understanding speech.
- Local AI
- AI that runs on your own computer, not on a company's servers. Once it's downloaded, it works offline and your data stays with you.
- Memory (RAM)
- Your computer's short-term workspace. A model has to fit in memory while it runs, so this is the number that matters most. 8 GB and 16 GB are common in laptops.
- Graphics card (GPU)
- A chip built for doing many small calculations at once. Originally for games, it also runs AI much faster than the main processor. NVIDIA cards are the best supported.
- VRAM
- The graphics card's own memory. If a model fits in VRAM, it runs fast; if not, part of it spills into regular memory and slows down.
- Unified memory
- On Apple Silicon Macs, the processor and graphics share one pool of memory, so a Mac with 36 GB can give most of that to an AI model.
- Quantized
- A slimmed-down version of a model, like a compressed photo. It takes far less memory with only a small loss in quality. Names like Q4 or Q5 tell you how much it was compressed.
- Parameters (7B, 14B)
- Roughly, the number of adjustable settings inside a model, in billions. More usually means smarter, but also bigger and slower.
- Tokens per second
- How fast a chat model writes. A token is about three-quarters of a word. Around 10 per second reads comfortably.
- Embedding model
- A small model that turns text into a list of numbers describing its meaning, so a computer can find passages about the same idea even when the words differ.
- Ollama
- A free app that downloads and runs models with one command. The easiest starting point on Windows, Mac, and Linux.
- llama.cpp
- The open-source engine underneath many local AI apps. More knobs than Ollama, popular with people who want control.
- AI coding assistant (agent)
- An AI tool such as Claude Code, Codex, or OpenCode that can run commands on your computer, with your permission, to set things up for you.