Local AI server hardware requirements (2026): GPUs, memory and speed
The right machine depends on the models you want to run and how many people will use it. These are the things that matter, in plain terms.
Last updated October 2026
Memory capacity decides which models fit
A model has to fit in memory to run well, and the memory it needs grows with its number of parameters, the learned values that make up the model. Models with more parameters generally give better answers but need more memory, and longer documents and more people at once need extra room on top. Running a model in a compressed form uses less memory, at some cost to quality.
Memory bandwidth decides how fast it writes
Each token a model writes means reading through a large part of the model in memory. So tokens per second depends heavily on how quickly the hardware can move data in and out of memory, its bandwidth, often matters more for speed than raw processing power.
GPUs vs unified-memory systems
Some systems run AI on dedicated GPUs (graphics cards), which have very fast memory of their own but a limited amount of it. Others share one large pool of memory between the processor and graphics, called unified memory, which can hold larger models but is often slower to read from. Neither is best for everyone: it's a trade-off between how big a model fits and how quickly it answers.
Workstation, server or compact desktop
- Compact desktop: quiet and small enough for an office, suited to one person or a few.
- Workstation: more capable, sits under a desk, can serve a team; louder under load.
- Server: the most capacity and room to grow, best in a server room or cupboard because of noise and heat.
Power, heat and noise
AI hardware draws the most power and makes the most noise while it's answering. Think about where it will sit, whether the room stays cool, and whether the circuit can handle it. A machine that's fine in a cupboard may be too loud next to a desk.
Start from the work, not the box
Test the models you'd use with your real documents before buying. What you already have may be enough to start. Our setups by size show how this usually comes together.