Define the model workload
Local AI is not one workload. Record the model class and size, intended precision or quantisation, context length, concurrent users and whether the priority is interactive latency, throughput, experimentation or fine-tuning.
Use representative prompts and documents. A model loading successfully is not proof that it meets the application's quality, context or response-time requirement.
- Model and quantisation assumptions
- Context and batch size
- Concurrent sessions
- Inference, training or development priority
Plan the memory hierarchy
VRAM often sets the immediate boundary for accelerator-resident models, but system RAM still supports the operating system, data preparation, offload and concurrent tools. Storage must hold models, environments, datasets and working copies with sensible backup separation.
Estimate capacity with headroom and test uncertain assumptions before purchase. A specification should state what must fit, what may be offloaded and the performance consequence.
Include power, heat and operations
High-power accelerators affect the power supply, enclosure, acoustics, room heat and sustained performance. Remote access, permissions, software updates, model provenance and logging are also part of a usable workstation.
A staged route can start with feasibility on available or rented compute, then specify local hardware once model quality and resource behaviour are understood.