The real problem
Running a local model is great, but hardware never disappears from the equation. VRAM and latency end up mattering more than benchmark rankings.
The approach
I would measure tokens per second, time to first token, KV cache usage, and quality on real tasks.
Decisions that matter
Dense versus MoE, quantization, and context size are interdependent choices.