Qwen3.5 9B
DenseA compact member of the Qwen3.5 family. Useful as a general local model when multilingual and multimodal capabilities matter.
Scale9B parameters
Q4 weightsroughly 5–6 GB
Strengthgeneral + multilingual
Classsingle-GPU friendly
Model explorer
Choose by workload, not hype.
A practical snapshot of model families that make sense for local experimentation. Memory estimates below are rough weight-only rules of thumb for 4-bit quantization; real usage is higher once context and runtime overhead are included.
Gemma 4 12B
DenseGoogle's medium-sized multimodal model aimed directly at local workflows. Google describes it as suitable for dedicated-GPU laptops with 16 GB VRAM or unified memory.
Scale12B parameters
Q4 weightsroughly 7–8 GB
Inputtext + vision + audio
Targetlocal multimodal AI
DeepSeek R1 Distill Qwen 7B
ReasoningA distilled reasoning checkpoint built on Qwen2.5-Math. Small enough for many consumer setups while retaining strong math and reasoning behavior.
Scale7B parameters
Q4 weightsroughly 4–5 GB
AIME 202455.5 reported
LiveCodeBench37.6 reported
DeepSeek R1 Distill Qwen 14B
ReasoningA practical middle ground for local reasoning. It improves substantially over the 7B variant on the original DeepSeek evaluation while remaining deployable on enthusiast hardware.
Scale14B parameters
Q4 weightsroughly 8–10 GB
AIME 202469.7 reported
LiveCodeBench53.1 reported
Qwen3.6 27B
DenseA 27B multimodal model positioned around agentic coding. The official release reports strong SWE-bench and Terminal-Bench results at a size still realistic for high-memory local systems.
Scale27B parameters
Q4 weightsroughly 16–18 GB
SWE-bench Verified77.2 reported
Focuscoding + agents
DeepSeek R1 Distill Qwen 32B
ReasoningThe strongest Qwen-based R1 distill from the original release. It needs significantly more memory, but offers a major reasoning step up over smaller variants.
Scale32B parameters
Q4 weightsroughly 19–22 GB
GPQA Diamond62.1 reported
MATH-50094.3 reported
Mistral Small 4
MoEA hybrid instruct/reasoning/coding model with multimodal support. It has 119B total parameters but activates 6.5B per token; storage and memory still reflect the large total model size.
Total / active119B / 6.5B
Context256K
LicenseApache 2.0
Classworkstation / server
Memory figures are approximations, not vendor specifications. Quantization format, tokenizer, KV cache size, context length, GPU offload and runtime implementation can change actual memory use materially.