Which local AI model gets your camera commands right?
| # | Model | out of 100 | did nothing when it should | until it acts | 1 in 10 is slower | Licence | ||
|---|---|---|---|---|---|---|---|---|
| 1 | Ornith 1.5 35B-A3B (Q4_K_M)Ornith | 84 | 97% | 8/8 | 1.3 s | 2.1 s | 21.7 GB | Unstated |
| 2 | Qwen3.6 35B-A3B (UD-Q3_K_XL)Qwen | 82 | 97% | 8/8 | 1.3 s | 2.0 s | 16.9 GB | Apache-2.0 |
| 3 | Nemotron 3.5 Bolt E4B (UD-Q4_K_XL)NVIDIA | 82 | 95% | 8/8 | 1.2 s | 1.7 s | 5.1 GB | NVIDIA Open Model |
| 4 | Qwen3.5 35B-A3B (UD-Q3_K_XL)Qwen | 80 | 97% | 8/8 | 1.5 s | 2.7 s | 16.6 GB | Apache-2.0 |
| 5 | gpt-oss 20B (MXFP4)OpenAI | 80 | 100% | 8/8 | 1.9 s | 5.2 s | 12.1 GB | Apache-2.0 |
| 6 | Spark-X2.6 4B (Q4_K_M)XHToken | 77 | 92% | 8/8 | 1.3 s | 1.7 s | 3.5 GB | Apache-2.0 |
| 7 | Ornith 1.5 9B (Q4_K_M)Ornith | 75 | 97% | 8/8 | 2.4 s | 3.7 s | 5.8 GB | Unstated |
| 8 | Qwen3 4B Instruct 2507 (Q4_K_M)Qwen | 75 | 94% | 8/8 | 2.0 s | 2.6 s | 2.5 GB | Apache-2.0 |
| 9 | Gemma 4 26B-A4B (Q4_0)Google | 74 | 91% | 7/8 | 1.3 s | 1.8 s | 14.6 GB | Apache-2.0 |
| 10 | GLM-4.7-Flash 30B-A3B (Q4_K_M)Z.ai | 72 | 97% | 8/8 | 3.6 s | 5.8 s | 18.3 GB | MIT |
| 11 | Nemotron 3.5 Lightning 30B-A3B (UD-IQ2_M)NVIDIA | 71 | 91% | 7/8 | 1.6 s | 2.4 s | 19.4 GB | OpenMDW |
| 12 | Gemma 4 E2B (Q4_K_M)Google | 70 | 83% | 8/8 | 0.7 s | 1.0 s | 3.1 GB | Apache-2.0 |
| 13 | Spark-X2.6 1.7B (Q4_K_M)XHToken | 69 | 80% | 8/8 | 0.5 s | 0.7 s | 1.8 GB | Apache-2.0 |
| 14 | Underdog Poodle 4B (Q4_K_M)Conway Research | 69 | 88% | 8/8 | 1.5 s | 2.1 s | 2.8 GB | CC-BY-NC-4.0 |
| 15 | Qwen3.5 4B (Q4_K_M)Qwen | 67 | 86% | 8/8 | 1.4 s | 2.5 s | 2.7 GB | Apache-2.0 |
| 16 | MiniCPM5 2B (Q8_0)OpenBMB | 66 | 83% | 8/8 | 1.1 s | 1.7 s | 2.7 GB | Apache-2.0 |
| 17 | MiniCPM5 2B (Q4_K_M)OpenBMB | 66 | 83% | 8/8 | 0.9 s | 1.5 s | 1.6 GB | Apache-2.0 |
| 18 | G9v3 3B (Q4_K_M)ai9stars | 66 | 86% | 8/8 | 1.7 s | 2.6 s | 1.9 GB | Apache-2.0 |
| 19 | Qwen3.5 9B (Q4_K_M)Qwen | 65 | 88% | 8/8 | 2.3 s | 3.0 s | 5.7 GB | Apache-2.0 |
| 20 | Jan v3.5 4B (Q4_K_M)Jan | 63 | 86% | 8/8 | 2.2 s | 3.5 s | 2.7 GB | Apache-2.0 |
| 21 | K2 Horizon 3.7B (Q4_K_M)MBZUAI IFM | 57 | 83% | 8/8 | 2.1 s | 2.7 s | 3.2 GB | Apache-2.0 |
| 22 | Ministral 3 3B (Q4_K_M)Mistral | 56 | 79% | 7/8 | 1.6 s | 2.6 s | 2.1 GB | Apache-2.0 |
| 23 | Gemma 4 E2B (Q8_0)Google | 54 | 64% | 8/8 | 0.7 s | 0.8 s | 5.0 GB | Apache-2.0 |
| 24 | Qwen3.5 2B (Q4_K_M)Qwen | 54 | 74% | 6/8 | 0.9 s | 1.7 s | 1.3 GB | Apache-2.0 |
| 25 | LFM3 1.7B (Q4_K_M)Liquid AI | 53 | 74% | 7/8 | 0.9 s | 1.5 s | 1.1 GB | LFM Open |
| 26 | LFM2 24B-A2B (Q4_K_M)Liquid AI | 51 | 73% | 7/8 | 1.0 s | 1.7 s | 14.4 GB | LFM Open |
| 27 | Gemma 4 E4B (Q4_0)Google | 51 | 64% | 8/8 | 1.0 s | 1.3 s | 4.6 GB | Apache-2.0 |
| 28 | Nemotron 3 Nano 4B (Q4_K_M)NVIDIA | 46 | 61% | 8/8 | 1.0 s | 1.1 s | 2.8 GB | NVIDIA Open Model |
| 29 | Qwen3.5 0.8B (Q4_K_M)Qwen | 43 | 62% | 4/8 | 0.5 s | 0.9 s | 0.5 GB | Apache-2.0 |
| 30 | Qwen3.5 0.8B (Q8_0)Qwen | 37 | 55% | 2/8 | 0.6 s | 0.9 s | 0.8 GB | Apache-2.0 |
| 31 | xLAM-2 3B fc-r (Q4_K_M)Salesforce | 37 | 64% | 8/8 | 2.4 s | 6.0 s | 1.9 GB | CC-BY-NC-4.0 |
| 32 | xLAM-2 8B fc-r (Q4_K_M)Salesforce | 36 | 61% | 7/8 | 2.2 s | 3.0 s | 4.9 GB | CC-BY-NC-4.0 |
| 33 | Phi-4-mini 3.8B (Q4_K_M)Microsoft | 32 | 64% | 8/8 | 5.1 s | 6.6 s | 2.5 GB | MIT |
| 34 | LFM2.5 8B-A1B (Q4_K_M)Liquid AI | 28 | 67% | 7/8 | 3.6 s | 10.0 s | 5.2 GB | LFM Open |
| 35 | MiniCPM5 1B (Q4_K_M)OpenBMB | 14 | 52% | 4/8 | 4.1 s | 6.4 s | 0.7 GB | Apache-2.0 |
| 36 | SmolLM3 3B (Q4_K_M)Hugging Face | 12 | 52% | 8/8 | 4.2 s | 10.0 s | 1.9 GB | Apache-2.0 |
| 37 | FunctionGemma 270M (Q4_K_M)Google | 5 | 26% | 4/8 | 10.0 s | 10.0 s | 0.3 GB | Gemma |
English only for now: one script of 33 turns, small models run three times with the median published, the others once. Treat scores within a few points of each other as ties. Every model hears the same script, a creator filming a video with requests to their camera app in between, and is checked turn by turn. The score, out of 100, weighs doing the right thing most, then speed. Pick a model to see every turn.