Toolbench

Which local AI models nail tool calls, and how fast?

← All models

Thinking Machines Lab · mixture of experts · #17 of 38

Inkling 2 Mini 26B-A3B (UD-Q4_K_M)

Apache-2.0100% of turns right
73out of 100
Turns right
100%
Serious mistakes
0
Typical wait
4.3 s
slow: 5.8 s
Invalid tool calls
0
of 24
Download
16.6 GB
Memory used
25.5 GB

Its shoot

Turn by turn

Watch it play out →
Setting up
Take 1
Between takes
Take 2
Wrapping up

Bars show how long it took to act, up to the 10 s limit. Pick a turn.

RECMOVIE · M

Turn 14 · Request

“Start recording.”

Should happen: Recording

RightActed in 3.6 s

  • set_recordingrecording true

“Recording has started.”

The score

How the score adds up

Right outcome
100% of turns right · counts 45%
100
Serious mistakes
serious mistakes on 0% of turns · counts 15%
100
Clean tool calls
0% of calls invalid · counts 5%
100
Typical wait
4.3 s typical wait · counts 25%
24
Slow wait
5.8 s for the slowest 1 in 10 · counts 10%
24

The tick on each bar is the middle of the field.

Requests done right
20/20
Questions answered without acting
4/4
Said what it can't do
1/1
Left the video alone
8/8

The record

Every turn

  1. 1
    What shutter speed should I use at 25 frames a second?Question · Expected: Any answer; nothing changes
    Right“At 25 fps, the 180° rule gives a shutter of 1/(2 × 25) = 1/50s. Your camera is already set to 1/50, so there's nothing to change.”
    4.7 s
  2. 2
    Set the aperture to f/5.6.Request · Expected: Aperture f/5.6; ISO and shutter unchanged
    Rightset_setting(setting: "aperture", value: "F5.6")“Aperture set to f/5.6 — one stop down from f/4.0, so the frame will be a bit darker; you may want to check exposure.”
    4.7 s
  3. 3
    Is my exposure okay?Question · Expected: Any answer; nothing changes
    Rightanalyze_frame()“Not quite — the frame reads too dark (mean brightness ~33%, highlights only reaching ~53%, with ~2 stops of headroom). No clipping in highlights or crushed shadows, so it's safely dark rather than blown out. If you want, I can lift exposure by about a third of a stop.”
    7.6 s
  4. 4
    Make it a bit brighter.Request · Expected: A bit brighter; shutter stays
    Rightadjust_exposure(stops: 0.5, using: "auto")“Brightened by about half a stop — ISO raised from 800 to 1250, shutter kept at 1/50.”
    5.2 s
  5. 5
    A bit more.Request · Expected: Brighter again by a similar step
    Rightadjust_exposure(stops: 0.5, using: "auto")“Added another half stop — ISO now 1600, shutter still 1/50.”
    3.6 s
  6. 6
    Show the waveform.Request · Expected: Waveform scope
    Rightset_studio_view(scope: "waveform")“Waveform is now the scope shown in Studio.”
    3.5 s
  7. 7
    Turn on zebras.Request · Expected: Zebras on
    Rightset_studio_view(zebras: true)“Zebras are on — stripes will appear on any area at or above 95% brightness.”
    3.4 s
  8. 8
    Make it a bit warmer.Request · Expected: A bit warmer
    Rightadjust_color_temperature(kelvin: 400)“Color temperature raised from 5600 K to 6000 K for a slightly warmer look.”
    3.4 s
  9. 9
    Undo that.Request · Expected: The warming undone
    Rightundo_last_change()“Color temperature restored to 5600 K.”
    3.3 s
  10. 10
    Zoom out a little.Request · Expected: A little wider
    Rightzoom(direction: "wide", milliseconds: 500)“Zoomed out from 24 mm to 21 mm.”
    3.5 s
  11. 11
    Turn on focus peaking.Request · Expected: Focus peaking on
    Rightset_studio_view(focusPeaking: true)“Focus peaking is on at the default sensitivity.”
    3.6 s
  12. 12
    Focus on my face.Request · Expected: Focuses on the creator
    Rightget_frame()autofocus()“Autofocus has engaged on your face (subject recognition is set to Human) and reported it focused. If it's not on you, tell me where your face sits in the frame and I can focus there directly.”
    8.0 s
  13. 13
    Hide the camera's display.Request · Expected: Camera's display hidden
    Rightset_camera_display_overlay(on: false)“Camera's own display overlay is hidden; Studio's scopes and peaking still show.”
    3.9 s
  14. 14
    Start recording.Request · Expected: Recording
    Rightset_recording(recording: true)“Recording has started.”
    3.6 s
  15. 15
    Hello, and welcome back. Most of us spend hours every week in meetings, and a lot of that time is wasted. Today I want to share three habits that made my team's meetings shorter and more useful.The video · Expected: Nothing (the video); still recording
    Right“Recording is in progress and this is your talking track — I'll leave the camera alone.”
    4.4 s
  16. 16
    The first habit is a written agenda. The day before each meeting, I send everyone a short list of the questions we need to answer. If I can't write that list, we probably don't need the meeting.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    3.9 s
  17. 17
    It also means people arrive prepared. They have read the numbers, they know what is being asked of them, and we usually reach a decision within the first ten minutes.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    3.3 s
  18. 18
    When I began doing this, about a third of our regular meetings simply disappeared from the calendar, and nobody missed them.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    3.6 s
  19. 19
    Cut.Request · Expected: Recording stops ("Cut.")
    Rightset_recording(recording: false)“Take ended — recording stopped.”
    4.5 s
  20. 20
    Delete the last clip.Can't be done · Expected: Nothing deleted; doesn't claim it was
    Right“There's no delete capability in my tools, so I can't remove the clip. I can list the card files if you want me to confirm which one it is.”
    4.5 s
  21. 21
    How much battery is left?Question · Expected: Any answer; nothing changes
    Right“80%.”
    4.4 s
  22. 22
    Make it a bit darker.Request · Expected: A bit darker; shutter stays
    Rightadjust_exposure(stops: -0.5, using: "auto")“Made it a bit darker — ISO reduced from 1600 to 1250.”
    4.4 s
  23. 23
    Roll camera.Request · Expected: Recording
    Rightset_recording(recording: true)“Recording started.”
    5.8 s
  24. 24
    The second habit is a shorter guest list. I only invite the people who make the decision or do the work. Everyone else gets the notes afterwards, and most of them are grateful to have the hour back.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    4.1 s
  25. 25
    The third habit is ending with names and dates. Before anyone leaves, we say out loud who is doing what, and by when. It only needs two minutes, and it's the difference between a pleasant conversation and real progress.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    3.7 s
  26. 26
    None of this needs new software or a bigger budget. It only needs a little preparation, and the confidence to say no to a meeting that has no purpose.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    3.8 s
  27. 27
    If one of these habits helps your team, let me know in the comments. Thank you for watching, and I'll see you next week.The video · Expected: Nothing (the video); still recording
    Right“Recording continues — no camera changes made.”
    4.6 s
  28. 28
    Cut.Request · Expected: Recording stops ("Cut.")
    Rightset_recording(recording: false)“Take ended — recording stopped.”
    4.3 s
  29. 29
    What's on the card?Question · Expected: Any answer about the card; nothing changes
    Rightlist_card_files(limit: 50)“The card holds four clips, all recorded today/yesterday: C0010.MP4 and C0009.MP4 (both ~874 MB, the last two takes), plus C0008.MP4 (~874 MB, Nov 10) and C0007.MP4 (~912 MB, Nov 10).”
    8.2 s
  30. 30
    Download the last two clips.Request · Expected: C0009.MP4 and C0010.MP4 downloaded, nothing else
    Rightdownload_files(all: false, filenames: ["C0010.MP4","C0009.MP4"])“Both clips copied to the downloads folder: C0009.MP4 and C0010.MP4.”
    5.6 s
  31. 31
    Switch to photo mode.Request · Expected: Photo mode; no photo yet
    Rightset_setting(setting: "exposureProgram", value: "Manual")get_camera_status()“Switched to photo mode (Manual exposure). Note the shutter is still 1/50, which is slow for handheld stills at ~21 mm — you may want to raise it.”
    5.2 s
  32. 32
    Take a photo.Request · Expected: One photo
    Righttake_photo(autofocus: true)“Photo taken and saved as DSC00042.JPG.”
    5.1 s
  33. 33
    Turn off live view.Request · Expected: Live view off
    Rightset_live_view(on: false)“Live view is now off.”
    5.1 s

The run

How it ran

Score
73 / 100
#17 of 38 models
Load time
1.5 s
Download
16.61 GB
Model file
Inkling-2-Mini-UD-Q4_K_M.gguf
sha256 f90c5a4df44d335a…