[bmdpat]
Download BMD free

Agent check · measured 2026-10-12

gpt-oss:20b MXFP4 passed 0 of 10 checked coding tasks.

Codex CLI in OSS mode drove each model through Ollama in a container. Each task's check was written before the run and ran after the agent stopped, with run_check.py and tests/ restored from the fixture first.

A task passes when its check exits 0 and the showwork gate reads GREEN. Hardware: one RTX 5090. Agent: codex-cli 0.162.0. Runtime: ollama 0.31.2.

In 10 of 10 tasks, Ollama closed the stream before the agent finished, so no turn completed and no token count was reported. The check counts each one as a fail. This measures this agent and runtime pair, not the model's code alone.

  1. Add a command-line option

    Failed

    Task: Add a --limit option to main() in cli.py: an integer, default 10, that caps how many lines main prints.

    Why it failed: The model runtime closed the stream before the agent finished.

    31 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-cliflag-20261012T190353Z

  2. Handle missing and bad input

    Failed

    Task: Implement load_port in config_reader.py so it does what its docstring says.

    Why it failed: The model runtime closed the stream before the agent finished.

    40 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-config-20261012T190353Z

  3. Implement an ordered, case-blind dedupe

    Failed

    Task: Implement dedupe in dedupe.py so it does what its docstring says.

    Why it failed: The model runtime closed the stream before the agent finished.

    26 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-dedupe-20261012T190353Z

  4. Parse a small input format

    Failed

    Task: Implement parse_duration in duration.py so it does what its docstring says.

    Why it failed: The model runtime closed the stream before the agent finished.

    29 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-duration-20261012T190353Z

  5. Merge nested settings without side effects

    Failed

    Task: Implement deep_merge in merge.py so it does what its docstring says.

    Why it failed: The model runtime closed the stream before the agent finished.

    68 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-merge-20261012T190353Z

  6. Fix an off-by-one bug

    Failed

    Task: page() in pager.py returns the wrong items. Page numbers start at 1. Fix the bug.

    Why it failed: The model runtime closed the stream before the agent finished.

    33 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-paginate-20261012T190353Z

  7. Rename a function across two files

    Failed

    Task: Rename get_usr to get_user everywhere in this project. Remove the old name.

    Why it failed: The model runtime closed the stream before the agent finished.

    23 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-rename-20261012T190353Z

  8. Read a CSV file correctly

    Failed

    Task: Implement total in sheet.py so it does what its docstring says.

    Why it failed: The model runtime closed the stream before the agent finished.

    35 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-sheet-20261012T190353Z

  9. Implement a function from its docstring

    Failed

    Task: Implement slugify in textutil.py so it does what its docstring says.

    Why it failed: The model runtime closed the stream before the agent finished.

    32 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-slugify-20261012T190353Z

  10. Fix shared state between calls

    Failed

    Task: Calls to add_tag in tags.py leak tags into each other. Fix the bug without changing the function signature's meaning.

    Why it failed: The model runtime closed the stream before the agent finished.

    73 s · tokens not reported · showwork gate RED

    Changed: nothing

    showwork session agent-check-gpt-oss-20b-tags-20261012T190353Z

What this does not show

Ten small, single-file Python tasks are not a benchmark. A pass means the tests passed, not that the code is good. Other quants, other runtimes and other agents can score differently.

Want checks on your own agent work? BMD runs them after each Codex or Claude Code task. Download BMD free.