Agent check · measured 2026-10-12
gemma4:26b Q4_K_M passed 10 of 10 checked coding tasks.
Codex CLI in OSS mode drove each model through Ollama in a container. Each task's check was written before the run and ran after the agent stopped, with run_check.py and tests/ restored from the fixture first.
A task passes when its check exits 0 and the showwork gate reads GREEN. Hardware: one RTX 5090. Agent: codex-cli 0.162.0. Runtime: ollama 0.31.2. Tokens: 2.3M in, 47k out, $0 in API fees. AgentGuard prices the same tokens at gpt-5.3-codex list rates at $4.71, before cache discounts (price table 2026-09-26).
Add a command-line option
PassedTask: Add a --limit option to main() in cli.py: an integer, default 10, that caps how many lines main prints.
245 s · 208k in / 5k out · showwork gate GREEN
Changed: cli.py, manual_test.py, test_manually.py
The agent's diff
diff --git a/cli.py b/cli.py index fd0a366..159a2be 100644 --- a/cli.py +++ b/cli.py @@ -4,7 +4,8 @@ import argparse def main(argv: list | None = None) -> int: parser = argparse.ArgumentParser(prog="lines") parser.add_argument("count", type=int, help="how many numbered lines to print") + parser.add_argument("--limit", type=int, default=10, help="cap how many lines to print") args = parser.parse_args(argv) - for n in range(1, args.count + 1): + for n in range(1, min(args.count, args.limit) + 1): print(f"line {n}") return 0showwork session agent-check-gemma4-26b-cliflag-20261012T181256Z
Handle missing and bad input
PassedTask: Implement load_port in config_reader.py so it does what its docstring says.
46 s · 251k in / 5k out · showwork gate GREEN
Changed: config_reader.py
The agent's diff
diff --git a/config_reader.py b/config_reader.py index 39bdf10..28b3df9 100644 --- a/config_reader.py +++ b/config_reader.py @@ -1,3 +1,6 @@ +import json +import os + def load_port(path: str) -> int: """Return the "port" value from the JSON file at `path`. @@ -5,4 +8,21 @@ def load_port(path: str) -> int: ValueError when the port is not an integer from 1 to 65535 (a bool is not an integer here). """ - raise NotImplementedError + if not os.path.exists(path): + return 8080 + + try: + with open(path, "r", encoding="utf-8") as f: + data = json.load(f) + except (json.JSONDecodeError, OSError): + return 8080 + + if not isinstance(data, dict): + return 8080 + + port = data.get("port", 8080) + + if not isinstance(port, int) or isinstance(port, bool) or not (1 <= port <= 65535): + raise ValueError("Port must be an integer between 1 and 65535") + + return portshowwork session agent-check-gemma4-26b-config-20261012T181256Z
Implement an ordered, case-blind dedupe
PassedTask: Implement dedupe in dedupe.py so it does what its docstring says.
26 s · 156k in / 2k out · showwork gate GREEN
Changed: dedupe.py
The agent's diff
diff --git a/dedupe.py b/dedupe.py index f2ae1ed..c9b8105 100644 --- a/dedupe.py +++ b/dedupe.py @@ -3,4 +3,11 @@ def dedupe(names: list) -> list: Keep the spelling of the first time a name appears. Do not change the input. """ - raise NotImplementedError + seen = set() + result = [] + for name in names: + lower_name = name.lower() + if lower_name not in seen: + seen.add(lower_name) + result.append(name) + return resultshowwork session agent-check-gemma4-26b-dedupe-20261012T181256Z
Parse a small input format
PassedTask: Implement parse_duration in duration.py so it does what its docstring says.
42 s · 175k in / 5k out · showwork gate GREEN
Changed: duration.py
The agent's diff
diff --git a/duration.py b/duration.py index 0450915..5a10455 100644 --- a/duration.py +++ b/duration.py @@ -1,7 +1,38 @@ +import re + def parse_duration(text: str) -> int: """Return the number of seconds in a duration like "1h30m", "45s" or "2h5m10s". Units are h, m and s, in that order, each at most once. Raise ValueError for anything else, including an empty string. """ - raise NotImplementedError + if not text: + raise ValueError("Empty duration string") + + pattern = r'^((?P<h>\d+)h)?((?P<m>\d+)m)?((?P<s>\d+)s)?$' + match = re.fullmatch(pattern, text) + + if not match: + raise ValueError(f"Invalid duration format: {text}") + + groups = match.groupdict() + total_seconds = 0 + has_match = False + + if groups['h']: + total_seconds += int(groups['h']) * 3600 + has_match = True + if groups['m']: + total_seconds += int(groups['m']) * 60 + has_match = True + if groups['s']: + total_seconds += int(groups['s']) + has_match = True + + if not has_match: + # This covers cases where the regex matched but no units were found, + # which shouldn't happen for non-empty text with this regex, + # but just in case. + raise ValueError(f"No valid duration units found in: {text}") + + return total_secondsshowwork session agent-check-gemma4-26b-duration-20261012T181256Z
Merge nested settings without side effects
PassedTask: Implement deep_merge in merge.py so it does what its docstring says.
61 s · 244k in / 7k out · showwork gate GREEN
Changed: merge.py, verify.py, verify_extra.py
The agent's diff
diff --git a/merge.py b/merge.py index f224354..02773ba 100644 --- a/merge.py +++ b/merge.py @@ -4,4 +4,13 @@ def deep_merge(base: dict, override: dict) -> dict: When both values for a key are dicts, merge them the same way. Otherwise the value from `override` wins. Never change `base` or `override`. """ - raise NotImplementedError + result = base.copy() + for key, value in override.items(): + if key in result and isinstance(result[key], dict) and isinstance(value, dict): + result[key] = deep_merge(result[key], value) + else: + if isinstance(value, dict): + result[key] = deep_merge({}, value) + else: + result[key] = value + return resultshowwork session agent-check-gemma4-26b-merge-20261012T181256Z
Fix an off-by-one bug
PassedTask: page() in pager.py returns the wrong items. Page numbers start at 1. Fix the bug.
29 s · 160k in / 3k out · showwork gate GREEN
Changed: pager.py, pager_patch.diff, patch_pager.py, verify_fix_after_patch.py
The agent's diff
diff --git a/pager.py b/pager.py index 0bd5957..708ec87 100644 --- a/pager.py +++ b/pager.py @@ -2,5 +2,5 @@ def page(items: list, number: int, size: int) -> list: """Return page `number` (starting at 1) of `items`, `size` items per page.""" if number < 1 or size < 1: raise ValueError("number and size must be at least 1") - start = number * size + start = (number - 1) * size return items[start:start + size]showwork session agent-check-gemma4-26b-paginate-20261012T181256Z
Rename a function across two files
PassedTask: Rename get_usr to get_user everywhere in this project. Remove the old name.
35 s · 212k in / 3k out · showwork gate GREEN
Changed: report.py, users.py
The agent's diff
diff --git a/report.py b/report.py index 8062d48..874fd6e 100644 --- a/report.py +++ b/report.py @@ -1,5 +1,5 @@ -from users import get_usr +from users import get_user def summary(ids: list) -> str: - return ", ".join(get_usr(i) for i in ids) + return ", ".join(get_user(i) for i in ids) diff --git a/users.py b/users.py index b2368c1..670a56f 100644 --- a/users.py +++ b/users.py @@ -1,5 +1,5 @@ _USERS = {1: "ada", 2: "linus"} -def get_usr(user_id: int) -> str: +def get_user(user_id: int) -> str: return _USERS[user_id]showwork session agent-check-gemma4-26b-rename-20261012T181256Z
Read a CSV file correctly
PassedTask: Implement total in sheet.py so it does what its docstring says.
67 s · 486k in / 7k out · showwork gate GREEN
Changed: sheet.py
The agent's diff
diff --git a/sheet.py b/sheet.py index 300f952..e307540 100644 --- a/sheet.py +++ b/sheet.py @@ -1,3 +1,5 @@ +import csv + def total(path: str, column: str) -> float: """Return the sum of `column` in the CSV file at `path`. @@ -5,4 +7,20 @@ def total(path: str, column: str) -> float: contain commas as thousands separators, like "1,250.50". Raise KeyError when the column does not exist. """ - raise NotImplementedError + with open(path, mode='r', encoding='utf-8', newline='') as csvfile: + reader = csv.DictReader(csvfile) + if column not in reader.fieldnames: + raise KeyError(f"Column '{column}' not found in CSV.") + + total_sum = 0.0 + for row in reader: + value_str = row[column].strip() + if value_str: + # Remove commas used as thousands separators + clean_value_str = value_str.replace(',', '') + try: + total_sum += float(clean_value_str) + except ValueError: + pass + + return total_sumshowwork session agent-check-gemma4-26b-sheet-20261012T181256Z
Implement a function from its docstring
PassedTask: Implement slugify in textutil.py so it does what its docstring says.
31 s · 130k in / 3k out · showwork gate GREEN
Changed: textutil.py
The agent's diff
diff --git a/textutil.py b/textutil.py index f8bf9f4..f714be8 100644 --- a/textutil.py +++ b/textutil.py @@ -1,3 +1,10 @@ +import re + def slugify(text: str) -> str: """Return a URL slug: lowercase, runs of a-z and 0-9 joined by single hyphens, nothing else.""" - raise NotImplementedError + # Lowercase the text + text = text.lower() + # Replace any non-alphanumeric character with a hyphen + text = re.sub(r'[^a-z0-9]+', '-', text) + # Remove leading and trailing hyphens + return text.strip('-')showwork session agent-check-gemma4-26b-slugify-20261012T181256Z
Fix shared state between calls
PassedTask: Calls to add_tag in tags.py leak tags into each other. Fix the bug without changing the function signature's meaning.
53 s · 295k in / 6k out · showwork gate GREEN
Changed: run_tests.py, tags.py
The agent's diff
diff --git a/tags.py b/tags.py index 52fb0b3..a34c8ab 100644 --- a/tags.py +++ b/tags.py @@ -1,4 +1,8 @@ -def add_tag(tag: str, tags: list = []) -> list: +def add_tag(tag: str, tags: list = None) -> list: """Return a list that holds the given tags plus `tag`. Never change the caller's list.""" - tags.append(tag) - return tags + if tags is None: + new_tags = [] + else: + new_tags = list(tags) + new_tags.append(tag) + return new_tagsshowwork session agent-check-gemma4-26b-tags-20261012T181256Z
What this does not show
Ten small, single-file Python tasks are not a benchmark. A pass means the tests passed, not that the code is good. Other quants, other runtimes and other agents can score differently.
Want checks on your own agent work? BMD runs them after each Codex or Claude Code task. Download BMD free.