The Model Was Fine: What I Actually Did to Make Ornith Usable

by

Part two of my local coding agent build. The model did not need replacing. The tools and the rules around it did.

The first post was the build: Ornith 1.5 9B on my own hardware with Pi and Pi Web. This is what happened after using it every day on a real project — my blog, with a database, uploads, a login page, and templates that have to look right in a browser. Same model. I changed the tools it can reach, the rules it works under, and the settings it runs with.

Why Everyone Says These Models Are Bad

The YouTube version is always the same. Download a 7B or 14B, load it with the defaults, ask for a Python function, get something that will not run, conclude the model is bad. I believed it my first week.

What those videos skip: a model does not choose the next word alone. The server filters the choices first, and Ollama, vLLM and llama.cpp each ship different defaults. Three people can test the same model, get three results, and none of them say what was switched on.

Those defaults were tuned for chat, and two of them wreck code.

Repeat penalty. Code repeats on purpose — the function name, self., the import, the closing brace. A strong penalty pushes the model off words it already used, so it invents new names for the same thing. It looks fine and falls over when you run it.

Top-k cut-off. The exact variable name in your file is often not in the top twenty. Once it is gone the model has to pick another word, and another word in code is a NameError.

Both come out looking like bad code, so people blame the model. I have not found a post that reran the prompt with the penalties off before deciding.

Small models do know less. But you can tell the two apart: if the plan is right and the code is mangled, the filter ate the words; if the plan is wrong, the model did not know. Only one of those is a shopping problem.

  • Good English, broken code. The filter. Retry with the penalties off.
  • Fine, then it forgets the plan. Context. Check what the server gives it.
  • It will not use a tool that exists. The tool is not in the session.
  • It repeats something it proved wrong. The summary, not the model.

Why I Picked This One

Half the reason is the GPU: the Q4 build is about 5.5 GB, so it fits on my 4070 with room for a long context. A 30B does not, and one that fits leaves nowhere for the conversation.

The other half is how Ornith-1.5 was trained. The team calls it end-to-end self-improvement: the model writes its own practice tasks, writes a scaffold for each — instructions, tools, plan — attempts it, and gets scored on all three. Target difficulty is a task it passes about one time in five, so practice gets harder as the model improves, with nobody writing the exercises. It is why the prompts that work describe a procedure, not a personality: I am handing it the shape it was trained to use.

Numbers, and I have not reproduced them: SWE-bench Verified 70.6, against 52.0 for Gemma-4-31B and 53.2 for the Qwen3.5-9B it is built on — a 9B beating a 31B at something close to real bug fixing. Terminal-Bench 2.1 is 46.2, or 47.0 under the Claude Code harness. Tool use is more mixed: 54.2 on MCP-Atlas, level with Gemma-4-31B and behind the bigger 35B at 62.8. What settled it was tool calling. A model that holds a tool call together through a long session is worth more to me than one that writes prettier functions.

The Context Window Is the Real Feature

It was trained on 262,144 tokens. I run 131,072 — half — because the cache is quantised to q8_0 and 128k keeps it in budget on a 12 GB card. Past the trained length you need RoPE scaling, and the card says what that costs: the factor applies to every request, so it shaves quality off short ones.

What the window buys, in order:

  • Room for the tools. Three servers, 43 tools, all prompt space before I type a word.
  • Room for the ledger. 4,000 characters of settled facts at the top of every request.
  • Room to read a file and keep the plan instead of losing the reason I opened it.
  • Room to recover. On compaction Pi keeps the last 48,000 tokens word for word and holds 16,384 back for the reply.

The Tool That Was Never Switched On

For two days I was sure the sandbox bridge was broken. It was not. The tool was not there.

Pi ships codemode, a JavaScript sandbox, switched off. A normal session gets four tools: read, bash, edit, write. Pi Web starts chats the same way, so every browser chat had no sandbox. The giveaway was how it failed: asked for the sandbox, it ran the JavaScript through bash, where it dies as a shell command, or wrote the script to a file and never ran it. Both mean “I do not have that tool.” It looked lazy. It was being honest.

One line in ~/.pi/agent/settings.json:

{
  "defaultTools": ["+codemode"]
}

A plain name replaces the tool list; +codemode adds to the four already there. Restart Pi Web, not just the page — the list is read at session start. Check from the terminal first:

pi --tools +codemode -p "call codemode: return 1+1"

This is not a hack: Pi Web reads the same file and labels the picker “From settings.json defaultTools.” The button in the web page writes that line. I had spent two days fighting the server over a checkbox.

With the sandbox on, the next failure cost more:

Script completed
Wall time 0.1 seconds
Output:

Nothing after Output:, fast every time, which from inside the chat looks like the server never answered — two more days blaming the network. It was how the script ended:

async function main() {
  const page = await tools.mcp__chrome_devtools__new_page({ url: "http://localhost:8000/" });
  return page.content[0].text;
}
main();

main() walks away. The script finishes, the sandbox shuts it down, and the browser call in flight is cancelled: status: "cancelled", error: "MCP request aborted", one level down. The tool ran. The answer was thrown in the bin.

Skip the wrapper; the sandbox already runs your script:

const page = await tools.mcp__chrome_devtools__new_page({ url: "http://localhost:8000/" });
return page.content[0].text;

It kept writing the wrapper anyway. A 9B has habits, and a habit beats a bold paragraph in the system prompt once the chat is long. So I fixed the last line before the script runs:

const TRAILING_CALL = /(^|[}\n;])[ \t]*main\(\)(?:\s*\.\s*(?:catch|then)\s*\([\s\S]*)?\s*;?[ \t]*$/;
return code.replace(TRAILING_CALL, "$1\nreturn await main();");

It only touches a script that declares main and ends with a bare call, and it logs every call, changed or not — the difference between “the hook never fired” and “the hook fired and did nothing.” It fired on the first run, on a script copied straight out of the model’s habit. Do not ask a small model to break a habit. Fix the habit where it happens.

Three Servers That Made It Useful

Everything above is the model behaving itself. An agent that cannot look anything up is a confident guesser, and a 9B has less to guess from.

Context7, documentation. resolve-library-id finds the library, query-docs reads the part it needs, and it quotes what it found. Before this, a Flask helper question got an answer that looked like the docs and was not.

SearXNG and Crawl4AI, search. On my own network, port 3003, so nothing leaves the house. Nine tools exposed, four used: search_web, scrape_url to read a page instead of trusting the snippet, batch_scrape, crawl_website. This is what breaks loops: on an error it does not understand, search the exact error text instead of trying the same command again. That habit replaced most of the thrashing.

Chrome DevTools, the browser. 32 tools, and the only part of the setup that can tell me what a page actually does. All three run "exposure": "direct": callable from the chat and from inside the sandbox, no bridge.

$ pi mcp list
context7: connected, 2 tools (direct, global)
searxng-crawl4ai: connected, 9 tools (direct, global)
chrome-devtools: connected, 32 tools (direct, global)

I run that before debugging anything now. Two of my worst days were spent chasing servers that were connected the whole time. The server was fine; the tool was not in the session, and nothing says so.

The Browser Is the Only Honest Witness

What I cared about was not whether it could write code, but whether it could tell me a page works without me opening a browser.

npx -y chrome-devtools-mcp@latest \
  --headless --isolated --viewport 1280x800 \
  --screenshotFormat jpeg --screenshotQuality 70 --screenshotMaxWidth 1024

--isolated is the flag I would not give up. Without it the agent’s browser and mine share a profile and steal each other’s tabs, which looks like a broken tool and is two people using the same mouse.

It paid for itself in an hour. The agent opened my blog, read the title, and said it was empty. Real bug: Flask handed site_tagline to every page but never site_title, so all four public pages had no title or description. Two lines of Python.

I had been checking that page with curl against a container serving an old copy of the file. The browser reading the same empty title is what made me read the file instead of trusting the container.

The Rules I Wrote Down

A prompt lives in the chat and dies with it. Rules that must survive a long session go in a file: mine is ~/.pi/agent/APPEND_SYSTEM.md, which Pi adds to the system prompt for every project, terminal and web UI alike.

  • Tool gate. Check docs, search or browser before answering about a library, version or page, with a line that says when none applies.
  • Loop rules. Fail twice, stop, search the exact error. Fail three approaches, stop and ask. Never run the same command more than twice. A 9B does not notice it is going in circles, so the rule notices for it.
  • Probes. A command only counts as evidence if it exercises the code the way the app does. Make one that can succeed before you trust one that fails.
  • Quiet rules. Three paragraphs of thinking on one decision, then act. Do not explain twice. Do not argue with yourself.

One file: edit it, run /reload, both interfaces pick it up. Every rule makes it longer, and length competes with the real task, so I have deleted more than one that was only taking up space.

The Loop It Could Not Forget

The worst loops were not inside one answer. They were in the summary between answers. When the chat fills up Pi squashes the conversation so it can continue, and the summary has a Key Decisions section where anything the model felt strongly about becomes fact. Mine was “the browser tools are unreachable.” They were not. It had called a tool, got the empty answer above, invented a reason, and the summary recorded it as a decision. For hours it proved a problem that did not exist while the tool worked fine when I ran it.

You cannot prompt your way out, because the summary is written by the same model that got it wrong. So I moved the facts out of the chat. .pi/DEAD_ENDS.md is append-only; an extension pastes it in as the first message of every request, tagged as context, rebuilt from disk each time so the summary cannot reword it.

## YYYY-MM-DD - one line verdict
- Settled: what is actually true
- Do not retry: the exact command or approach
- Why: one line on why it cannot work
- Instead: what to do

Before retrying something marked do not retry, it must give one sentence of new evidence, and a summary is not evidence. The honest problem is size: mine is over 100 KB and only the last 4,000 characters get pasted, so old entries fall off and the injected text says something was cut. Cheap enough to clean, cheaper than the same dead end at 3 a.m.

The Filter I Added Myself

I also run a loop detector, and for a while it did exactly what I complained about in the first section: cut good work. Its defaults expect a chatty model that thinks out loud; mine thinks with the display off, so:

  • Thinking window 0 — no thinking text to read.
  • Output window 400, up from 100 — the repeat I care about is a paragraph, not boilerplate.
  • Fingerprint 160 characters — shorter ones fired on normal boilerplate.
  • Semantic threshold 7, up from 3, which cut in mid-explanation.
  • Tool ban 1 — doing the same thing twice is the bug.

Before the first two changes it cut long edits off halfway, which looks exactly like a model that cannot write a forty line function. Same mistake as section one, with my own hands on it. The cost is that a blocked call stays blocked, so a chat can get stuck filling up with refusals. The detector is a seatbelt: it stops the crash, it does not steer.

I Used Bigger AI to Fix My Small AI

Reading what I could not. Pi Web’s front end is bundled and minified; a big model found the defaultTools line and explained how the picker fills its list, which is how I learned my fix was the intended path. Small fiddly bits: I described the broken script exactly and got the pattern above, then ran it against the real input. Checking my work: I asked what the hook would do if main() appeared in a comment or was nested; it found two cases I had missed.

What they were bad at was anything about my machine: three Chrome flags that do not exist, a settings key not in the schema, and a reinstall of a server that was never the problem, all with total confidence and one evening each. Let a big model read and explain what is in front of you; do not let it tell you facts about your own hardware.

The Settings, In Plain English

{
  "contextWindow": 131072,
  "maxTokens": 12288,
  "samplingParams": {
    "reasoning_budget_tokens": 2048,
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "min_p": 0.0,
    "presence_penalty": 1.0,
    "repeat_penalty": 1.0
  }
}

Temperature 0.6 — leave it. Higher wanders mid-edit, lower grabs the first file it sees. Also the card’s number for precise coding.

Presence penalty 1.0 is the one I got wrong, and the card says so. Their coding preset is temperature 0.6, top_p 0.95, top_k 20, min_p 0, presence_penalty 0.0, repeat 1.0 — my list except the penalty. I set 1.0 because with the thinking display off a long chat otherwise repeats its last paragraph as its next answer. That trade belongs at the tool call, not the model file. If you copy my config, take their number.

Top k 20 is straight from the card; min p 0 is doing nothing on purpose, since one cutoff is enough for code.

Reasoning budget 2048 is half the first post’s: cheaper answers, less circling. I raise it to 4096 when chasing a bug across files. Max tokens 12288 also caps summaries — cut it to 4096 and they get chopped, and the wrong belief problem returns in a new coat.

What I Would Tell You Before You Start

  • Rerun the prompt with the penalties off before you judge a model. It is the difference between a guess and a review.
  • Read the model card’s sampling presets. I spent weeks tuning numbers they had already written down.
  • Give it a window it was trained for. Half is fine; stretching past it with scaling gives you a model that is big and vague at once.
  • Connect the tools first. Docs, search and a browser improve the agent more than any sampling value.
  • Check the tool list before you debug the tool. Connected and available are different questions, and the second one bites.
  • Write the loop rules down, in a file. In a prompt they last one chat.
  • Start the dead ends file on day one, not after the first bad summary — by then the wrong answer is already a decision.
  • Fix habits in a hook, not in the prompt. A hook runs every time; a prompt works until the chat gets long.
  • Check pages in a browser, check files by reading them back. Only one tells you what the app really does.

What I Kept

  1. The model is the whole thing, not just the weights. The filter, the tools, the template and the summary all break in ways that get blamed on the model.
  2. Broken code usually means eaten words. Renamed variables are a settings report.
  3. Tools are the upgrade. Docs, search and a browser turned a clever toy into something I can hand a real task.
  4. A model trained to build scaffolds wants one.
  5. Fix the habit where it happens. Three lines at the tool beat a paragraph of shouting.
  6. The tool you think is broken might just be switched off.
  7. Anything you need to keep belongs on disk. A summary is a storyteller, not a witness.

Same stack as the first post, with everything around it changed: three servers, a page of rules, a hook for its worst habit, a list on disk it cannot forget, and a detector tuned until it stopped eating good edits.

If your small model writes code that looks right and runs wrong, do not go shopping yet. Check the filter first. A lot of the time the model was fine and the settings were calling it stupid.

Leave a Comment

WordPress Appliance - Powered by TurnKey Linux