Agents that hold up when nobody is watching.

Where I've gone deep

I build systems end to end. Every system on this page I designed and built myself, from an empty directory to something that works.

The part I have gone furthest on is what decides whether one survives contact with real use. An agent that behaves well under supervision tells you very little about how it behaves at 3am in week eleven, and that gap is where I have spent this year.

Open on the rest

That is expertise, not a menu. I like problems I have not met before, and the methods travel — it is the same problem in a support workflow, a data pipeline or a build system as it is in a trading loop. The tools under this work will not be the same ones in two years, so the rate you pick the next one up at matters more than the list you already have. If it is something I have not built yet, that is a reason I want it.

What I've had to solve

Four problems came up on my own systems that cost enough to be worth writing down. Each is worked out in full further down; this is what they were.

None of these are trading problems. They are what happens to any system that has to keep working when nobody is checking, and they are the reason I would rather be handed a system that is already misbehaving than a blank page.

The work, and what each one took

These are private, live systems, with one exception below whose whole source is the page you can open. What follows is the engineering at the level of detail a technical buyer can judge. I walk through the code directly, in writing.

How the loop is put together

The shape below is the one I keep rebuilding, in trading and outside it. Two details carry most of the weight: state is read from ground truth on every pass instead of being carried forward in the model's own summary, and the places where the system can refuse are explicit, few, and sit on the path itself.

The durability ladder

Across 27 fixes I shipped and then tracked, roughly half held and half decayed, and the split lined up with one candidate axis: not correctness, not importance, but the location of authority. Read the caveat below before you believe it — in this sample I cannot separate that axis from a second one.

Nouns rot. Verbs hold. That is the whole finding in four words.

The permanence test

Does the fix still hold for an actor who never read it? A fresh context inherits the codebase's behaviour and never an instruction's force. Any fix enforced by “the actor reads X and chooses to comply” is a noun with extra steps, and it will rot the first session the actor is under time pressure or has not loaded X.

The top rung is conditional

I learned that by being wrong about it. I removed an affordance — a browser tab that could be disturbed mid-job — and called that class of bug impossible. It recurred: the removal was itself an action someone had to perform, and on a path nobody had walked it never ran. So the test is stricter: is the affordance never created, or is its removal a verb that fires whichever path the actor took? A removal-by-action is rung 2.5, and it fails exactly where you never walked.

Three findings underneath it

A verb guards an action; a rule names a behaviour. “Don't disturb a running job” has at least three action-faces — kill it, touch it, over-poll it. I guarded the kill face, marked the behaviour fixed, and the touch face failed weeks later. Enumerate every face and guard each, or write down which remain unguarded.

Recognition needs a chokepoint too. At the moment of recurrence the lookup have I seen this exact thing before? never fires. I had six dated recurrences of one bug on record and read none while diagnosing the seventh.

Delegate the work, never the clock. Nothing self-re-animates a dormant agent — an inline sleep returned early, a background child never woke its parent. The durable form of “continue this later” is the foreground re-invoked by a harness event.

Guards and their ceilings

The hardest constraint problem I have worked on is enforcing a limit against an actor with both the access and the motive to route around it. Person or agent, it is the same problem: the guard has to hold at exactly the moment the actor most wants it gone.

I built five mechanisms and put each in front of live conditions. Every one has a real ceiling, and the pattern across them is the transferable finding.

That last row is the one I would carry into any system with an agent or a person in the loop. A partial block does not stop the behaviour; it moves it into the channel you cannot see. Covering most of the surface does not buy most of the protection — it buys the same behaviour, now invisible, and usually larger. Defend the resource, not the interface.

What did hold

A layered posture, correctly labelled. Friction that announces itself: a re-entry path that takes ten minutes and cannot be done quietly. A single instrumented channel, so every action lands on one recorded surface and diagnosis afterwards is possible at all. And ground-truth measurement: judge by the external system's own state, never by your log, because the path you care about leaves no rows in it.

When the evaluator is the thing lying to you

Everything above is about making behaviour hold. This is about whether the evidence you decided on was real, and it comes from the research platform.

Its flagship candidate came back at Sharpe 13.76, probability of overfitting 0.0%, deflated Sharpe 1.0000. Every standard referee passed it. It did not survive contact with live execution. The search was not broken: it was optimizing against a simulator with a defect, and the overfitting statistics were computed inside that same defective simulator. Re-run on an honest execution model over the identical 89 sessions, the same candidate inverts to Sharpe −1.69. The backtest had been faithfully describing an engine that did not exist.

Three laws, each with a price attached

• An optimizer converges on your evaluator's defects, not your objective. Goodhart is the default output of any search loop given enough iterations, and it is the modal outcome rather than an edge case.
• Standard referees check the return side and cannot see a violation on the signal side. A separate screen passed bootstrap, placebo, a comparator and all 13 trigger years at +150.9bp. With one look-ahead field removed it went to −9.2bp, no longer distinguishable from zero at that sample. Only an information-set audit catches that class.
• A green test suite certifies capability presence, not operational effect. 1,571 passing tests across the two engines covered 16 self-improvement mechanisms. In a real multi-hour run, two of them fired. The rest were wired, enabled, tested and inert, most killed by a single silently-empty collection at an adapter boundary.

What worked was inverting the order: validate the validator against a known null before trusting any positive. Synthetic driftless data, the full gate stack, roughly 66,000 trades, zero survivors out of thirty. On data with no edge the evaluator manufactured none — recorded with its own bound, since 0/30 constrains the false-positive rate to about 10% and no better.

If you run an agent loop against an LLM judge or a synthetic eval, this is your problem and not a trading problem. The loop will find your judge's blind spots faster than it finds the behaviour you asked for.

Building the control group

Every gauge measured the calls the system took. Across 409 recorded decisions it had never once scored what it rejected, so the question underneath all the others had never been asked. The instrument: each real decision scored against random entries from the same day, same direction and same stop distance, differing only in which minute was chosen.

That run read 0.496 against a 0.500 chance line, and the honest statement of it is precise rather than dramatic: at that sample size no effect was detectable — a different claim from “there is none”, with the interval saying how large an effect would still have been missed.

Which is what it was for. It produced the first result capable of redirecting the work rather than decorating it, and it did: selection moved out of the live loop into an offline pipeline where a rule is pre-registered, run once and killed cheaply. That pipeline is the current work.

Then the new instrument tried to fool me, twice, in one evening

First real use: three mechanical rules, frozen in a commit before anything ran. One came back at 0.725, its confidence interval clear of chance — a survivor on the first attempt, exactly the result I wanted.

One question killed it. Is this measuring the moment the rule chose, or where in the local price range it happened to enter? Under a fixed stop distance with no profit target, an entry below its neighbours has a longer run to the close — it wins arithmetically whether or not it predicted anything.

The control had a free parameter of its own — how close a matched entry has to sit — so both settings are published rather than one. Tightening the match from a quarter of the stop distance to a tenth takes the survivor from 0.618 to 0.529, on a matched pool that is itself centred at 0.499. And the parameter is not neutral at any setting: run the same arm on an edge-free tape and the placebo reads 0.5226, 0.5065, 0.4813, 0.4494 as the band widens from a tenth of the stop to a whole one — moving in one direction throughout, so there is no band that is simply the right one. Four candidate corrections have been measured and all four failed, which makes this a live limitation rather than a closed one, and it is exactly why tuning the band until the null lands on 0.500 is a standing prohibition here: it would be calibrating the instrument on the tape it exists to judge. Where two settings disagree, both get shown.

Read the third row: the “best” rule was the one wasting the most favourable entries it had been handed. Once the control matched on price as well as time, all three collapsed toward chance and the survivor was withdrawn into the same append-only ledger that had recorded it — and the pace gauge then went on reporting survivors: 1, because an append-only log and a read-every-row-as-current gauge are individually reasonable and jointly incapable of representing a retraction. Fixed to take the latest verdict per candidate, proved by mutation, with a regression test asserting both values.

Three further false findings surfaced the same night, each caught by running a check rather than reading it: an integrity gate excluding the primary instrument over “weekend” bars that were my own timezone convention; a table appearing to show older history degrading, where every degraded year was a public holiday; and a cross-feed price disagreement of fourteen points, because it compared one feed's close against the other's within-minute average.

A control group that differs from the treatment in more than one way measures the difference you did not intend. If you A/B test agent behaviour, that is the failure most likely to cost you: not a broken metric, but a well-built one whose comparison arm quietly differs on a second axis. The tell is that the result is the one you wanted and it arrived sooner than expected.

Letting an agent modify its own machinery without letting it ratchet

The agent designs and implements its own tooling, which removes the natural over-build alarm, so the constraints have to be structural.

• A budget on what may refuse, re-counted at session start. It began as a cap of eight surfaces, net-zero: add one, retire one. Two things went wrong in a single evening. Re-running the registry's own enumeration showed the count was stale rather than breached — at least eleven surfaces could already refuse a live order and four had never been booked, so every earlier “within budget” verdict had been computed against a false baseline. Then I replaced the cap with a two-class scheme I was confident about, and the external seat returned TERMINAL on it the same night: the example motivating my change was forbidden by three artifacts that already existed and was arithmetically impossible, and what the change actually did was let a caller exempt itself by declaring its own class. The cap was restored, and the ceiling reset only after the full enumeration was put in front of the person whose money it protects. My redesign was the thing that was wrong, and the outside seat is what caught it. The count itself warns rather than blocks — by the ladder above that makes it a noun, and it is owed a verb.
• Tiered change authority. Reversible fixes ship autonomously, parameter tunes ship but are logged, and any new mechanism requires an explicit proposal. What a change touches sets its tier, not how the agent narrates it: “it's only a knob” is not a downgrade.
• External, non-self-graded refutation. Any significant design is attacked by a fresh context given the proposal and the project's known anti-patterns and told to break it, with the verdict pasted in verbatim. Its disclosed limitation is written into the seat itself: the brief it receives was written by the author, so it is told not to accept that framing as fact. The first change made under this charter was found redundant and built on a false premise, and reverted to zero net code.
• Ship the gauge with the mechanism. Every mechanism carries an outcome-based success criterion, and an audit fails the build if a decision record declares a gate without shipping its measurement. “The gauge fired” is not evidence.

Underneath all of it is one habit: audit the system's claims about itself and write down where they were false, in place, next to the claim being corrected. A rule set documented as enforced was audited and found to be enforced in some places, advisory in others and absent elsewhere — and the document was rewritten to say so in bold rather than quietly patched. This matters most in agent systems, because an agent will confidently narrate an obligation as satisfied. Every claim of “done” that is not derived from ground truth is one you will pay for later.

How I work

Written-first

You get decisions in a form you can re-read three weeks later instead of a meeting you have to remember — the record is part of what I deliver, and it is the same discipline the systems above run on. Calls when a decision needs one. I am on CET, overlapping European hours and US mornings.

What I look at first

What the work turns out to be depends on your system and I would not pretend to know before reading it, but the opening questions are the same ones every time: where each guard actually lives, which obligations are written in a document rather than wired into the runtime, whether any delegation boundary is losing work silently, whether “done” is derived from ground truth or reported by the thing that did it, and whether the evaluation measures what you think it measures. Scope comes out of that conversation instead of out of a template.

Built to be handed over

Every system here was built to be picked up cold by someone who was not there. That constraint is why the operating contract is injected rather than remembered, and why the Unity project prints its own next task with the traps and the done-condition attached and refuses to close it without evidence. If your team has stopped trusting what the agent reports, that handover surface is what I would build first.

Why I take this work, and the time I have for it

The heavy build on my own systems is behind me: I stopped the live loop and moved the work offline behind a stopping rule I had signed in advance. What remains runs without me sitting in front of it, which was the point of building it that way, so the time is real rather than squeezed in around something else.

What contract work gives me is the part I cannot build alone: other people's architectures, other people's constraints, and failure modes I have not met yet. This field moves fast enough that working across it is how the transferable part stays transferable. And I like the work, which is worth saying plainly when you are deciding who to spend six weeks with.

The engagements that go well have a hard, specific problem in them — a system misbehaving in production, or one about to be built while the tool surface, the refusal points and the measurement can still be decided early. That is the part I am good at and the part I enjoy, and it is worth pointing at before we start.

On what stays private

Apart from the game and the MCP server at github.com/zorionarrillaga/oas-execute-mcp, the systems here are unpublished — no repository, no code, no proprietary logic. That is a policy rather than a gap, and it is the same policy your work would sit under. Instead I write up any part of it in as much detail as you want to read, and keep the first piece small enough that you are judging the work rather than the description of it.