Case study · May 2025 — Present
Two LLM agents that trade unattended on my own hardware, on my own network
What I built, what broke, and what I changed
Two independent agents run on Raspberry Pi hardware in my apartment, reachable only over a private Tailscale network, with no human in the loop. They research, decide, place orders, and evaluate themselves. Below is the architecture, and then the run where they lost seven times out of seven and what the postmortem turned up.
- Python
- Anthropic Claude API
- Raspberry Pi
- Tailscale
- Linux
- systemd
- SQLite
- Backtesting
On my own hardware, on my own network
The agents run on two Raspberry Pis on a shelf in my apartment rather than in a cloud account. That was a deliberate limit, and an interesting one to work under. Fixed hardware makes you decide what runs locally, what a model call actually costs, what happens when the power blips, and how the system recovers with nobody watching.
The two boards and my laptop sit on a private Tailscale mesh. There is no public ingress and no open port on my router — I reach the agents the same way from home or from anywhere else, and nothing else can. Deploys, log inspection, and the operator CLI all run over that mesh.
Uptime here is something I built rather than something I bought. The agents have run continuously since May 2025, through power blips, network drops, and a few of my own bad deploys.
Making hardware and models behave like one system
The part I find most interesting sits where the model meets the machine. An LLM is non-deterministic, occasionally slow, and sometimes wrong. A Raspberry Pi has finite memory, an SD card with a limited write life, and a power supply sharing a socket with a lamp. Neither is production-grade on its own.
What holds it together is fairly ordinary systems engineering applied to a non-deterministic component: systemd units with restart policies, the two Pis health-checking each other so one notices when the other stops reporting, state in SQLite with automated backup and verified restore, and a deterministic path that runs before the model for the decisions I did not want left to its judgement.
The result is a system I do not administer day to day. I do not SSH in to keep it running, and I have not needed to watch a deploy land. Running an LLM without a person standing behind it is the part of this I would most want to talk through.
Routing: three model tiers instead of one
Every task is dispatched by complexity. Haiku handles classification and triage, Sonnet handles analysis, and Opus is reserved for decisions that justify the cost and latency. Cost control lives in the routing table rather than in a budget alarm after the fact.
There is a tradeoff: cheaper tiers occasionally misclassify, so the routing rules escalate to a stronger model rather than accepting a low-confidence answer.
Evaluation: the system reverts its own regressions
Strategy changes are validated against an eight-year backtest harness. The agents score their own recent performance, and a change that measures worse than what it replaced is rolled back automatically.
This is the piece I would defend hardest. Automatic reversion is what makes the autonomy safe to leave alone; without it, a bad change just runs until someone happens to look.
The failure: seven closed round trips, seven losses
Over one clean evaluation window, every closed trade lost money. The obvious read was that the strategy had no edge. The postmortem found something different: the system had no working take-profit path at all.
One position peaked 12% above its own target while the agent's reasoning insisted the target was still achievable, then round-tripped back to a breakeven stop. Every single realized exit in the window was a stop fill. Winners never closed. Only losers did.
A mechanism bug had been presenting as a strategy problem. Since then I separate the two explicitly whenever I evaluate anything.
What I found underneath it
A volatility input had been silently defaulting for weeks. A length guard on the data fetch was never satisfied by the range being requested, so the code fell through to a hardcoded default without logging anything. Nothing errored, so nothing surfaced.
Two learning gates were set to thresholds the system could not reach, so that feedback loop had never fired once. And one agent's self-reflection was writing confident summaries of trades whose profit and loss it had never been shown.
The fixes, and how I verified them
Deterministic target exits now run before the model is consulted, so taking profit no longer depends on an LLM agreeing it is time. The volatility fetch was widened and emits a degraded-mode warning instead of falling back silently. The learning gates were recalibrated to reachable thresholds. Self-reflection now receives the actual profit and loss.
All five shipped the same day, verified by checksum on both machines with a clean restart and no crash loop, plus a live circuit-breaker drill on each Pi.
What I would do differently
Instrument the exit path first. I had good monitoring for whether the system was alive and very little for whether it could still do the thing it existed to do. Liveness was the easy signal to collect, which is probably why I collected it.
Alert on defaults. The volatility bug went unnoticed for weeks because falling back to a default looked exactly like working correctly, so fallbacks now log.
Do not let a component narrate its own performance without giving it the numbers. A model asked to reflect will produce a confident reflection whether or not it has the data behind it.
The circuit breaker, to scale
Three levels, each with its own trigger and its own blast radius. Trip one to see what it does.
Armed. Select a level to see what it halts.