BTA · 2026

LLM Trader & Backtest Agent

A controlled experiment: can a blind LLM read gamma maps as well as rules? Same days and same execution — only the decision-maker changes.

RESEARCH · PAPER

10:00 bias · LLM vs rulesBTA

LOADING_

Same engine and execution over 147 days; only the 10:00 bias read changes. Claude finished below “always short”.

Cumulative P&L by bias read

LOADING_

Rule-based vs Claude vs a naive baseline, same days.

Experiment design

Collector pulls minute-level gamma frames within API credit limits
Claude reads text “ladders” at fixed checkpoints, blind to outcomes
- Restricted toolset; follows a written playbook with worked examples
A deterministic tool turns its tickets into orders

▶ Scored on 1-second futures prices against a rule-based baseline

Rules bias P&L: $8.1k
LLM bias P&L: $2.9k
LLM, disagree days: −$271
Test days: 147
Blind agent-days: 184
Blind-run PF: 2.45

* LLM vs rules: 147 days, same engine and execution, 4 MNQ. Blind runs: 184 agent-days.

Overview

Experiment: a Claude agent reads text renderings of intraday gamma maps at fixed checkpoints, blind to outcomes, and writes trade tickets; a deterministic tool sets price, size and stops. To isolate the LLM’s contribution, I swapped only the 10:00 bias read inside the IB Agent engine and kept everything else fixed.

Data layer: a credit-aware collector stores the maps as raw JSON plus derived Parquet minute tables, with each expiry netting kept as a separate view so they can’t be mixed up. The agent has paper-traded every weekday since October 2026.

Highlights

Result: over 147 days the rule-based bias made $8.1k and Claude’s $2.9k — below simply always shorting ($3.3k). On the 76 days where they disagreed, the LLM lost $271 while the rules made $4.9k.
Small-sample caution: the blind runs looked promising (PF 2.45 over 184 agent-days) but rested on two setups and very few fresh days. A good-looking profit factor on a thin sample proves little.
Design: the LLM only proposes; deterministic code owns price, size and risk.
A mechanical minute-bar backtester (decide at t, fill at the t+1 open, stop-first on ambiguous bars) with cached candidates and ablation variants; 362 tests.

Notes & caveats

Verdict: the LLM stays a research assistant, not a decision-maker.

Stack

PythonpandasPyArrowClaude (Opus)matplotlibpytestlaunchd