A multi-agent orchestrator in raw Python — one agent loop, typed inter-agent messages, hard budget reservations, and runs you can replay event by event. No LangChain, no LangGraph, no CrewAI, no vendor SDK. Every abstraction is one I designed and can defend.
You give it a task in plain language. A planner decomposes it into a dependency graph of subtasks. Workers execute the independent ones concurrently, each with its own restricted toolset. A critic reviews every result against that subtask's own success criteria and sends work back when it falls short. The whole run is bounded by a hard budget and recorded as a trace you can step through afterwards.
Emits a task DAG. Rejected and repaired in-loop if it has a cycle, an orphan node, a dangling dependency, or a success criterion no reviewer could actually check.
researcher · writer · coder. Profiles are tool-name lists, not subclasses. Each sees only its own subtask brief — never the whole run history.
Sees one node's criteria and artifacts, nothing else. Structurally cannot return APPROVE while any criterion is marked unmet.
This is not a re-enactment — it is the actual trace.jsonl from a
10-node run, played back in your browser. Every state transition, tool call, critic verdict,
compaction and dollar is in the file. If the replay can't explain why the system did something,
the trace has a hole in it.
scripts/make_demos.py: one frame per trace
event → headless Chromium screenshots → VP8/WebM. No screen recording.Scored by a mechanical rubric — never an LLM judge. A model grading a model drifts with the model and would make the eval as unreliable as the thing it measures. Every check is a predicate over the run state, the workspace and the trace.
A deterministic backend scores 100% with zero variance, which on its own proves nothing. So each ablation removes exactly one guard and the rubric has to notice. All five are detected. A guard whose removal costs no score is either untested or not earning its place.
llm/ — provider-agnostic client
client.py · error taxonomy
anthropic_adapter.py · raw httpx
openai_adapter.py · same protocol
retry.py · backoff + jitter
budget.py · reserve / settle
agents/
base.py · the one agent loop
planner.py · worker.py · critic.py
tools/ registry · files · python_exec
web_search · web_fetch · memory_search
memory/ shortterm compaction · sqlite FTS5
orchestrator/ engine.py · recovery.py
obs/ tracer.py · timeline.py · svg.py
evals/ tasks · harness · ablations
agents/base.py.
Roles differ only by system prompt, allowed tools, and output schema.mypy --strict is clean.AnthropicAdapter wire format + error class
↓
RetryingClient backoff, Retry-After, abort hook
↓
BudgetedClient reserve → call → settleBudget sits outside retry so one reservation covers the whole retry saga. Flip the order and a transient 429 becomes a hard budget failure — a different failure class, routed to a different recovery.
submit_result toolRejected: structured outputs. They guarantee valid JSON but are provider-specific, constrain the whole response in a loop that also needs tool calls, and move the "I'm done" signal out of the tool trace. Lock-in decided it — the finish contract must survive a provider swap.
With N concurrent agents, N can each
see "budget remains" and collectively overshoot by N×. reserve() charges the
worst case before dispatch. A test starts three agents against a cap that fits one and proves
exactly one reaches the model.
The critic sees only a node's criteria and artifacts, so "sources are high-quality" makes it rubber-stamp or revise forever. Prompt guidance alone didn't hold, so the validator rejects soft words with nothing countable alongside — and says how to rewrite.
A run produces ~100 facts, and query and documents share a vocabulary, so the mismatch embeddings solve is largely absent. A vector store adds an embedding call per fact and makes retrieval non-deterministic — which would break eval comparability.
Needed three things its callback model
makes awkward: honour Retry-After, emit one trace event per attempt, and consult
an abort hook so a budget breach stops a retry storm mid-sequence. ~40 lines I can defend line
by line.
Events stay small enough to stream
into the live view; full prompts go to blobs/<sha256>. The system prompt
repeated across 82 calls is stored once. A resumed run continues the same sequence rather than
restarting it.
ESCALATE verdict.
Running out of money looked like "this subtask can't succeed as specified", so the run replanned,
spent more, and reported FAILED instead of BUDGET_EXCEEDED. An
affordability problem must never be reported as a quality problem.memory or ShortTermMemory(...) silently discarded an injected memory,
because ShortTermMemory defines __len__ and an empty one is falsy.
Compaction never fired and nothing failed.python_exec put elapsed milliseconds into the model-visible summary.
That made runs non-reproducible and invalidated the prompt-cache prefix on every single call.
Timing belongs in the trace, not the prompt.python_exec is a resource guard, not a security boundary. Timeout and
process-group kill are solid; RLIMIT_AS is best-effort on macOS and there is
no network isolation. The environment is scrubbed so API keys never reach the child.--resume started alongside a live
run enforces the cap twice. Fixing it needs the reservation table in SQLite, not a lock.max_tokens of output, so
budgets under ~$0.25 refuse to plan at all. Deliberate direction, surprising in practice.cache_read_input_tokens
has ever been observed, because no live call has been made.Full versions in
docs/limitations.md, docs/security.md, docs/memory.md and
docs/failure-policy.md.
git clone https://github.com/Gariyuuu/orchestrion && cd orchestrion
uv venv --python 3.11 && uv pip install -e ".[dev]"
.venv/bin/python run.py --showcase deep # 10-node run, simulated backend
.venv/bin/python replay.py runs/<run_id> # step through what happened
export ANTHROPIC_API_KEY=sk-ant-... # the real thing
.venv/bin/python run.py "Research X and write a sourced report."213 tests, mypy --strict clean,
~5.6k lines of library code and ~3.1k lines of tests.