ADR 0002: Child flow instances run on the engine's own subworkflow node (when they run at all)

• Status: proposed (spike complete; awaiting owner acceptance — this ADR is the gate for #471)
• Date: 2026-08-14
• Deciders: Hierarchical Supervisor Workflow milestone (#466); spike + ADR issue #470
• Design of record: Hierarchical-supervisor workflow model (§3 options, §4 phases, Phase 1 gate criteria G1–G3)
• Spike evidence: spikes/issue-470/ — four throwaway prototypes, all executed 2026-08-14

Context

A Bloom project is one workflow run with one current_node pointer, parked on await_project_event for the whole steady state, while the real concurrent work lives in per-entity state machines inside run.state: engine_runs (per-issue, EngineRunState: ready → in_progress → pr_opened ⇄ changes_requested → done plus blocked/failed/needs_attention) and specialist_reviews (per-PR review rounds), alongside the pending plan-change/decision/approval/lifecycle gates.

Phase 0 of the design has shipped: the child-flow projection (services/project_flows.py, #467) derives typed ChildFlow records from that persisted state, project_detail serves them (#468), and the web swimlane renders them (#469). No runtime machinery was added; children are inferred, not executed.

Phase 2 — TRUE child flow instances, where a child's transitions are effected by a runtime rather than inferred — is deliberately gated (design §4 Phase 1). This ADR settles which runtime backs Phase 2 if and when the gate opens, so #471 is a build decision, not a research project. Relevant background: the spec schema has always reserved a subworkflow node type (workflows/schema/workflow.schema.json, subworkflowConfig), and the built-in engine explicitly refuses it (workflow/engine.py:164 — NotImplementedError). The architecture anticipated hierarchical composition; it was never wired.

The gate criteria (from the design note, Phase 1)

Enter Phase 2 only if at least one holds after Phase 0 has been in production:

• G1 — inference is lying. The projection demonstrably mis-states child position (label truth vs engine_runs truth diverge in ways users see) and reconciliation-by-reading cannot fix it.
• G2 — control flow needs to move. A product requirement needs per-child suspend/resume the orchestrator's imperative paths cannot express safely (pausing one ticket's rework loop while others proceed; owner-driven per-ticket gates).
• G3 — the imperative loop is a defect factory. The review→rework→escalate paths keep producing regressions of the #288/#457/#455 shape, arguing for a declared, testable child graph driving them.

Issue #470 reads these as three lenses on a runtime: continuity (resume one child without disturbing siblings or the parked supervisor), durability (survive restart; idempotent event redelivery), and operational fit (dependencies, persistence formats, servers, migration). Each option below was spiked against the same ticket child lifecycle and scored on those lenses plus lock-in, new-dependency cost, and how directly it maps onto the already-concurrent engine_runs/specialist_reviews machines.

Options considered

A. Bloom's native subworkflow node — chosen

Wire the reserved node: a child flow instance is an ordinary RunState row in bloom_runs, keyed {thread_id}:ticket:{n} (precedent: the chat index already rides in a non-project RunState row), running a child spec written in the same JSON dialect and validated by the same loader as bloom-sdlc.

Spike A (spike_a_native_subworkflow.py) wired it without touching the engine core (an Engine subclass, ~60 lines) and demonstrated, executed:

• Continuity: two children spawned by one subworkflow node, each suspended independently on event_wait; resuming one (rework loop, round caps as spec data) never touched the sibling or the parked supervisor.
• Durability: every step round-tripped through the run's JSON serialization — the exact bloom_runs JSONB row — and the child's own processed_keys made webhook redelivery a no-op. Same fence as the parent, narrower scope: redelivery can no longer interleave across siblings through the shared parent row.
• Operational fit: zero new dependencies, zero new persistence formats, no new failure domain. Child advancement rides the existing durable job queue.
• Mapping: the child's current_node maps 1:1 onto EngineRunState — the child spec is a transcription of the enum, not an invention — so the shipped #467 projection doubles as the shadow-mode parity check for rollout, and specialist_reviews rounds map the same way onto a bloom-pr-review child.
• Lock-in: none; the interpreter is ~190 lines Bloom owns, and the spec dialect was designed to compile to other engines.

Cost honestly stated: suspend-at-arbitrary-event_wait, per-child job scheduling, and parent/child lock ordering (child lock only during child advance; parent effects through the existing thread lock, never nested the other way — design §8 O5) are ours to build and test. The spike shows they are small; they are not zero.

B. LangGraph subgraphs — rejected as default; retained as fallback candidate

Already a dependency (langgraph>=0.2, the #401 LangGraphCoordinator spike behind BLOOM_COORDINATION_ENGINE). Spike B built the ticket child as a StateGraph with interrupt() + checkpointer and found, executed:

• Per-child suspend/resume and durability are real — but each independently-suspending child needs its own thread_id, namespaced exactly like spike A's row keys: the supervisor/child split gets hand-built either way, on a heavier substrate.
• The checkpoint lineage is a second durable format beside bloom_runs + the event store (5 checkpoints for one child's short life in the spike), with its own growth/retention story.
• A mid-flight duplicate Command(resume=...) was consumed as a fresh verdict (one logical review event moved state twice) — the processed_keys-style fence still has to be built around the graph.
• The labels-as-truth impedance mismatch (docs/design/langgraph-sketch.md §7) is unchanged: the projection remains necessary; the runtime does not replace it.

Net: LangGraph adds a load-bearing dependency without removing any of the work option A needs. It stays what it is today — a candidate implementation behind the same Engine seam, useful if the built-in interpreter ever hits a genuine capability wall (e.g. native parallel super-steps).

C. Temporal parent/child workflows + signals — rejected

Spike C ran against a real local Temporal dev server (uv run --with temporalio; the SDK downloads a server binary — already the tell). Executed: a parent workflow spawned detached children (ParentClosePolicy.ABANDON), signals drove one child through rework-to-done and its sibling to escalation, continuity and durability impeccable.

The primitives fit the problem best-in-class — and that is precisely the problem: they replace, wholesale, the durable machinery Bloom already built and tests — the job queue with atomic claim (persistence/jobs.py), the processed_keys/event-key fences, the scheduler heartbeat, waiting_on suspend/resume. Adoption means rewriting ~5k lines of orchestrator effect paths into activities under determinism constraints, dual-running during cutover, re-founding the test suite, and operating a server cluster (or paying for cloud) beside today's single-Postgres deployment — against the same "$0 constraint" posture ADR 0001 records. Maximum lock-in, maximum new-dependency cost, for semantics option A gets from rows we already persist.

D. Actor model — rejected

No mature asyncio-native actor framework exists to adopt (Pykka is thread-based, Ray is a cluster runtime), so per "search before you build" spike D hand-rolled the minimal mailbox-per-ticket + supervisor — which is itself the finding: choosing actors means owning bespoke actor infrastructure. Executed: an actor crash lost the in-flight event (supervision restarted a blank actor), and queued mailboxes are process memory a restart drops — closing those holes means rebuilding the durable job queue and bloom_runs rows Bloom already has. Actors model who acts; the problem is representing work items' lifecycles, which are state machines. The view would still be a projection, over state even less inspectable than today's dicts.

Scorecard

Higher is better for Bloom specifically. G-columns score how well the runtime would serve the need if that gate criterion forces Phase 2.

	G1: authoritative truth + parity	G2: per-child suspend/resume	G3: declared, testable child graph	Continuity	Durability	Ops fit	Lock-in (5 = none)	New deps (5 = none)	engine_runs/specialist_reviews mapping
A. native subworkflow	5 (parity vs #467 is 1:1)	4	5 (same dialect, same loader)	5	5	5	5	5	5 (spec = enum transcription)
B. LangGraph subgraphs	3 (parity across two formats)	4	4	5	4	3	3	3	3 (state re-modeled per graph)
C. Temporal	3 (truth moves server-side)	5	4	5	5	1	1	1	2 (machines become workflow code)
D. Actor model	2	3	2 (loops, not declared graphs)	4	1	2	2	2 (bespoke build)	2

Decision

If and when the G1–G3 gate opens, TRUE child flow instances run on the built-in engine's own subworkflow node — option A. A child is an ordinary RunState row in bloom_runs under a namespaced key, running a declared child spec (bloom-ticket first, because EngineRunState already is its state machine), advanced by the existing job and webhook handlers, fenced by its own processed_keys. Rollout is shadow-mode: the child run advances beside the untouched engine_runs writes with a parity check against the #467 projection, and becomes authoritative only when parity holds in production (design §4 Phase 2). LangGraph remains the named fallback behind the same Engine seam if the interpreter hits a real capability wall; Temporal and actors are rejected outright.

Rationale in one sentence: Bloom's scarce property was never durable concurrent execution (the job queue + fences provide it) but an honest entity-granular representation of it — Phase 0 shipped that — so the only defensible runtime is the one that adds executed children at near-zero marginal machinery, zero new dependencies, and a parity check the projection already provides.

Gate status for #471 — explicit

As of 2026-08-14: the gate is NOT passed. #471 stays blocked.

• G1 (inference is lying): no evidence. The projection shipped this week (#467–#469) and surfaces label-vs-run-state divergence as visible disagreement rather than silent error; no user-visible mis-statement has been observed, and none has survived reconciliation-by-reading.
• G2 (control flow needs to move): no product requirement on the books needs per-child suspend/resume the orchestrator cannot express (no per-ticket pause, owner-driven per-ticket gates, or similar has been requested).
• G3 (defect factory): the #288/#457/#455 regression class predates Phase 0; watch whether it recurs now that divergence is rendered. No post-Phase-0 recurrence yet — the observation window has barely opened.

Evidence that would open the gate, and where to look: recurring disagreement badges in the swimlane that reconciliation cannot clear (G1); a feature request for per-ticket pause/resume or per-ticket owner gates (G2); another review-loop regression of the known shape (G3). Any one criterion, plus the owner's explicit greenlight recorded on #471, is sufficient — the runtime question is settled here, so #471 begins at implementation, not evaluation.

Consequences

• #471, when greenlit, implements the ticket child on the built-in engine behind a feature flag, shadow-mode first, parity-tested against services/project_flows.py; no runtime evaluation remains in its scope.
• The NotImplementedError at workflow/engine.py:164 is the designated insertion point; until #471 is greenlit it stays exactly as it is — no engine-core change ships from this spike.
• Child specs as data (bloom-ticket.workflow.json etc., design §4 Phase 0 step 1) are worth landing ahead of the gate: they cost what documentation costs, load through the existing validator, and give the projection's enum-exhaustive contract tests a target.
• LangGraph stays a dependency at today's status (optional coordinator behind BLOOM_COORDINATION_ENGINE); this ADR adds no new load-bearing use.
• No Temporal server, no actor framework, and no bespoke actor infrastructure enter the stack; temporalio was installed only ephemerally for the spike and is not a dependency.
• spikes/issue-470/ is throwaway by declaration: nothing imports it, and it may be deleted once this ADR is accepted without loss (the executed findings are recorded here).

Revisit when any of these change the calculus: (a) a gate criterion is met with evidence contradicting the scorecard (e.g. G2 arrives needing parallel super-steps inside one child, where LangGraph's engine is genuinely stronger than the interpreter), (b) the built-in engine grows obligations that make owning it more expensive than adopting a framework, or (c) Bloom's deployment posture changes such that operating a workflow server stops being a new failure domain (which would reopen Temporal's ops score, not its migration score).