Durable state and read-only projections

Long-running work does not recover because a system stores more conversation. It recovers because each important fact has a stable owner and can be projected into a fresh decision. This chapter establishes the LoopX state substrate: which surfaces own facts, which surfaces only help readers, and why a page that looks like current state is not automatically a write API.

What you should learn

After this chapter, you should be able to:

• distinguish the registry, event ledger, active-state workbench, run history, and status projections;
• decide whether a field belongs to canonical state, an external authority, or a derived read model;
• explain how append-only events, replay, idempotency, and freshness relate;
• explain why a dashboard, prompt, or task graph must not become a second state machine;
• start protocol work from the authoritative contract rather than a volatile Python function name.

Goal identity does not belong to a chat thread

The durable identity in LoopX is the Goal, not a Host thread:

Goal
├── objective and boundary
├── Todos, Gates, and evidence lineage
├── registered peer identities
└── runtime and projection routes

Session / thread
└── one temporary executor context

Codex App, Codex CLI, or another Host can advance the same Goal at different times. One session can also read more than one Goal. Reading a Goal does not grant write authority, and ending a session does not delete the Goal.

Reuse an exact Goal instead of guessing from text

Goal reuse depends on a stable goal_id and registry connection, not fuzzy objective similarity:

one registered Goal
  -> reuse that exact Goal boundary

multiple registered Goals
  -> read-only goal_selection_gate
  -> choose one exact goal_id
  -> rerun before any mutation

When a project has several registered Goals, start-goal --guided should list their ids, status, and exact rerun commands. Todo writes, Agent registration, and Host activation wait until the selection is resolved. Similar objective text, a shared repository, or overlapping acceptance criteria do not authorize LoopX to merge Goal boundaries silently.

Keep Goal reuse separate from Agent takeover. A new Agent can read the same public frontier and history, but it registers a fresh agent_id by default. Reusing an existing Agent identity requires the user to select that exact id. This preserves historical lineage without letting a new session inherit execution responsibility under another identity.

Recovery is therefore not:

restore = replay the old conversation

It is:

next decision =
  replay(durable project facts)
  + inspect(fresh workspace and external facts)

An old conversation may help interpretation. It cannot outrank current Git state, an unresolved Gate, current CI, or LoopX canonical lifecycle state.

Five state surfaces

1. Registry: identity, connection, and durable policy

The registry answers “which Goal is this, where is it connected, and which runtime routes are allowed?” It can carry:

• Goal id, repository, and active-state route;
• local or global runtime roots;
• registered Agent identities;
• coordination, write scope, and workspace guards;
• configuration for default-off features.

The registry does not prove that a Host successfully started. It also does not store every Agent result. It owns connection and policy facts, not execution receipts.

2. Event ledger: what happened

event_sourced_state_contract_v0 represents Todo, Gate, run, evidence, projection, and quota changes as append-only events.

At least four invariants matter:

Invariant	Why it matters
Append-only	New facts extend history instead of rewriting it to hide an earlier action
Ordered	Replay reconstructs the same lifecycle sequence
Idempotent	Replaying the same event_id with the same payload does not duplicate the effect
Privacy-partitioned	Public-safe summaries do not mix with local or private payloads

Marking a Markdown checkbox is not sufficient proof that a Todo completed. A legal transition should keep the Todo id, producer, completion evidence, time, and event lineage so status, review packets, and the next quota decision can reuse the same fact.

3. Active-state workbench: a human-readable work surface

ACTIVE_GOAL_STATE.md helps humans and Agents read the Objective, Next Action, User Todos, Agent Todos, and Progress. It is an important workbench, but “all truth lives in Markdown” is the wrong model.

During migration or compatibility windows, Markdown may still participate in Todo reads. Canonical writes should still pass through LoopX lifecycle commands and controlled writeback so they form governed events. Editing a projected paragraph does not automatically perform a lifecycle transition.

active_state_structured_projection_v0 defines a typed, read-only view of Todos, Gates, and Next Action from that workbench. The contract preserves several boundaries:

• a projection can be recomputed;
• a projection grants no write authority;
• a generated compatibility id is not automatically a migration-ready canonical id;
• duplicate ids or missing sections become diagnostics instead of disappearing silently.

4. Run history: what one bounded turn observed and delivered

Run history keeps a compact index for one bounded turn, such as:

• participating Agent, Todo, and Goal;
• whether the run was observation, delivery, or blocker work;
• validation and evidence references;
• delivery scale and outcome;
• successor, replan, or no-follow-up;
• whether spend conditions were satisfied.

A run snapshot is not complete project memory. It answers what this turn saw, attempted, proved, and wrote back. Goal lifecycle still depends on the combination of Todos, Gates, events, evidence, and acceptance.

Rich logs, raw transcripts, and verifier tails can stay in local or private runtime artifacts. A public projection should retain only the bounded references required for review and recovery.

5. Status and other projections: how a consumer reads now

loopx status, quota should-run, dashboards, review packets, and task graphs are different read models for different consumers.

They may:

• aggregate several source facts;
• compress large payloads;
• reorganize state for a user, Agent, CLI, or operator;
• surface staleness, gaps, repair needs, and attention signals.

They must not:

• invent a Todo absent from the source;
• turn display order into lifecycle priority;
• let card or graph edits bypass the write API;
• treat a stale external observation as a current fact.

task_graph_projection_v0 is explicit about this boundary. Relationships such as blocks, validates, continues, and hands_off_to are derived graph edges, not new scheduling commands.

Canonical state, workbench, projection, and external fact

Keep these four categories separate:

Layer	Typical content	Who can change it	Can it directly authorize a transition?
Canonical state	Events, typed Todos, Gate resolution, quota spend	LoopX lifecycle writer	Yes
Workbench	Active-state Markdown and human explanation	Controlled writeback or compatibility editing	Only after normalization into governed facts
Projection	Status, quota packet, dashboard, task graph	Projection builder	No; it informs a decision
External fact	Git commit, PR, CI result, cloud resource	The corresponding external system	Only through fresh readback and evidence

A page saying “the PR is merged” may be stale. A previous run saying “tests passed” may refer to an older commit. Inspect the external system and verify revision, freshness, and scope before using that observation for a current transition.

Storage medium is not the authority contract

The current LoopX control plane is local-first: the project registry, active-state workbench, event and run history, and runtime state live in project-local or user-local storage. This does not make a Markdown file the authority by itself, and replacing files with a database does not automatically create correct concurrency or recovery semantics.

event_sourced_state_contract_v0 allows JSONL, SQLite, or another local-first append-only implementation when it preserves:

• stable event ids and ordered replay;
• idempotent append;
• projection head alignment with the event-store head;
• public-safe, local-private, and private-pointer partitions;
• Markdown as a workbench or projection rather than an arbitrary write API.

local_state_write_correctness_v0 is currently marked as a public-safe protocol draft. Its stronger write-correctness target separates prepare -> preview -> apply -> record -> project:

• one idempotency_key should not duplicate the logical effect;
• an expected_revision mismatch should fail closed or recompute a non-overlapping patch from fresh state;
• a foreign or expired lease should never be silently cleared;
• the default target boundary is one Goal, narrowed only for a single order-independent Todo write;
• external writes, credentials, production, and private reads remain behind independent Gates.

Current Todo lifecycle commands reread and write under the active-state file lock, and preview exposes a write intent. The protocol also states that hard idempotency, uniform optimistic CAS, and lease-conflict enforcement are promoted writer by writer. Do not assume that every writer already enforces the complete Draft.

Files, SQLite, and future providers answer “where are the bytes?” Events, revisions, CAS, leases, and authority answer “which transition is legal?”

Shipped boundary versus design boundary

The current public architecture keeps the CLI as the compatibility baseline and describes a local server/daemon as a roadmap. Detailed multi-Host authority, offline queue, and shared-control-plane designs live in an RFC whose status is Draft. They are not installed cloud features.

You can currently rely on:

• local project state and the global registry projection;
• the Todo lifecycle writer's active-state file lock, preview/readback, and currently implemented idempotency behavior;
• registered peers, soft claims, optional task leases, and independent-worktree guards;
• several Hosts reading one registry and Goal through controlled writeback.

Do not currently promise:

• automatic online authority shared across devices;
• new claims, completion, lease renewal, or protected writes while a device is offline;
• consistent distributed state from putting the project directory in a sync drive;
• NoKV, a database, or IM automatically replacing the LoopX lifecycle owner.

A future cross-device control plane should retain one canonical LoopX authority, revision-bound idempotent commands and receipts, and separate message delivery, context memory, and state authority. Until the Draft is promoted and validated, this book teaches those boundaries rather than a fictional cloud-mode quickstart.

Three layers of integrity for historical artifacts

LoopX can prevent research, validation, and decision artifacts from being rewritten silently. That does not make an old conclusion perpetually applicable. Evaluate historical evidence in three layers:

Layer	Question	Typical checks
Lineage integrity	Who produced the artifact, when, and was it appended, corrected, or superseded?	event_id, run_id, producer, recorded revision, append-only references
Current applicability	Do its inputs, scope, and external facts still match the current problem?	Commit, target key, source revision, time window, Gate scope, fresh readback
Supersession	Did later evidence or a decision replace, narrow, or revoke it?	supersedes, superseded_by, compensating event, newer decision, replan delta

Append-only lineage protects history from silent alteration; it does not prove that an old conclusion is fresh. Research notes, test results, and PR readbacks need stable join keys and a new applicability check after material inputs change. When applicability is unknown, retain the artifact as a historical observation or stale evidence instead of deleting it or treating it as current authority.

agent_scoped_evidence_ledger_v0 provides a bounded, read-only Agent chronology for replan and handoff. It does not replace current status, a quota decision, or external-system readback.

Replay does not preserve an old conclusion forever

Assume the ledger contains:

todo_added(T1)
todo_claimed(T1, agent-a)
gate_added(G1, scope=public_claim:action:homepage)
run_recorded(R1, tests_passed_at=commit-a)

Git later advances to commit-b, and the user changes the homepage direction. Replay still proves that R1 and G1 existed. It does not prove:

• R1 remains valid for commit-b;
• G1 covers the revised homepage action;
• agent-a is still executing in the correct workspace;
• the current frontier is ready to publish.

The recovering executor combines replayed project facts with a fresh environment inspection.

A projection gap is a control-plane failure

Do not choose whichever surface is most convenient when sources disagree:

• an event contains an open Todo, but status omits it;
• a Gate is resolved, but quota still reports operator wait;
• active state has a Next Action with no corresponding Todo;
• a dashboard reports runnable work while the workspace guard points to another worktree.

These are projection gaps. Handle them in order:

1. identify the authoritative source;
2. classify source-write failure, stale projection, migration drift, or stale external observation;
3. repair through the owning lifecycle or writeback path;
4. recompute the projection and verify its source revision;
5. do not run delivery that depends on the disputed state until it is consistent.

Manually editing several displays into agreement only hides the fault.

Decide where a new field belongs

Ask in this order:

1. Does it describe durable identity, configuration, or routing? Put it in the registry.
2. Does it describe a lifecycle transition? Put it in an event.
3. Does it describe one observation or delivery turn? Put it in a run snapshot or evidence bundle.
4. Does it serve only one reader view? Derive it as a projection from existing facts.
5. Does GitHub, CI, or another system own it? Keep that external authority and store bounded readback.
6. Is it a domain-specific result for Issue-Fix, Explore, or another pack? Keep it in Domain State rather than forcing it into generic Todo or quota state.

If one field tries to carry configuration, event, display, and permission semantics, the protocol boundary probably needs to be split first.

Protocol reading routes

This chapter owns the learning sequence, not the complete schemas. For state changes, start with:

• event_sourced_state_contract_v0 for events, replay, ordering, and privacy;
• active_state_structured_projection_v0 for the typed read model over the Markdown workbench;
• task_graph_projection_v0 for the read-only relation graph;
• long_horizon_agent_state_protocol_v0 for source and projection ownership, concurrent Agents, and lifecycle in long-running work;
• agent_scoped_evidence_ledger_v0 for the Agent-scoped chronological read model used before replan and handoff;
• the Status Data Contract for Agent and operator-facing aggregation.

The next chapter builds a work graph on this substrate: who may do what, which condition blocks it, and how work legally continues, hands off, or ends.