[object Object]

← back to Ticket System

Process open tickets in board order with guarded worker input

6ef59fc3dd3be1717e0c8bb23a5ec32c6a7686ab · 2026-09-05 00:21:41 -0700 · Steve Abrams

Files touched

Diff

commit 6ef59fc3dd3be1717e0c8bb23a5ec32c6a7686ab
Author: Steve Abrams <steve@designerwallcoverings.com>
Date:   Sat Sep 5 00:21:41 2026 -0700

    Process open tickets in board order with guarded worker input
---
 CODEX-YOLOFOREVER.md                   |  11 ++++
 scripts/codex-yoloforever-prompt.md    |   8 +--
 scripts/codex-yoloforever.sh           |  39 +++++++++++-
 scripts/ordered-open-tickets.js        |  51 +++++++++++++++
 test/ordered-open-tickets.test.js      | 109 +++++++++++++++++++++++++++++++++
 test/yoloforever-preflight-boundary.sh |   1 +
 6 files changed, 214 insertions(+), 5 deletions(-)

diff --git a/CODEX-YOLOFOREVER.md b/CODEX-YOLOFOREVER.md
index ce036fa4..aac10ac6 100644
--- a/CODEX-YOLOFOREVER.md
+++ b/CODEX-YOLOFOREVER.md
@@ -15,3 +15,14 @@ tail -f codex-yoloforever.log
 If a machine crash leaves the atomic lock behind, `unlock` removes it only when launchd does not report the job running.
 
 The prompt requires a fresh canonical ticket snapshot, DTD selection, canonical-owner execution, Cody's adversarial gate, final DTD, empirical E2E proof, ticket logging, and a JSONL loop ledger. It preserves all normal approval gates. The only carried production approval is the exact TK-10993 scope recorded in the prompt; Track C and E retain their additional proof/order constraints.
+
+Each cycle captures every open task from the live board API in ascending board-rank order,
+using `scripts/ordered-open-tickets.js`. The runner passes that snapshot directly to Codex
+and retains it at `data/codex-yoloforever/open-queue.json`. If the board read fails or a task
+has no valid rank, no worker starts and the next scheduled interval retries.
+
+The worker reviews the complete ordered queue and executes up to six safe increments sequentially,
+using cabinet agents and applicable skills. Gates and owner collisions receive an explicit
+disposition and do not use execution slots. Independent source changes can use isolated worktrees
+when unrelated runtime files are dirty. Source tickets carry completion/blocker evidence;
+coordination-only passes use standing TK-11287 rather than creating monitoring tickets.
diff --git a/scripts/codex-yoloforever-prompt.md b/scripts/codex-yoloforever-prompt.md
index 79b0a362..1358edfc 100644
--- a/scripts/codex-yoloforever-prompt.md
+++ b/scripts/codex-yoloforever-prompt.md
@@ -6,13 +6,13 @@ This cycle has a maximum of six execution iterations and must terminate cleanly
 
 SANDBOX: Your Codex environment REJECTS `rm`, `rm -rf`, and `rm -f` ("rm -f style commands are not permitted"). Never emit them — a rejected command fails the whole cycle. For any temp/DTD working directory use a fresh unique path via `mktemp -d` (e.g. `DTD_DIR=$(mktemp -d)`) and do NOT delete it; the OS reclaims `/private/tmp`. Never build a command whose cleanup step is `rm`.
 
-1. Set `TK_AGENT=codex-yoloforever`. Read `tk inbox`, then take a fresh ticketmaster snapshot from the canonical `~/.claude/tickets/events.jsonl`. Do not trust a prior ranking.
-2. Exclude work owned by an active agent/session, work that collides with a dirty worktree, and any item whose next step is gated. Select the highest-value safe, reversible, non-colliding open item. Take or resume its canonical ticket and continuously log material actions and correlation evidence there.
-3. Run DTD to choose the exact bounded increment. Then route execution through the canonical owner. Delegation never widens authority; follow the A2A contract and independently verify any handoff.
+1. Set `TK_AGENT=codex-yoloforever`. Read `tk inbox`, then use the FRESH OPEN-TICKET QUEUE appended by the runner. It is a read-only snapshot of the canonical board, sorted by board rank ascending. Review ALL queue entries in that order, including low-priority entries. Re-read the canonical `~/.claude/tickets/events.jsonl` ticket and check owner/activity before every claim; the appended order is not authorization. Ticketmaster supplies ownership/urgency evidence, but must not silently replace the supplied order. Open non-task designations are monitored, not launched as new workers.
+2. For each entry, record one concrete disposition: execute-local, prepared-gated, owner-active, external-blocked, no-safe-increment, or deferred-execution-cap. Work on the first safe item before advancing to the next. Gates, collisions and read-only assessment do NOT consume the six execution slots and do NOT count as implementation progress. Prepare still-missing concrete gated work locally; otherwise record the exact blocker and continue. Never edit another active owner's surface. An unrelated dirty runtime file alone is not a reason to abandon an independent code fix: inspect ownership and use an isolated worktree/check-out where safe, preserving other changes. Take or resume the source ticket and log material actions, artifacts and correlation evidence there. Persist the complete ordered disposition list, including untouched items deferred by the execution cap, with this cycle's evidence. Do not repeatedly debate an unchanged known blocker.
+3. Run zero-cost DTD to choose each exact bounded increment. Then route execution through the appropriate cabinet owner AGENT, using the applicable SKILLS and a scoped A2A packet. Execute tickets sequentially in queue order; subagents may handle independent subtasks within that ticket. Delegation never widens authority; independently verify every handoff and require the child to log evidence on its source ticket. Leave done only after real E2E proof, or blocked with the precise external dependency/gate. Never claim a ticket closed because local prep is finished when its requested customer/operational outcome is still unverified.
 4. Invoke Cody the Contrarian on the actual result. Reproduce each claimed defect before promoting it.
 5. Run a final DTD forced choice: SHIP, FIX-THEN-SHIP, HOLD-FOR-STEVE, or KILL. Apply only reproduced, safe, reversible fixes.
 6. End with empirical E2E proof appropriate to the changed boundary. Web/UI work requires CTA plus screenrecord as `/yoloforever` specifies; non-UI work requires the real canary/API/test flow. Write reusable evidence and log it to the ticket and loop ledger.
-7. Append a concise cycle record to `/Users/macstudio3/Projects/ticket-system/data/codex-yoloforever/ledger.jsonl`: timestamp, ticket, action, DTD verdict, Cody verdict/top fix, final DTD verdict, tests/evidence, next seed, gated memos, cost. Use a safe file append mechanism. Then exit; launchd supplies the next cycle.
+7. Append a concise cycle record to `/Users/macstudio3/Projects/ticket-system/data/codex-yoloforever/ledger.jsonl`: timestamp, source tickets and ordered dispositions, actual implementation progress separately from monitoring, action, DTD verdict, Cody verdict/top fix, final DTD verdict, tests/evidence, next seed, gated memos, cost. Use a safe file append mechanism. Coordination-only notes belong on standing loop ticket TK-11287-drive-open-tickets-in-board-order-using; DO NOT create a new ticket for each unchanged monitoring pass. A newly discovered distinct defect may get its own linked ticket. Then exit; launchd supplies the next cycle with a fresh queue.
 
 HARD GATES: never perform customer-facing, destructive/irreversible, spend/money, identity, DNS/domain, publish, send-to-list, remote push, production deploy/write, canonical `dw_unified`, Shopify, or Merchant Center write unless that exact action has explicit current approval. Draft such actions to `~/.claude/yolo-queue/pending-approval/` and proceed only with a conservative safe default. Do not interpret “dangerous mode” as permission to bypass these gates.
 
diff --git a/scripts/codex-yoloforever.sh b/scripts/codex-yoloforever.sh
index bddd4b17..c9ffb3da 100755
--- a/scripts/codex-yoloforever.sh
+++ b/scripts/codex-yoloforever.sh
@@ -7,6 +7,7 @@ STATE_DIR="$ROOT/data/codex-yoloforever"
 LOCK_DIR="$STATE_DIR/cycle.lock"
 STOP_FILE="$STATE_DIR/STOPPED"
 PROMPT_FILE="$ROOT/scripts/codex-yoloforever-prompt.md"
+QUEUE_SCRIPT="$ROOT/scripts/ordered-open-tickets.js"
 LOG_FILE="$ROOT/codex-yoloforever.log"
 ERR_FILE="$ROOT/codex-yoloforever.err.log"
 LAST_MESSAGE="$STATE_DIR/last-message.md"
@@ -24,6 +25,7 @@ if [[ "${CYF_TEST_MODE:-0}" == 1 ]]; then
   LOCK_DIR="$STATE_DIR/cycle.lock"
   STOP_FILE="$STATE_DIR/STOPPED"
   PROMPT_FILE="$ROOT/scripts/codex-yoloforever-prompt.md"
+  QUEUE_SCRIPT="${CYF_TEST_QUEUE:-$ROOT/scripts/ordered-open-tickets.js}"
   LOG_FILE="$ROOT/codex-yoloforever.log"
   ERR_FILE="$ROOT/codex-yoloforever.err.log"
   LAST_MESSAGE="$STATE_DIR/last-message.md"
@@ -52,6 +54,12 @@ status() {
   [[ -f "$STATE_DIR/last-exit" ]] && print "last_exit=$(cat "$STATE_DIR/last-exit")"
 }
 
+input_failure() {
+  print "$(date -u +%FT%TZ) $1; no worker launched, retry next interval" >> "$LOG_FILE"
+  print "75" > "$STATE_DIR/last-exit"
+  date -u +%FT%TZ > "$STATE_DIR/last-finished-at"
+}
+
 run_cycle() {
   if [[ -f "$STOP_FILE" ]]; then
     print "$(date -u +%FT%TZ) stopped sentinel present; cycle skipped" >> "$LOG_FILE"
@@ -79,6 +87,35 @@ run_cycle() {
     return 78
   fi
   print "$(date -u +%FT%TZ) zero-cost DTD preflight PASS" >> "$LOG_FILE"
+  # Freeze board order for this bounded pass; a failed read must never reuse an old queue.
+  local queue_next="$STATE_DIR/open-queue.next.json"
+  if ! node "$QUEUE_SCRIPT" > "$queue_next" 2> "$STATE_DIR/open-queue-error.log"; then
+    input_failure 'open queue unavailable'
+    return 75
+  fi
+  if ! mv "$queue_next" "$STATE_DIR/open-queue.json"; then
+    input_failure 'open queue promotion failed'
+    return 75
+  fi
+  local cycle_input="$STATE_DIR/current-input.md"
+  local input_next="$STATE_DIR/current-input.next.md"
+  if ! [[ -s "$PROMPT_FILE" ]]; then
+    input_failure 'worker prompt missing or empty'
+    return 75
+  fi
+  if ! {
+    cat "$PROMPT_FILE" &&
+    printf '\n\nFRESH OPEN-TICKET QUEUE — use this order; re-read canonical state before acting:\n' &&
+    cat "$STATE_DIR/open-queue.json"
+  } > "$input_next"; then
+    input_failure 'worker input assembly failed'
+    return 75
+  fi
+  if ! mv "$input_next" "$cycle_input"; then
+    input_failure 'worker input promotion failed'
+    return 75
+  fi
+  print "$(date -u +%FT%TZ) ordered open queue captured; worker input=$cycle_input" >> "$LOG_FILE"
   DTD_ZERO_COST=1 DTD_COST_MODE_FILE="$DTD_COST_MODE_FILE" TK_AGENT=codex-yoloforever "$CODEX" exec \
     --dangerously-bypass-approvals-and-sandbox \
     --cd /Users/macstudio3 \
@@ -86,7 +123,7 @@ run_cycle() {
     --skip-git-repo-check \
     --color never \
     --output-last-message "$LAST_MESSAGE" \
-    - < "$PROMPT_FILE" >> "$LOG_FILE" 2>> "$ERR_FILE"
+    - < "$cycle_input" >> "$LOG_FILE" 2>> "$ERR_FILE"
   local rc=$?
   print "$rc" > "$STATE_DIR/last-exit"
   date -u +%FT%TZ > "$STATE_DIR/last-finished-at"
diff --git a/scripts/ordered-open-tickets.js b/scripts/ordered-open-tickets.js
new file mode 100644
index 00000000..e7534dd7
--- /dev/null
+++ b/scripts/ordered-open-tickets.js
@@ -0,0 +1,51 @@
+#!/usr/bin/env node
+'use strict';
+
+// A derived read-only queue, never a replacement for the canonical ticket ledger.
+const BOARD_URL = 'http://127.0.0.1:9794/api/tickets';
+const ID = /^TK-\d+(?:-[a-z0-9-]+)?$/;
+
+function buildQueue(records, now = new Date().toISOString()) {
+  if (!Array.isArray(records)) throw new Error('Board response must be a ticket array');
+  const seen = new Set();
+  for (const t of records) {
+    if (!t || !ID.test(t.id) || seen.has(t.id)) throw new Error('Invalid or duplicate board ticket id');
+    seen.add(t.id);
+  }
+  const open = records.filter(t => t.status === 'open');
+  const tasks = open.filter(t => (t.kind || 'task') === 'task');
+  if (tasks.some(t => !Number.isInteger(t.rank) || t.rank < 1)) {
+    throw new Error('Open task lacks a valid board rank; refusing guessed order');
+  }
+  tasks.sort((a, b) => a.rank - b.rank || a.id.localeCompare(b.id, 'en', { numeric: true }));
+  const summary = t => ({
+    id: t.id, rank: t.rank ?? null, title: t.title, project: t.project,
+    assignee: t.assignee || '', status: t.status, kind: t.kind || 'task',
+    updated_at: t.updated_at, blocker: t.blocker || null,
+  });
+  return {
+    schema_version: 1, captured_at: now, source: BOARD_URL,
+    order: 'board rank ascending; numeric ticket id breaks ties',
+    open_count: open.length, task_count: tasks.length,
+    queue: tasks.map((t, i) => ({ position: i + 1, ...summary(t) })),
+    open_non_tasks: open.filter(t => (t.kind || 'task') !== 'task').map(summary),
+    in_flight: records.filter(t => t.status === 'doing').map(summary),
+    instruction: 'Assess every queue entry in order. Gates, collisions and reads do not consume execution slots or count as implementation progress. Re-read each canonical ticket before claiming or mutating; this snapshot grants no authority.',
+  };
+}
+
+async function fetchQueue(fetchImpl = fetch) {
+  const auth = Buffer.from(process.env.TK_AUTH || 'admin:DW2024!').toString('base64');
+  const response = await fetchImpl(BOARD_URL, {
+    headers: { Authorization: 'Basic ' + auth },
+    signal: AbortSignal.timeout(20000), redirect: 'error',
+  });
+  if (!response.ok) throw new Error('Board queue read failed: HTTP ' + response.status);
+  return buildQueue(await response.json());
+}
+
+if (require.main === module) {
+  fetchQueue().then(queue => process.stdout.write(JSON.stringify(queue, null, 2) + '\n'))
+    .catch(error => { console.error('ordered-open-tickets: ' + error.message); process.exitCode = 75; });
+}
+module.exports = { buildQueue, fetchQueue };
diff --git a/test/ordered-open-tickets.test.js b/test/ordered-open-tickets.test.js
new file mode 100644
index 00000000..6524a00b
--- /dev/null
+++ b/test/ordered-open-tickets.test.js
@@ -0,0 +1,109 @@
+'use strict';
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { spawnSync } = require('node:child_process');
+const { buildQueue, fetchQueue } = require('../scripts/ordered-open-tickets.js');
+
+const ticket = (id, rank, extra = {}) => ({ id, rank, title: id, status: 'open', kind: 'task', ...extra });
+test('queue covers every open task in board-rank order with deterministic numeric ties', () => {
+  const input = [ticket('TK-20', 4), ticket('TK-3', 1), ticket('TK-10', 4),
+    ticket('TK-1', 2, { status: 'doing' }), ticket('TK-2', 3, { status: 'blocked' }),
+    ticket('TK-4', null, { kind: 'scheduled_job' }), ticket('TK-5', 5, { status: 'done' })];
+  const original = JSON.stringify(input);
+  const q = buildQueue(input, '2026-09-05T00:00:00Z');
+  assert.deepEqual(q.queue.map(t => [t.position, t.id]), [[1, 'TK-3'], [2, 'TK-10'], [3, 'TK-20']]);
+  assert.equal(q.open_count, 4);
+  assert.equal(q.task_count, 3);
+  assert.equal(q.open_non_tasks[0].id, 'TK-4');
+  assert.equal(q.in_flight[0].id, 'TK-1');
+  assert.equal(JSON.stringify(input), original);
+});
+test('queue refuses malformed data instead of guessing order or silently omitting tasks', () => {
+  for (const records of [{}, [ticket('TK-1', null)], [ticket('TK-1', -1)],
+    [ticket('TK-1', 1), ticket('TK-1', 2)], [ticket('TK-1;unsafe', 1)]]) {
+    assert.throws(() => buildQueue(records));
+  }
+  assert.deepEqual(buildQueue([]).queue, []);
+});
+test('HTTP failure never becomes an empty successful queue', async () => {
+  await assert.rejects(fetchQueue(async () => ({ ok: false, status: 401 })), /HTTP 401/);
+  await assert.rejects(fetchQueue(async () => ({ ok: true, json: async () => ({ error: 'bad response' }) })), /ticket array/);
+});
+
+function fixture({ failQueue = false, failPreflight = false } = {}) {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'tk11287-runner-'));
+  fs.mkdirSync(path.join(root, 'scripts'));
+  fs.mkdirSync(path.join(root, 'data/codex-yoloforever'), { recursive: true });
+  fs.writeFileSync(path.join(root, 'scripts/codex-yoloforever-prompt.md'), 'SCOPED TEST PROMPT\n');
+  fs.writeFileSync(path.join(root, 'scripts/ordered-open-tickets.js'), failQueue
+    ? 'process.stderr.write("queue unavailable");process.exit(75);'
+    : 'console.log(JSON.stringify({schema_version:1,queue:[{id:"TK-7",rank:1}]}));');
+  fs.writeFileSync(path.join(root, 'fake-preflight'), '#!/bin/sh\nexit ' + (failPreflight ? '78' : '0') + '\n', { mode: 0o755 });
+  fs.writeFileSync(path.join(root, 'fake-launchctl'), '#!/bin/sh\nexit 0\n', { mode: 0o755 });
+  fs.writeFileSync(path.join(root, 'fake-codex'), '#!/bin/sh\ncat > "$CYF_INPUT_EVIDENCE"\nprintf "%s" "$DTD_ZERO_COST" > "$CYF_GUARD_EVIDENCE"\n', { mode: 0o755 });
+  fs.writeFileSync(path.join(root, 'data/codex-yoloforever/open-queue.json'), '{"queue":["STALE"]}');
+  return root;
+}
+function runFixture(root, extraEnv = {}) {
+  return spawnSync('zsh', [path.join(__dirname, '../scripts/codex-yoloforever.sh'), 'run'], {
+    encoding: 'utf8', timeout: 10000,
+    env: { ...process.env, CYF_TEST_MODE: '1', CYF_TEST_ROOT: root,
+      CYF_TEST_CODEX: path.join(root, 'fake-codex'), CYF_TEST_PREFLIGHT: path.join(root, 'fake-preflight'),
+      CYF_TEST_LAUNCHCTL: path.join(root, 'fake-launchctl'),
+      CYF_INPUT_EVIDENCE: path.join(root, 'received-input'), CYF_GUARD_EVIDENCE: path.join(root, 'received-guard'), ...extraEnv },
+  });
+}
+test('runner delivers the fresh ordered queue to its consumer and releases overlap lock', () => {
+  const root = fixture();
+  const result = runFixture(root);
+  assert.equal(result.status, 0, result.stderr);
+  const input = fs.readFileSync(path.join(root, 'received-input'), 'utf8');
+  assert.match(input, /SCOPED TEST PROMPT[\s\S]*FRESH OPEN-TICKET QUEUE[\s\S]*TK-7/);
+  assert.doesNotMatch(input, /STALE/);
+  assert.equal(fs.readFileSync(path.join(root, 'received-guard'), 'utf8'), '1');
+  assert.equal(fs.existsSync(path.join(root, 'data/codex-yoloforever/cycle.lock')), false);
+});
+test('queue failure launches no consumer and leaves the scheduler free to retry', () => {
+  const root = fixture({ failQueue: true });
+  const result = runFixture(root);
+  assert.equal(result.status, 75, result.stderr);
+  assert.equal(fs.existsSync(path.join(root, 'received-input')), false);
+  assert.equal(fs.existsSync(path.join(root, 'data/codex-yoloforever/STOPPED')), false);
+  assert.equal(fs.existsSync(path.join(root, 'data/codex-yoloforever/cycle.lock')), false);
+  assert.match(fs.readFileSync(path.join(root, 'codex-yoloforever.log'), 'utf8'), /no worker launched/);
+});
+test('failed cost preflight still stops before queue or consumer work', () => {
+  const root = fixture({ failPreflight: true });
+  const result = runFixture(root);
+  assert.equal(result.status, 78, result.stderr);
+  assert.equal(fs.existsSync(path.join(root, 'received-input')), false);
+  assert.equal(fs.existsSync(path.join(root, 'data/codex-yoloforever/open-queue.next.json')), false);
+  assert.equal(fs.existsSync(path.join(root, 'data/codex-yoloforever/STOPPED')), true);
+});
+
+for (const failure of ['missing-prompt', 'empty-prompt', 'queue-promotion', 'input-write', 'input-promotion']) {
+  test(failure + ' prevents worker launch and releases the overlap lock', () => {
+    const root = fixture();
+    const state = path.join(root, 'data/codex-yoloforever');
+    const prompt = path.join(root, 'scripts/codex-yoloforever-prompt.md');
+    const env = {};
+    if (failure === 'missing-prompt') fs.renameSync(prompt, prompt + '.saved');
+    if (failure === 'empty-prompt') fs.writeFileSync(prompt, '');
+    if (failure === 'input-write') fs.mkdirSync(path.join(state, 'current-input.next.md'));
+    if (failure.endsWith('-promotion')) {
+      const bin = path.join(root, 'bin'); fs.mkdirSync(bin);
+      const failSource = failure === 'queue-promotion' ? 'open-queue.next.json' : 'current-input.next.md';
+      fs.writeFileSync(path.join(bin, 'mv'), '#!/bin/sh\ncase "$1" in *' + failSource + ') exit 1;; esac\nexec /bin/mv "$@"\n', { mode: 0o755 });
+      env.PATH = bin + ':' + process.env.PATH;
+    }
+    const result = runFixture(root, env);
+    assert.equal(result.status, 75, result.stderr);
+    assert.equal(fs.existsSync(path.join(root, 'received-input')), false);
+    assert.equal(fs.existsSync(path.join(state, 'cycle.lock')), false);
+    assert.equal(fs.existsSync(path.join(state, 'STOPPED')), false);
+    assert.match(fs.readFileSync(path.join(root, 'codex-yoloforever.log'), 'utf8'), /no worker launched/);
+  });
+}
diff --git a/test/yoloforever-preflight-boundary.sh b/test/yoloforever-preflight-boundary.sh
index 2dcd0878..0fb70eaa 100755
--- a/test/yoloforever-preflight-boundary.sh
+++ b/test/yoloforever-preflight-boundary.sh
@@ -15,6 +15,7 @@ make_root() {
   local root="$1"
   mkdir -p "$root/scripts" "$root/data/codex-yoloforever" "$root/bin"
   cp "$ROOT/scripts/codex-yoloforever-prompt.md" "$root/scripts/"
+  printf 'console.log(JSON.stringify({schema_version:1,queue:[]}));\n' > "$root/scripts/ordered-open-tickets.js"
   cat > "$root/bin/codex" <<'SHIM'
 #!/usr/bin/env bash
 printf 'runner codex invoked DTD_ZERO_COST=%s\n' "${DTD_ZERO_COST:-unset}" >> "$CYF_MARKER"

← e63d775c Record verified local trade approval repair cycle  ·  back to Ticket System  ·  Record ordered-loop launch and failure-boundary verification fcba51cf →