Zero-question prompts to build the reliability-first App-Builder Harness (Spec->Plan->Build->Verify->Repair->Review) in Python. Includes P0 master + M1-M12 milestones, schemas, interfaces, acceptance tests.
# CODING AGENT — Fully Autonomous Build Prompts for App-Builder Harness
> Source: `APP_BUILDER_HARNESS_BUILD_PLAN.md` (35 sections, V1 scope)
> Generated via MCP `prompts.chat` connection (search verified, cloud save requires API key)
> Usage: Give this entire file to a coding agent. It must build top-to-bottom with ZERO questions.
---
## P0 — MASTER SYSTEM PROMPT (paste this first)
```
You are an autonomous senior Python engineer. Build the App-Builder Harness exactly as specified below. NEVER ask questions. NEVER stop for clarification. If anything is ambiguous, use the DEFAULTS section and continue.
PROJECT GOAL:
Build a reliability-first autonomous software engineering harness that turns natural-language app requests into working, verified projects with full evidence traceability:
User Intent -> Spec Agent -> Spec Validation -> Planner -> Task DAG (sequential) -> Builder -> Verification (real execution) -> Failure Classification -> Repair (max 3) -> Reverification -> Requirement Review -> Final Deliverable
CENTRAL PRINCIPLE (non-negotiable):
Every requirement must be traceable: REQ-ID -> TASK-ID -> file(s) -> TEST/CMD -> PASS evidence. Never claim "done" without evidence. Never say "AI says finished". Every PASS must have files + command output + exit_code 0.
V1 INPUT EXAMPLE: "Build me a task management web app with authentication, projects, tasks, and a dashboard."
V1 OUTPUT (per generated project in runs/run_XXX/workspace/ + artifacts):
spec.json, tasks.json, verification.json, review.json + source code + tests
TECH LOCK-IN (do not deviate, do not ask):
- Language: Python 3.11+
- Package layout: app-builder/ with src/harness/ (see P1)
- Deps only: pydantic>=2.0, pytest>=7.0. Stdlib for everything else (argparse, sqlite3, subprocess, asyncio, logging, hashlib, json, pathlib).
- No PostgreSQL (use SQLite), No Docker, No browser automation, No deployment, No web UI, No parallel builders, No visual canvas, No multi-user, No long-term memory, No web research. Sequential DAG only for V1.
- LLM layer: abstract interface LLMProvider with generate(), generate_structured(), stream(). Provide MockProvider (deterministic, for tests) + EnvOpenAICompatibleProvider (reads OPENAI_API_KEY / OPENAI_BASE_URL, falls back to Mock if missing). Agents depend ONLY on interface.
- Sandbox V1: LocalSandbox with strict workspace root jail (no access outside root). Docker later, not now.
- Observability from day 1: JSONL runs/run_XXX/events.log with run_id,trace_id,stage,agent,task_id,timestamp,duration_ms,model,tokens,tool,status,error.
- Checkpoints: runs/run_XXX/state.json + checkpoints/001-spec.json,002-plan.json,003-task-TASKxxx.json,004-verification.json etc. Must support resume after crash.
- CLI: python -m harness.cli new "request" using argparse only. Output 5-stage progress as in spec.
- All models = Pydantic v2 BaseModel. All enums = str Enum. All IDs: REQ-XXX, TASK-XXX, TEST-XXX, run_XXX.
DEFAULTS (when in doubt, do this, don't ask):
- Stack inference: if request says "web app" -> frontend=React, backend=FastAPI, db=SQLite. If "API" -> FastAPI+SQLite. If "todo" -> FastAPI+SQLite+minimal HTML. If unspecified -> FastAPI+SQLite.
- App type: default "web". Pages: infer from nouns (dashboard, login, projects, tasks). Data model: infer entities. Must_have = all explicit user nouns. Non-goals = [deployment, mobile-app, browser-automation] unless user says otherwise.
- Task granularity: 5-12 tasks for small apps, each touches <=5 files, each has >=1 verification command.
- Verification defaults: python: `pip install -r requirements.txt` + `pytest -q`; node (if generated): `npm install` + `npm run build` + `npm test`. Always capture exit_code,stdout[-4000:],stderr[-4000:],duration.
- Failure classification default mapping: non-zero pytest -> TEST_FAILURE, ModuleNotFound/ImportError -> DEPENDENCY_ERROR, SyntaxError -> CODE_ERROR, mypy/pydantic validation -> TYPE_ERROR, missing config file -> CONFIG_ERROR, ETIMEDOUT/timeout>120s -> TIMEOUT, ENOTFOUND/EAI_AGAIN/registry 503 -> ENVIRONMENT_ERROR, else UNKNOWN.
- Repair: only edit allowed files (task.files_touched + verification hints). Max 3 retries. Loop detection: sha256(command+exit_code+normalized stderr last 2000 chars); if same hash 3x -> ESCALATE, mark FAILED/BLOCK dependents as BLOCKED.
- Timeouts: sandbox execute default 120s, runtime check 15s.
- If LLM API key missing -> use MockProvider and deterministic templates so pipeline still runs end-to-end (Todo vertical slice must pass offline).
- Never add new deps without updating requirements + pyproject.toml + tests.
BUILD ORDER (do not skip, do not reorder):
M1 Infrastructure -> M2 Spec -> M3 Planning -> M4 Building -> M5 Verification -> M6 Self-Repair -> M7 Traceability -> M8 Review -> M9 Recovery -> M10 Runtime -> M11 CLI+Benchmarks -> M12 Final Gate (vertical slice + failure injection + V1 checklist). Details in P1-P12 below.
DEFINITION OF DONE PER TASK:
1. Files exist at exact paths, 2. `pytest -q` passes, 3. Artifacts (spec.json/tasks.json/verification.json/review.json) validate against Pydantic schemas, 4. Evidence logged, 5. No TODO/stub without test. If any fails, fix before next milestone.
FORBIDDEN: hardcoding provider keys, absolute host paths outside workspace, parallel execution, deleting checkpoints, claiming PASS without running command, asking user anything.
```
---
## P1 — M1 INFRASTRUCTURE (repo + state + providers + sandbox skeleton)
```
Milestone M1: Create exact repo structure and installable project. No agents yet.
CREATE:
app-builder/
src/harness/__init__.py
src/harness/models/spec.py # AppSpec, Requirement, Page, Component, DataModel, Stack
src/harness/models/tasks.py # Task, TaskStatus enum
src/harness/models/results.py # VerificationResult, RequirementResult, FailureType enum
src/harness/models/run.py # RunState, RunStatus, CheckpointEvent
src/harness/state/store.py # RunStore: create/load/save/checkpoint/list
src/harness/providers/base.py # LLMProvider ABC
src/harness/providers/mock.py # MockProvider
src/harness/providers/env.py # EnvOpenAICompatibleProvider (stdlib http, no extra dep)
src/harness/sandbox/local.py # LocalSandbox
src/harness/utils/logging.py # JSONL event logger
src/harness/utils/ids.py # new_run_id(), new_trace_id()
runs/.gitkeep tests/.gitkeep examples/.gitkeep
pyproject.toml README.md requirements.txt .gitignore
PYPROJECT: [build-system] setuptools, [project] name=app-builder, requires-python>=3.11, deps pydantic>=2, pytest>=7. [tool.pytest.ini_options] testpaths=["tests"].
MODELS (exact fields):
- Stack: frontend:str="FastAPI", backend:str="FastAPI", db:str="SQLite"
- Requirement: id:str (REQ-001), text:str, must_have:bool=True, acceptance:str=""
- AppSpec: app_type:str, stack:Stack, pages:list[Page], components:list[Component], data_model:list[DataEntity], requirements:list[Requirement], must_have:list[str], explicit_non_goals:list[str], acceptance_criteria:list[str]
- Task: id:str, description:str, depends_on:list[str]=[], files_touched:list[str]=[], requirements:list[str]=[], verification:list[str]=[], status:TaskStatus=PENDING
- TaskStatus: PENDING,READY,RUNNING,COMPLETED,FAILED,TIMED_OUT,BLOCKED
- VerificationResult: task_id:str, status:str (PASS/FAIL), command:str, exit_code:int, stdout:str, stderr:str, duration_ms:int, failure_type:FailureType|None
- FailureType: CODE_ERROR,TEST_FAILURE,TYPE_ERROR,DEPENDENCY_ERROR,CONFIG_ERROR,ENVIRONMENT_ERROR,TIMEOUT,UNKNOWN
- RequirementResult: requirement_id:str, status:str, evidence:list[str]=[], missing:list[str]=[]
- RunState: run_id, trace_id, user_request:str, workspace:str, spec:AppSpec|None, plan:list[Task]=[], task_results:dict[str,Any]= {}, verification_results:list[VerificationResult]=[], review:dict|None, checkpoints:list[str]=[], status:RunStatus
- RunStatus: CREATED,SPEC_DONE,PLAN_DONE,BUILDING,VERIFYING,REPAIRING,REVIEW_DONE,PASS,FAIL
SANDBOX LocalSandbox(root:Path):
create_workspace(run_id)->Path; write_file(rel:str,content:str)->Path (reject ../ escape); read_file(rel)->str; list_files(rel=".")->list[str]; delete_file(rel); execute(cmd:list[str]|str, cwd:Path|None, timeout_s:int=120)->dict{exit_code,stdout,stderr,duration_ms} via subprocess.run(shell=False if list, True only if str + log warning).
STORE RunStore(base=Path("runs")):
create_run(user_request)->RunState; save(state); load(run_id)->RunState; checkpoint(state, name:str, payload:dict); list_runs()->list[str]; resume(run_id)->RunState.
LOGGING emit(base, run_id, trace_id, stage, agent, task_id, status, duration_ms=0, model="", tokens=0, tool="", error="") appends JSON line to runs/{run_id}/events.log.
LLMProvider ABC: generate(prompt:str, system:str="")->str; generate_structured(prompt:str, schema:type[BaseModel])->BaseModel; stream(prompt:str)->Iterator[str] (Mock yields words).
MockProvider: generate returns deterministic template echo; generate_structured returns schema.model_validate({minimal valid}) for AppSpec/TaskList used in tests.
Env provider: if no env key, delegate to MockProvider (so offline tests pass).
ACCEPTANCE:
- `pip install -e .` succeeds, `pytest -q` collects (even if 0 tests, add tests/test_models.py validating AppSpec + Task roundtrip + sandbox jail rejects ../ + store save/load).
- No network needed. No questions.
DO NOT: build agents, DAG, verifier yet.
```
## P2 — M2 SPEC AGENT + VALIDATOR
```
Milestone M2: User request -> validated spec.json. No code generation.
FILES:
src/harness/agents/spec.py # SpecAgent(llm:LLMProvider).generate(user_request:str)->AppSpec (prompt template + parse + fallback deterministic parser if LLM fails)
src/harness/validation/spec_validator.py # SpecValidator.validate(spec)->list[str] errors (empty=PASS)
tests/test_spec.py
SPEC AGENT LOGIC:
System: "You output ONLY valid JSON matching AppSpec schema. No prose. Infer stack/pages/data_model/requirements/must_have/non_goals/acceptance."
User template includes: request + stack defaults from P0 + require REQ-001..N, acceptance per requirement.
Post-process: assign REQ-001.. sequential, ensure must_have non-empty, acceptance_criteria non-empty, explicit_non_goals default 3 items.
If LLM JSON invalid -> fallback rule-based parser (keyword scan for auth/projects/tasks/dashboard -> entities) so offline PASS.
VALIDATOR CHECKS (each returns error string):
Structural: app_type non-empty, stack.* non-empty, requirements>=1, acceptance_criteria>=1, IDs unique matching REQ-\\d{3}.
Logical: every page.entity (if field) in data_model names; no requirement text substring in explicit_non_goals; if any requirement mentions API/backend then stack.backend non-empty; duplicate requirement text (case-insensitive) error; every must_have maps to >=1 requirement (substring or explicit link).
Pipeline helper: spec_stage(store, run_id, llm, max_attempts=2): generate->validate; if FAIL retry with error feedback; if still FAIL raise; else save checkpoint 001-spec.json + state.spec.
ACCEPTANCE:
- tests/test_spec.py: valid request -> PASS; conflicting non-goal (req "login" + non-goal "login") -> FAIL detected; missing acceptance -> FAIL; retry loop succeeds on 2nd attempt (mock failing once).
- Example artifact examples/todo_spec.json (Todo app, 4 REQs).
DO NOT: plan or build.
```
## P3 — M3 PLANNER + DAG ENGINE (sequential)
```
Milestone M3: AppSpec -> tasks.json DAG + sequential executor. No parallel.
FILES:
src/harness/agents/planner.py # PlannerAgent(llm).plan(spec)->list[Task]
src/harness/orchestration/dag.py # DagEngine + PlanValidator
tests/test_planner.py tests/test_dag.py
PLANNER RULES (enforce in code, not just prompt):
1. DB/schema task(s) first (files containing model/schema/db/migration).
2. Shared components before pages.
3. Auth infra before authenticated routes (files with auth).
4. Every REQ -> >=1 task.requirements; every task -> >=1 verification string + >=1 requirements + >=1 files_touched.
5. IDs TASK-001.. sequential, depends_on only earlier IDs (acyclic). If LLM violates, auto-fix: sort + repair deps deterministically.
6. LLM prompt: include spec JSON + ordering rules + output JSON list only. Fallback template: [schema/setup, auth, core entities CRUD, pages/API, tests/verify] if LLM fails.
7. files_touched must be relative POSIX paths inside workspace (no absolute, no ..).
DAG ENGINE (sequential V1):
class DagEngine(tasks:list[Task]): statuses dict; get_ready()->list[Task] (PENDING + all deps COMPLETED); mark(task_id,status); is_done(); blocked_propagation(): if dep FAILED/TIMED_OUT -> dependents BLOCKED; topological_order() raises on cycle.
statuses: PENDING,READY (computed, not stored — store PENDING until dispatched),RUNNING,COMPLETED,FAILED,TIMED_OUT,BLOCKED. FAILED != TIMED_OUT preserved.
Executor (in orchestration/runner.py skeleton, full impl M4/M5 but DAG part now): loop find READY -> RUNNING -> (placeholder hook) -> COMPLETED; if none READY and not done -> deadlock error listing BLOCKED.
PlanValidator: duplicate IDs, unknown dep, cycle, orphan REQ (no task), task without verification, overlapping files warning (not error in V1, but log for future parallel).
ACCEPTANCE:
- Todo spec (4 REQs) -> 5-8 tasks, all rules hold (assert in test).
- Cycle fixture -> validator error + engine raises.
- TASK-001 FAIL -> dependents BLOCKED (test).
- Checkpoint 002-plan.json saved via store.
DO NOT: run builders, no concurrency, no file-conflict scheduler yet.
```
## P4 — M4 BUILDER + TOOLS (sandbox-gated)
```
Milestone M4: Task -> real files. Minimal context only.
FILES:
src/harness/agents/builder.py # BuilderAgent(llm, sandbox, store)
src/harness/tools/files.py # read_file, write_file, list_files wrappers enforcing allowlist
tests/test_builder.py
BUILDER INPUT (only this, never full dump):
task:Task, requirements:list[Requirement] (filtered to task.requirements), spec_summary:dict (app_type,stack,data_model names), dependency_results:list[VerificationResult], allowed_files:list[str] (=task.files_touched), verification:list[str].
BUILDER PROMPT TEMPLATE:
"Implement TASK-{id}: {description}. Requirements: {req texts+acceptance}. Allowed files (ONLY these): {list}. Prior results: {dep summary}. Output file contents as JSON map {{relpath: content}}. Use FastAPI+SQLite if backend. Include imports, no stubs. If test file required, include pytest tests."
TOOL ENFORCEMENT:
Builder may call only sandbox.write_file/read_file/list_files within workspace + within allowed_files (write) — any other path -> PermissionError logged, task FAILED (TYPE_ERROR? use CONFIG_ERROR).
Never subprocess directly — only via sandbox.execute in M5. Builder writes files then returns {written:[...]}.
SCAFFOLD DEFAULTS (offline-safe):
If LLM returns invalid map -> fallback writes minimal FastAPI app: workspace/src/main.py (health GET /health), workspace/src/models.py, workspace/requirements.txt (fastapi,uvicorn,pydantic,pytest), workspace/tests/test_health.py asserting /health via TestClient or file exists check if fastapi missing. This guarantees vertical slice passes offline.
ACCEPTANCE:
- Given Todo TASK-001 (schema) in temp workspace -> files created under allowlist, outside-write rejected.
- Mock LLM offline still produces runnable scaffold + pytest passes.
- Store checkpoint 003-task-{id}.json after each task.
DO NOT: verify (M5) or repair yet — just write files.
```
## P5 — M5 VERIFICATION (real execution) + FAILURE CLASSIFICATION
```
Milestone M5: Actually run commands, capture structured results, classify.
FILES:
src/harness/verification/verifier.py # Verifier(sandbox).run(task, workspace, commands)->list[VerificationResult]
src/harness/verification/classifier.py # FailureClassifier.classify(result)->FailureType
tests/test_verifier.py tests/test_classifier.py
VERIFIER:
def verify_task(task:Task, workspace:Path)->list[VerificationResult]:
for cmd_str in task.verification (default if empty: ["pytest -q"]):
parse cmd_str via shlex.split (POSIX) -> sandbox.execute(cmd, cwd=workspace, timeout 120s)
capture exit_code,stdout[-4000:],stderr[-4000:],duration_ms; status PASS if 0 else FAIL; failure_type=None if PASS else classify().
persist 004-verification-{task_id}.json via store.
Also suite-level: verify_workspace(workspace, extra=["pip install -r requirements.txt" if exists]) helper.
Must NOT do static inspection only — must execute. Timeout -> TIMED_OUT + failure_type TIMEOUT.
CLASSIFIER (regex + exit_code, deterministic):
- TIMEOUT if duration>=timeout or "timed out"/"TimeoutExpired".
- ENVIRONMENT_ERROR if "EAI_AGAIN|ENOTFOUND|503|registry.*unavailable|Network is unreachable|pip.*Could not fetch".
- DEPENDENCY_ERROR if "ModuleNotFound|ImportError|No module named|npm ERR.*404|Could not resolve dependency".
- TYPE_ERROR if "mypy|TypeError:.*expected|pydantic.*ValidationError|TS2322|Property.*does not exist".
- TEST_FAILURE if "FAILED|AssertionError|1 failed|FAIL tests/" and exit!=0 and not above.
- CONFIG_ERROR if "FileNotFound.*config|missing.*pyproject|requirements.*not found|PORT in use" etc.
- CODE_ERROR if "SyntaxError|IndentationError|NameError|ReferenceError".
- else UNKNOWN.
Unit-test each with fixtures from spec (e.g., "npm registry unavailable" -> ENVIRONMENT_ERROR).
ACCEPTANCE:
- Create broken workspace (syntax error) -> verifier returns FAIL + CODE_ERROR, exit!=0, stderr captured.
- Good scaffold from M4 -> PASS.
- Timeout fixture (sleep 3 with timeout 1) -> TIMED_OUT.
DO NOT: auto-repair yet.
```
## P6 — M6 REPAIR LOOP + LOOP DETECTION
```
Milestone M6: FAIL -> classify -> repair -> reverify, max 3, loop guard.
FILES:
src/harness/orchestration/repair.py # RepairLoop + LoopDetector
src/harness/agents/repair_agent.py # RepairAgent(llm, sandbox)
tests/test_repair.py
LOOP DETECTOR:
def sig(cmd, exit_code, stderr): return sha256(f"{cmd}|{exit_code}|{normalize(stderr[-2000:])}".encode()).hexdigest()
normalize: lowercase, strip numbers/paths/timestamps (regex), collapse whitespace.
LoopDetector(seen:dict[sig,count]): add(sig)-> (is_loop:bool, count:int); is_loop True if count>=3 same sig.
REPAIR AGENT:
Input: task, spec_slice, failing VerificationResult + classifier label + last file contents (read via sandbox, truncated 8000 chars).
Prompt: "Fix {FailureType} in {files}. Error: {stderr}. Do NOT rewrite unrelated files. Output JSON map {{relpath: full corrected content}}."
Strategy by type: ENVIRONMENT_ERROR -> DO NOT rewrite code, retry once after 2s, if persists mark FAILED (env, not code); DEPENDENCY_ERROR -> fix requirements.txt/pyproject; TYPE_ERROR/CODE_ERROR/TEST_FAILURE -> patch code; CONFIG_ERROR -> fix config; TIMEOUT -> reduce scope/increase timeout once, else FAIL.
REPAIR LOOP:
def run_with_repair(task, workspace, verifier, repair_agent, max_retries=3):
attempt=0; while True: results=verifier.verify_task(...); if all PASS return PASS; classify; sig check -> if loop: log WARNING, one final repair try, then ESCALATE FAILED; if attempt>=max_retries: mark FAILED; else repair (write patched files via sandbox), checkpoint repair_started, attempt+=1, reverify.
Update DagEngine statuses + store.task_results.
ACCEPTANCE:
- Inject TypeScript/Python syntax error -> loop detects, repairs, PASS within <=3 (test with Mock LLM returning fixed content).
- Identical failure 3x (mock repair returns same broken file) -> loop stops, ESCALATE, task FAILED, dependents BLOCKED.
- ENVIRONMENT_ERROR does NOT trigger code rewrite (assert files unchanged, 1 retry only).
DO NOT: checkpoints beyond repair_started/task_completed yet (M9).
```
## P7 — M7 TRACEABILITY + M8 REVIEW AGENT
```
Milestone M7+M8: Requirement -> Task -> File -> Test -> Evidence + final audit gate.
FILES:
src/harness/trace/matrix.py # build_matrix(spec, tasks, verification_results, workspace)->dict[REQ, {tasks, files, tests, status}]
src/harness/agents/reviewer.py # ReviewAgent(llm|rule-based).review(spec, matrix, workspace)->dict{requirements:[{id,status,evidence,missing}], overall_status}
tests/test_trace.py tests/test_review.py
MATRIX:
For each REQ: tasks = [t for t in plan if REQ in t.requirements]; files = union files_touched; tests/commands = union verification; evidence = [f for f in files if workspace/f exists] + [v.command for v in results if v.status PASS and v.task_id in tasks]; status PASS only if >=1 file exists AND >=1 PASS result covering it, else FAIL with missing=[reasons].
No evidence -> FAIL (never PASS on prose).
REVIEWER (rule-based default, LLM optional):
Rule pass: check file exists + PASS verification + (if acceptance mentions keyword, grep file for keyword, else warn). LLM may add rationale but cannot override FAIL->PASS without evidence.
Output JSON exactly: {"requirements": [{"id","status":"PASS|FAIL","evidence":[paths+commands],"missing":[]}], "overall_status":"PASS|FAIL"} saved as review.json.
Overall PASS only if all REQs PASS.
ACCEPTANCE:
- Todo run with all PASS -> matrix shows 4/4, review overall PASS with evidence paths.
- Delete one implementation file -> that REQ FAIL with missing=["file src/... not found"], overall FAIL.
- review.json validates against RequirementResult list schema.
DO NOT: runtime/browser yet.
```
## P8 — M9 CHECKPOINTING + RECOVERY
```
Milestone M9: Crash-safe resume.
FILES:
src/harness/state/checkpoints.py # CheckpointManager (thin over RunStore)
tests/test_recovery.py
EVENTS (must emit via store.checkpoint):
spec_created, plan_created, task_started, task_completed, verification_completed, repair_started, review_completed.
Layout: runs/{run_id}/state.json (latest RunState), checkpoints/{seq:03d}-{event}-{task?}.json, workspace/ (generated app), events.log.
RESUME LOGIC in orchestration/pipeline.py::resume(run_id):
load state.json; find last completed checkpoint; recompute DAG statuses from task_results+verification_results; requeue PENDING/READY/RUNNING->PENDING (RERUN), keep COMPLETED/FAILED/BLOCKED; continue pipeline without redoing COMPLETED tasks (assert file hashes unchanged).
ACCEPTANCE:
- Start Todo run, kill after TASK-002 (simulate by saving partial state), resume() completes remaining without redoing TASK-001 (assert events.log shows TASK-001 once).
- Corrupt state.json -> resume raises clear error (not silent).
- tests cover checkpoint file naming + sequence.
DO NOT: parallel yet.
```
## P9 — M10 RUNTIME VERIFICATION (no browser in V1)
```
Milestone M10: Prove built app actually starts and serves.
FILES:
src/harness/verification/runtime.py # RuntimeVerifier
tests/test_runtime.py
RUNTIME VERIFIER STEPS (FastAPI default, Node fallback):
1. Detect entry: src/main.py:app or app.py:app or package.json main. If none, FAIL (CONFIG_ERROR).
2. Start: `python -m uvicorn src.main:app --port {free_port}` or `npm run dev -- --port {port}` via subprocess.Popen (through sandbox root), wait up to 15s.
3. Checks: process alive, TCP port open (socket.connect), GET /health or / returns 2xx, GET /docs or /api/health if exists 2xx, SQLite file exists/connects if expected.
4. Capture logs, kill process, return {status PASS/FAIL, checks:[{name, ok, detail}], evidence:[log snippet, http status]}.
5. On FAIL classify (CONFIG/DEPENDENCY/ENVIRONMENT) for repair loop reuse.
BROWSER (V1: STUB ONLY): create src/harness/verification/browser.py with `def verify_acceptance(...): raise NotImplementedError("Browser verification deferred post-V1")` + test asserting skip. Do NOT implement Playwright/Selenium now.
ACCEPTANCE:
- M4 scaffold app -> runtime PASS (health 200).
- Broken port (app exits) -> FAIL with diagnostics, repair hint.
DO NOT: Docker, parallel.
```
## P10 — M11 OBSERVABILITY + CLI + BENCHMARKS
```
Milestone M11: Operable harness.
OBSERVABILITY (src/harness/utils/logging.py finalize):
Every stage emits JSONL with all fields: run_id,trace_id,stage,agent,task_id,timestamp,duration_ms,model,tokens,tool,status,error. Provide `python -m harness.cli logs <run_id>` to pretty-print + `stats` (counts, retries, token sum, duration). Test asserts required keys on every line.
CLI (src/harness/cli.py, argparse only):
`builder new "request" [--run-id X --workspace Y --max-retries 3]` prints:
[1/5] Generating specification... ✓
[2/5] Planning N tasks... ✓
[3/5] Building... ✓ TASK-001 ... (⚠ fail + ↻ repairing)
[4/5] Verifying... ✓ Build ✓ Tests ✓ Runtime
[5/5] Reviewing requirements... ✓ 14/14 satisfied
BUILD COMPLETE / BUILD FAILED with paths to spec/tasks/verification/review.json
Also `builder resume <run_id>`, `builder review <run_id>`, `builder logs <run_id>`.
Map to orchestration/pipeline.py::run_new() orchestrating M2-M10 sequentially.
BENCHMARKS (benchmarks/*.json + src/harness/bench/runner.py):
5 fixed cases: todo, crud-dashboard, auth-app, api-db, ecommerce-prototype (each: request string + min REQs + expected tasks range). Runner executes pipeline with MockProvider, records {build_success, test_success, req_completion, repair_count, duration_s, tokens}. `pytest benchmarks/` or `builder bench --quick` (runs todo only). Do NOT judge by looks — assert metrics JSON written.
ACCEPTANCE:
- `builder new "Build a Todo app"` offline -> BUILD COMPLETE, artifacts exist, 5-stage output matches regex.
- events.log has all keys, bench quick passes.
DO NOT: web UI, Docker prod.
```
## P11 — M12 FINAL GATE: VERTICAL SLICE + FAILURE INJECTION + V1 CHECKLIST
```
Milestone M12: Prove reliability, then freeze V1.
TASKS (do in order, all must PASS):
1. VERTICAL SLICE: `builder new "Build a Simple Todo App with add/list/complete"` -> expect SPEC(>=3 REQs)->PLAN(>=3 tasks)->BUILD(files)->VERIFY(pytest PASS)->RUNTIME PASS->REVIEW PASS. Save under runs/demo_todo/. If any step FAILs, fix harness, do not proceed.
2. FAILURE INJECTION: introduce SyntaxError into workspace/src/main.py, rerun verify -> must DETECT (FAIL+CODE_ERROR), LOCALIZE (task_id), CAPTURE diagnostics, SEND to repair, REPAIR, REVERIFY PASS. Then inject identical failure 3x with no-op repair mock -> must WARNING->ESCALATE->FAILED+BLOCKED (assert).
3. V1 CHECKLIST (all ✓): valid spec, validated spec, valid DAG, sequential execution, real files, real verification, localized failures, repaired failures, loop detection, checkpoints+resume, traceability matrix, final review, working project in runs/.
4. DOCS: README.md (quickstart builder new/resume/logs, architecture diagram ASCII from plan §34, evidence principle), examples/todo_run/ (spec/tasks/verification/review JSON copies).
5. `pytest -q` entire repo green, `builder bench --quick` green.
FORBIDDEN IN V1 (assert not present): docker/, web_ui/, parallel workers, browser automation beyond stub, postgres, network research.
If all green -> tag V1 DONE. If not, loop M1-M11 fixes, never ship red.
```
---
## GLOBAL RULES APPENDIX (coding agent must obey)
1. NEVER ask questions. Use P0 defaults.
2. Build sequentially M1->M12. Do not start M(N+1) if M(N) tests red. Run `pytest -q` after each milestone.
3. Keep diffs small, files focused (<400 lines each, split if larger).
4. Every new module needs a test file. Every bug fix needs a regression test.
5. No secrets in repo. No absolute paths. No `shell=True` except verifier with logged warning.
6. Evidence over claims: every status change logs to events.log + checkpoint.
7. If LLM call fails/timeouts -> fallback deterministic path so pipeline never blocks.
8. Final deliverable per run: workspace/ + spec.json + tasks.json + verification.json + review.json + events.log + state.json.
9. End-of-run summary must list per REQ: Implemented? Tested? Runtime? Evidence? PASS/FAIL — never "AI says finished".
## prompts.chat MCP NOTE
- Searched via `prompts-chat_search_prompts/skills` (public, OK, 0 hits for niche harness queries — expected).
- `improve_prompt` / `save_prompt` require API key (`Authentication required`). To publish: set `PROMPTS_CHAT_API_KEY` in env, then call `prompts-chat_save_prompt(title="App-Builder Harness Autonomous Prompts", content=<this file>)`. Local file is authoritative until then.
- Suggested tags if publishing: `coding-agent, autonomous, python, harness, spec-driven, verification`.