# Utility digest - part 8/8 (scope: workspace/agents/opencode)
================
File: workspace/agents/opencode/engine/agent.md
================
# Workflow State Engine — Orchestration Toolkit
**Contract-driven state machine orchestration engine for AI agent workflows.**
State machine: `INIT → PLAN → PLAN_SCORED → EXECUTE → EXECUTE_SCORED → REVIEW → REVIEW_SCORED → COMPLETE`
Single source of truth for all AI agents. Reference skills and usage guides for depth. This file is `instructions[0]` — loaded by every agent at session start.
[![Contributors][contributors-shield]][contributors-url] [![Forks][forks-shield]][forks-url] [![Stargazers][stars-shield]][stars-url] [![Issues][issues-shield]][issues-url] [![License][license-shield]][license-url]
## Table of Contents
1. [1. Project Overview](#1-project-overview)
2. [2. Framework Architecture](#2-framework-architecture)
3. [3. Agent Reference — 16 Agents (Scrum Team)](#3-agent-reference--16-agents-scrum-team)
4. [4. Tool Usage Reference](#4-tool-usage-reference)
5. [5. Development Workflow](#5-development-workflow)
6. [6. Non-Negotiable Rules](#6-non-negotiable-rules)
7. [7. Skills Reference](#7-skills-reference)
(back to top)
## 1. Project Overview
### Core Concepts
| Concept | Description |
|---------|-------------|
| **Shared JSON Envelope** | `session/contract.json` — single source of truth for state, decisions, outputs, scoring |
| **State Machine** | 8 forward-flow states + BLOCKED escalation; agents transition through the workflow via the envelope |
| **Scoring Pipeline** | Three-tier scoring after every delegation (rule checks → LLM-as-judge → combined verdict) |
| **Agent Delegation** | Orchestrator delegates to specialized scrum-team agents (product-manager, system-analyst, developer, devops-engineer, quality-analyst, security-engineer, scrum-master, …) |
| **Cross-Model Review** | Reviewers run on a different model than executors; conflicting verdicts → BLOCKED, arbitrated by `council` |
| **Cross-Session Learning** | Lessons, patterns, gotchas persisted via `ctx_knowledge` |
### State Machine Transitions
```
INIT → PLAN → PLAN_SCORED → EXECUTE → EXECUTE_SCORED → REVIEW → REVIEW_SCORED → COMPLETE
↘ ↘ ↘
BLOCKED (score < 50 or retry ≥ 3)
↘
User intervention → retry with guidance
```
| Transition | Gate | Condition |
|-----------|------|-----------|
| INIT → PLAN | Session created | Always |
| PLAN → PLAN_SCORED | Plan produced | Always |
| PLAN_SCORED → EXECUTE | Plan accepted | Score ≥ 70 |
| EXECUTE → EXECUTE_SCORED | Implementation done | Always |
| EXECUTE_SCORED → REVIEW | Code review | Score ≥ 70 |
| REVIEW → REVIEW_SCORED | Review done | Always |
| REVIEW_SCORED → COMPLETE | All gates pass | Score ≥ 70 |
| Any → BLOCKED | Escalation | Score < 50 or retry ≥ 3 |
### Scoring Pipeline (Three-Tier)
1. **Tier 1 — Rule-Based Checks**: Schema valid (-15), permissions violated (-40), blast radius HIGH (-40), writing order wrong (-15), required fields missing (-15). Subtotal ≥ 70 → Tier 2.
2. **Tier 2 — LLM-as-Judge**: Scores 0-100 on requirements fulfillment (0-40), governance compliance (0-30), completeness (0-20), edge cases (0-10).
3. **Tier 3 — Combined Verdict**: PASS (≥70), RETRY (50-69, max 3 attempts), BLOCKED (<50).
### Architecture
```
agents/ → 16 agent instruction files (orchestrator, product-manager, scrum-master, developer, devops-engineer, quality-analyst, security-engineer, …)
skills/ → 66 skill directories (system-analyst, writing-plans, spec-driven-development, …)
commands/ → slash commands (impeccable)
contract/ → contract.template.json, contract.schema.json, state.template.md
rules/ → rules.json (state machine transitions, scoring thresholds, cross-model review)
doc/ → workflow guides, project.md, gap analysis, reports
usage/ → 4 tool usage guides (lean-ctx, graphify, context7, opencode)
```
(back to top)
---
## 2. Framework Architecture
### Deployment Layout
The engine ships inside the container image (`COPY engine/ /opt/engine/`) and is exposed to OpenCode through symlinks in the config directory. Agents read it at `/opt/engine/`:
```
~/.config/opencode/
├── opencode.json → (bind-mounted) config/opencode.json
├── agent.md → /opt/engine/agent.md (instructions[0])
├── /opt/engine/agents/ → /opt/engine/agents/ 16 agent instruction files
├── /opt/engine/skills/ → /opt/engine/skills/ 66 skill directories
├── /opt/engine/commands/ → /opt/engine/commands/ slash commands
├── /opt/engine/rules/ → /opt/engine/rules/ rules.json (state machine)
├── /opt/engine/contract/ → /opt/engine/contract/ contract.template.json, contract.schema.json, state.template.md
├── /opt/engine/usage/ → /opt/engine/usage/ 4 tool usage guides
└── /opt/engine/doc/ → /opt/engine/doc/ workflow guides, reports
```
Engine changes take effect on container rebuild; the bind-mounted config on restart.
### Directory Tree
The `engine/` source mirrors the runtime layout — edit here, rebuild to deploy:
```
├── agent.md ← THIS FILE (instructions[0]; includes the §lean-ctx code-intelligence block)
├── README.md ← Project overview (diagram, badges)
├── /opt/engine/agents/ ← 16 agent instruction files
├── /opt/engine/doc/ ← Workflow guides, project.md, reports
├── /opt/engine/rules/ ← rules.json (state machine, scoring, cross-model review)
├── /opt/engine/skills/ ← 66 skill directories
├── /opt/engine/commands/ ← slash commands
├── /opt/engine/contract/ ← contract.template.json, contract.schema.json, state.template.md
├── /opt/engine/usage/ ← 4 tool usage guides
└── /opt/engine/scripts/ ← Automation & validation scripts
```
(back to top)
---
## 3. Agent Reference — 16 Agents (Scrum Team)
> Full agent instruction files at `/opt/engine/agents/` (symlinked into the OpenCode config dir). Each file defines role, MCPs, skills, and post-flight protocol.
>
> **Model tiers**: `executor` = `opencode/big-pickle` (build/plan tier — developer, fixer, system-analyst); `default` = `sumopod/glm-5.3-flash`, fallback `sumopod/deepseek-flash` (all other agents). Reviewers never share the executor's model — this is the basis of **cross-model review**.
### Orchestrator
| Agent | Role | Key MCPs | Access | Contract States | Model | Skills | File |
|-------|------|----------|--------|----------------|-------|--------|------|
| **orchestrator** | Scrum lead / primary orchestrator — delegates, scores, drives the state machine | lean-ctx, context7 | Read/write | `*` (all) | default | (orchestrator — agent.md) | `/opt/engine/agents/orchestrator.md` |
| **design-orchestrator** | UI/UX specialist orchestrator — enforces 4-layer design pipeline (Exemplar > Generator > Taste > Gate) | lean-ctx, context7 | Read/write | `*` (all) | default | design-exemplar, ui-ux-pro-max, design-taste-frontend, impeccable | `/opt/engine/agents/design-orchestrator.md` |
### Product & Planning Agents (read-only)
| Agent | Role | Key MCPs | Access | Contract States | Model | Skills | File |
|-------|------|----------|--------|----------------|-------|--------|------|
| **product-manager** | Requirements + acceptance-criteria backlog (GWT) | lean-ctx, context7 | Read-only | `INIT`, `PLAN` | default | business-analyst, brainstorming, writing-plans, diagram-design | `/opt/engine/agents/product-manager.md` |
| **system-analyst** | Impact analysis, specs, detailed plans | lean-ctx | Read-only | `INIT`, `PLAN`, `PLAN_SCORED` | executor | system-analyst, business-analyst, writing-plans, diagram-design | `/opt/engine/agents/system-analyst.md` |
| **software-architect** | Architecture trade-offs, tech-debt assessment | lean-ctx | Read-only | `INIT`, `PLAN`, `PLAN_SCORED`, `REVIEW` | default | system-analyst, java-developer, diagram-design | `/opt/engine/agents/software-architect.md` |
### Implementation Agents (read/write)
| Agent | Role | Key MCPs | Access | Contract States | Model | Skills | File |
|-------|------|----------|--------|----------------|-------|--------|------|
| **developer** | Implements the plan, writes tests alongside code | lean-ctx | Read/write | `EXECUTE`, `EXECUTE_SCORED` | executor | test-driven-development, software-developer, design-taste-frontend, impeccable | `/opt/engine/agents/developer.md` |
| **devops-engineer** | CI/CD, containerization, deploy, migrations, observability, rollback | lean-ctx | Read/write | `EXECUTE`, `EXECUTE_SCORED`, `REVIEW` | default | devops-expert, observability-and-instrumentation, diagram-design | `/opt/engine/agents/devops-engineer.md` |
| **fixer** | Bounded, scoped fixes for well-defined tasks | lean-ctx | Read/write | `EXECUTE` | executor | systematic-debugging | `/opt/engine/agents/fixer.md` |
### Quality & Process Agents (read-only)
| Agent | Role | Key MCPs | Access | Contract States | Model | Skills | File |
|-------|------|----------|--------|----------------|-------|--------|------|
| **quality-analyst** | Independent quality review (5-axis: correctness, readability, architecture, security, performance) | lean-ctx | Read-only | `REVIEW`, `REVIEW_SCORED` | default | qa-expert, security-expert, code-review-and-quality, impeccable | `/opt/engine/agents/quality-analyst.md` |
| **security-engineer** | Independent security review — threat model, vulns, secrets, trust boundaries | lean-ctx | Read-only | `REVIEW`, `REVIEW_SCORED` | default | security-expert, code-review-and-quality | `/opt/engine/agents/security-engineer.md` |
| **scrum-master** | Process owner — guards the contract, state machine, and workflow; unblocks BLOCKED, DoD gate | lean-ctx | Read-only | `*` (all), `BLOCKED` | default | orchestration-template, audit-observability, adaptive-solver | `/opt/engine/agents/scrum-master.md` |
| **learner** | Post-execution retro — extract lessons, gotchas, patterns | lean-ctx | Read-only | `REVIEW_SCORED`, `COMPLETE` | default | verification-before-completion | `/opt/engine/agents/learner.md` |
### Support Agents (read-only)
| Agent | Role | Key MCPs | Access | Contract States | Model | Skills | File |
|-------|------|----------|--------|----------------|-------|--------|------|
| **explorer** | Fast codebase search / unknowns discovery | lean-ctx | Read-only | `*` (all) | default | (exploration) | `/opt/engine/agents/explorer.md` |
| **librarian** | External docs / library research | context7 | Read-only | `*` (all) | default | (research) | `/opt/engine/agents/librarian.md` |
| **council** | Multi-model consensus + cross-model arbitration | lean-ctx | Read-only | `*` (all) | default | (multi-LLM consensus) | `/opt/engine/agents/council.md` |
| **observer** | Visual analysis — images, PDFs, diagrams | lean-ctx | Read-only | `*` (all) | default | (visual analysis), impeccable, diagram-design | `/opt/engine/agents/observer.md` |
### Lane Name Mapping
| Lane | Agent | subagent_type |
|------|-------|---------------|
| `@product-manager` | product-manager | `product-manager` |
| `@scrum-master` | scrum-master | `scrum-master` |
| `@devops-engineer` | devops-engineer | `devops-engineer` |
| `@security-engineer` | security-engineer | `security-engineer` |
| `@explorer` | explorer | `explorer` |
| `@librarian` | librarian | `librarian` |
| `@fixer` | fixer | `fixer` |
| `@council` | council | `council` |
| `@observer` | observer | `observer` |
### Agent State Machine
See [`/opt/engine/doc/workflow.md`](/opt/engine/doc/workflow.md) for the canonical state machine, scoring pipeline, gates, and validation rules.
(back to top)
---
## 4. Tool Usage Reference
### 4.1 Quick Reference
| Tool | What It Does | Guide |
|------|-------------|-------|
| **lean-ctx** | File reads, shell, search, edit, impact analysis, knowledge persistence, sessions | `/opt/engine/usage/lean-ctx.md` |
| **Graphify** | Knowledge graph, community discovery, PR impact | `/opt/engine/usage/graphify.md` |
| **Context7** | Library/framework docs lookup | `/opt/engine/usage/context7.md` |
| **rag MCP** (`rag_search` / `rag_index` / `rag_status` / `rag_get`) | Semantic search over workspace code, docs, and engine skills before unfamiliar work | `/home/node/workspace/opencode/rag/README.md` |
| **Firecrawl plugin** | First-party V2 plugin: keyless developer search, keyed alexandria gated to ask, CLI skills registered at runtime | `/opt/engine/plugins/opencode-firecrawl/VENDORED.md` |
### 4.3 Tool Comparison Matrix
| Task | Best Tool | Why |
|------|-----------|-----|
| **Check blast radius before editing** | `ctx_impact` | Knowledge graph tracks callers/importers at d=1/d=2/d=3 depths |
| **Explore unfamiliar code flows** | `ctx_compose` | Returns process-grouped execution flows ranked by relevance |
| **Read/edit files** | `lean-ctx ctx_read`/`ctx_edit` | Cached reads (~13 tok), safe search-and-replace edits |
| **Shell commands** | `lean-ctx ctx_shell` | Compressed output, 95+ patterns, replaces bash |
| **Persist decisions/knowledge** | `lean-ctx ctx_knowledge remember` | Cross-session persistence with embeddings |
| **Save/resume sessions** | `lean-ctx ctx_session` | Survives OpenCode restarts |
| **External library research** | `context7_query_docs` | Official docs lookups with code examples |
| **Primary-source passages (issues, PRs, READMEs)** | `firecrawl_developer_search` | Keyless plugin search over raw sources; best for error hunting and API contract questions |
| **Single arbitrary page** | fetch MCP (`fetch`) | Keyless one-off HTTP GET; zero-dependency fallback |
| **Multi-model consensus** | `council_session` | Fresh-context adversarial review for high-stakes decisions |
### 4.4 Retrieval Balance (rag MCP)
`rag_search` over the workspace index is advisory, not a gate. Use it when entering an unfamiliar area, discovering project conventions, surveying existing implementations before writing code, or answering questions that span workspace code and docs. Skip it when the file is already in context, when `ctx_search` answers an exact-symbol lookup, or when the edit is trivial.
Guardrails:
1. Snippets are discovery anchors: open the cited file region with `ctx_read` before editing or quoting it as fact.
2. Stale index: if files changed since the last index, run `rag_index` first (incremental, hash-based, cheap).
3. Empty or weak hits: say so and fall back to `ctx_search` and direct reads. Never present memory-filled gaps as project facts.
4. Cite hits as `path:startline` so humans can verify.
5. Skip the search when the answer is already in context or you hold the exact file; `rag_search` is for discovery across the workspace, not for reading a known file.
Index scope: `/home/node/workspace`, built automatically at session boot when the server starts; run `rag_index` after bulk changes.
### 4.5 Retrieval Routing — ask the right layer once
Every retrieval question has exactly one right tool. Query the wrong layer and you pay an inference round-trip for nothing.
| Question type | Tool (1 call) | Never do |
|------|------|------|
| Exact symbol, callers, blast radius | `ctx_search` symbol / `ctx_impact` | grep sweeps across files |
| "How does our code do X" | `rag_search` | opening many files to re-derive |
| Library / framework API | `context7` | guessing from memory, random web search |
| Errors, issues, PR discussions | `firecrawl_developer_search` | broad web search |
| Single known page | `fetch` | scrape-first patterns |
| Past decisions, conventions, lessons | `ctx_knowledge recall` | re-deriving from scratch |
Scope `rag_search` with `glob` (e.g. `agents/opencode/engine/**`) before widening the query. Run `rag_index` after bulk edits — it is incremental and hash-based.
(back to top)
---
## 5. Development Workflow
### Session Model
Work is tracked in a single `session/` directory — no git branches required.
- All contract state lives in `session/contract.json`; snapshots are archived in `session/`.
- The engine is repository-agnostic — never block on branch or VCS checks.
- Every session is self-contained and resumable from `session/`.
### Before Any Edit
```
1. ctx_read / ctx_compose — understand the code first
2. ctx_impact({target: "symbol", direction: "upstream"}) # blast radius
3. If HIGH/CRITICAL → warn user before proceeding
```
Run this ladder in order before writing ANY code:
```
1. Does this need to exist? → skip it (YAGNI)
2. Standard library does it? → use it
3. Native platform feature? → use it
4. Already-installed dependency? → use it
5. Can this be one line? → one line
6. Only then: minimum code that works
```
**Not sacrificed:** Input validation at trust boundaries, data-loss error handling, security, accessibility, anything explicitly requested. Non-trivial logic leaves ONE runnable check (assert-based, no framework).
### Orchestration Flow (Scrum Team — Spec-Driven Development)
1. **Refine phase** — Delegate to @product-manager: capture intent, write Given-When-Then acceptance criteria, declare scope + non-goals. Attach user-journey or story-map diagrams via diagram-design when intent spans multiple actors.
2. **Discuss phase** — 5-lens check: Business, System, Dev, QA, DevOps. Clarify requirements.
- **Domain context (advisory)**: `rag_search` supplies domain context when the request touches unfamiliar code or docs.
3. **Plan phase** — Delegate to @system-analyst: impact analysis, edge cases, implementation plan. @software-architect reviews the plan (independent model). Plans spanning >3 files or crossing services include an architecture/impact diagram (diagram-design).
4. **Spec gate** (if >3 files, cross-service, or >30 min) — Write GWT-format specs. Cross-examine via DDD.
5. **Execute phase** — Delegate to @developer: implement plan step by step, write tests alongside code. @devops-engineer owns pipeline/deploy/migration. @fixer for scoped fixes. UI tasks load design-taste-frontend; developer passes the impeccable detector before marking UI work done (EXECUTE exit gate). Advisory retrieval: the developer consults `rag_search` for project conventions before writing non-trivial code, then verifies hits against the file before editing.
- **Context gate**: @developer loads project standards via `/skill context-standards` before edits when the target project defines `context/` files; @quality-analyst verifies compliance during review.
6. **Review phase** — Delegate to @quality-analyst (5-axis) AND @security-engineer (security) on the **reviewer model**, independent from the executor. Conflicting verdicts → BLOCKED, arbitrated by @council. UI diffs get the impeccable design-lens (slop detector) inside the readability axis.
7. **Process gate** — @scrum-master audits transitions, enforces gates, and owns the Definition of Done before COMPLETE. Unblocks BLOCKED sessions.
8. **Verify loop** — Run build commands, check conventions. Executors and fixers run `scripts/loop-validate.sh` as the inline done-gate before REVIEW.
9. **Ship** — Resolve issues, confirm deploy safety, run post-flight protocol; @learner captures lessons.
10. **Humanizer gate** — run `/skill humanizer` over all user-facing outputs shipped this session (READMEs, docs, PR descriptions, changelogs, release notes) before declaring COMPLETE. Docs that still carry AI patterns (em dashes, rule-of-three, promotional tone) are not done. Mandatory for user-facing text outputs; internal state files (session/state.md, contract JSON, code comments) are exempt.
### UI/UX Pipeline (Mandatory)
All UI work runs the 4-layer pipeline in this order — enforced by @design-orchestrator:
| # | Layer | Skill | Role |
|---|-------|-------|------|
| 1 | Exemplar | `design-exemplar` | Copy a proven look from the 84-brand DESIGN.md library (Linear, Stripe, Vercel, …) when a reference brand is named |
| 2 | Generator | `ui-ux-pro-max` | BM25-search a fitting design system (styles, palettes, fonts, patterns) for the user's own product |
| 3 | Taste | `design-taste-frontend` | Art direction pass — kill generic "AI-slop" patterns, enforce gallery-grade composition |
| 4 | Gate | `impeccable` | No UI task is done without an impeccable design-lens review |
Layer selection: user names a reference brand → Exemplar; user describes their own product → Generator; neither → Generator; trivial tweak (<10 lines, no new components) → pipeline may be skipped, impeccable gate still applies.
Rules: order is mandatory; never mix Exemplar + Generator for the same page; DESIGN.md (page) / MASTER.md (global) persistence per ui-ux-pro-max `--persist`.
Use **Doubt-Driven Development (DDD)** when uncertain: spawn a fresh-context adversarial reviewer to cross-examine non-trivial decisions before they stand.
### Before Commit (Post-Flight Protocol)
The canonical post-flight protocol is the **E1 table in [`doc/workflow/05-operations.md`](/opt/engine/doc/workflow/05-operations.md)** (8 steps: validate contract → impact verify → change detect → knowledge persist → session archive → save conversation → re-index graphify → metrics aggregation). Run it before every commit — do not maintain a second step list here.
**Pre-flight**: `bash /opt/engine/scripts/validate-contract.sh --file session/contract.json --score` (score < 70 = BLOCKED)
**Parallel conflict check**: `bash /opt/engine/scripts/detect-parallel-conflicts.sh --file1 /tmp/a.txt --file2 /tmp/b.txt`
**Atomic persist**: `bash /opt/engine/scripts/persist-contract.sh --file session/contract.json --inject-score 85` |
Exceptions: docs-only changes skip impact, change-detect, and re-index steps. Config-only changes skip impact and change-detect.
### Save Session Protocol
When the user says "save session" or a phase completes, save to **ALL** systems:
> **LOCAL-ONLY:** `session/`, `.session/` and everything inside them are runtime state — NEVER `git add`, commit, or push them (gitignored; history grows unbounded). Archive historical state logs to `.session/` instead.
```bash
# 1. Persist orchestration envelope to lean-ctx knowledge
lean-ctx ctx_knowledge remember key orchestration-contract value ""
# 2. Update session/state.md — append completed work items
# 3. Archive snapshot to session/ (contract files + state log + index)
bash /opt/engine/scripts/snapshot-contract.sh --snapshot-only
# 4. Save conversation context (survives OpenCode restart)
lean-ctx ctx_session save
# 5. Re-index Graphify knowledge graph (if graphify-out/ exists)
graphify --update 2>/dev/null || true
```
**One-shot alias**: `save session` = all 5 steps above. Always run the full protocol — partial saves lose audit trail, break resumption, or leave stale indexes.
### Update All States
After major changes (merged work or resumed sessions), synchronize all state systems:
```bash
# 1. Re-index Graphify knowledge graph
graphify --update 2>/dev/null || true
# 3. Snapshot contract state (session/ → archive)
bash /opt/engine/scripts/snapshot-contract.sh
# 4. Save conversation context (survives OpenCode restart)
lean-ctx ctx_session save
```
### Session Lifecycle Protocol
Every orchestration session persists contract state to the `session/` directory for cross-session traceability and safe resumption.
> **LOCAL-ONLY:** `session/`, `.session/` and everything inside them are runtime state — NEVER `git add`, commit, or push them (gitignored; history grows unbounded). Archive historical state logs to `.session/` instead.
#### Directory Layout
```
session/
├── contract.json ← Live orchestration contract (single source of truth)
├── contract.schema.json ← Schema for the contract
├── state.md ← Append-only chronological state log (LOCAL-ONLY, gitignored)
└── archive/ ← Timestamped snapshots (audit trail)
```
#### Lifecycle Protocol
| Event | Action |
|-------|--------|
| **State transition** | Update `session/contract.json` → snapshot via `/opt/engine/scripts/snapshot-contract.sh` |
| **Session end** (COMPLETE/BLOCKED) | Final snapshot → append summary to `session/state.md` |
| **New session** | Re-scaffold `session/contract.json` from `/opt/engine/contract/contract.template.json` |
#### Snapshot Command Reference
```bash
/opt/engine/scripts/snapshot-contract.sh # Full snapshot: copy files + update state.md
/opt/engine/scripts/snapshot-contract.sh --snapshot-only # Copy files only (skip state.md updates)
/opt/engine/scripts/snapshot-contract.sh --summary "State: ${STATE} — description" # Custom entry
/opt/engine/scripts/snapshot-contract.sh --dry-run --verbose # Preview without changes
```
#### Why It Matters
The `session/` archive preserves:
- **Audit trail**: `session/state.md` grows monotonically — every state transition, every decision, every blocker
- **Safe resume**: `session/contract.json` is the exact state from last session — no reconstruction needed
### Complex Tasks (Orchestration Template)
See [orchestration-template skill](/opt/engine/skills/orchestration-template/SKILL.md) and [`/opt/engine/doc/workflow.md`](/opt/engine/doc/workflow.md) for the full orchestration protocol.
(back to top)
---
## 6. Non-Negotiable Rules
- **Never commit secrets, credentials, tokens, `.env` files**
### Cross-Model Review
- **Reviewers (default tier, glm-5.3-flash) must run on a different model than the executor (build/plan tier, big-pickle)** they audit; fallbacks must not collapse separation — on double-fallback set `governance.cross_model_degraded` and cap verdict at RETRY.
- **Conflicting reviewer verdicts** (PASS vs BLOCK) → `BLOCKED`, arbitrated by `@council` (see `rules.json → enforcement.cross_model_review`)
- **Security-sensitive changes** (auth, secrets, credentials, config, crypto) auto-require `@security-engineer` review
### Code Safety
- **Never edit without `ctx_impact`** — run impact analysis first
- **Never commit without `ctx_delta`** — verify scope
- **Never rename with find-and-replace** — use `ctx_refactor`
- **Ignore HIGH/CRITICAL impact warnings** = governance violation
### Access (Mandatory)
- **All agent tool access flows through lean-ctx, the mandatory access layer**: reads via `ctx_read`, file discovery via `ctx_glob`/`ctx_search`, shell via `ctx_shell` (which auto-routes core primitives), code context via `ctx_compose`/`ctx_context`, impact via `ctx_impact`, knowledge via `ctx_knowledge` (`remember` requires a category), sessions via `ctx_session`
- **File edits use the host's native edit tool** (lean-ctx does not handle writes); external library documentation goes through context7
- **Bypassing the lean-ctx channel with a raw execution path is a governance violation**; deviations from any of the above require an explicit reason in the agent output
### Shell Safety
- **Always use `lean-ctx ctx_shell`** — never `bash` or `snip` (both denied in permissions)
- `bash` and `snip` trigger permission prompts and block automation
### Sandbox (Playground)
- **All scratch work goes to `sandbox//` at the project root**: probes, clones, screenshots, throwaway builds, rendered tests. Not to repo paths, not to `/tmp`, not into another session's folder.
- **One folder per live session**, slugified from the OpenCode session title (`opencode optimizer` → `opencode-optimizer`); on collision append a 4-character session-ID prefix.
- **Write only inside your own folder.** Other sessions' folders are read-only; deleting one needs explicit user approval.
- **Never write at `sandbox/` root.** Only `README.md`, `.gitignore`, and `.gitkeep` live there, and only those are tracked.
- **Sandbox contents are never committed** (gitignored on the `certs/` pattern). Promote anything durable to its project folder before COMPLETE.
- **No secrets, tokens, credentials, certificates, or customer data.** Bind servers to `127.0.0.1`, use ports 4100-4999, stop them before finishing.
- **Budget:** 200 MB and 7 days per session folder. Rules, naming, and prune commands: `sandbox/README.md`; machine-readable: `rules.json → enforcement.sandbox`.
### Communication
- Explain tradeoffs, not just decisions
- Admit unknowns. Be token-efficient: concise, no filler, no full-file dumps
- Return only what was asked
(back to top)
---
### Tool Call Economy (all agents)
Every tool call is one full LLM inference round-trip. The rules below are mandatory discipline, not style:
1. **Batch rule** — 2+ independent tool calls: issue them in ONE turn (parallel), never serialized.
2. **Chain rule** — dependent commands on the same tool: ONE call chained with `&&` (commit-push-sync-verify is one call), never one call per step.
3. **Gate-first rule** — check policy/security constraints BEFORE composing a command. A blocked command costs a full wasted inference; never retry-blind — reroute through the sanctioned path.
4. **Spawn-tax rule** — every subagent prompt carries complete context (requirements, decisions, retry issues, exact paths). Subagents start at turn 1 of real work — zero re-discovery turns.
5. **Tier rule** — execution tiers by blast radius × reversibility × ambiguity:
- **T1 Direct** (1–2 files, git-revertible, clear scope): orchestrator executes inline — edit → check → commit → push → verify. No subagent, no ceremony. Budget ≤10 turns.
- **T2 Light** (3–10 files, parallelizable): max ONE subagent with a full-context prompt; orchestrator verifies after. Budget ≤20 turns.
- **T3 Full** (security-sensitive, ambiguous scope, multi-service, or user-requested audit): analyst → developer → review with all gates.
6. **Verdict rule** — mechanical decisions (score pass/fail, routing choice, risk class): single-shot, no-tools inference on the cheapest capable model. Never a tool-loop turn.
7. **lean-ctx fluency rule** — follow usage/lean-ctx.md §13 (Field Guide): descend the discovery→zoom→edit ladder, use `diff`-mode re-reads, verify exit status alongside compressed output, recover archives with `ctx_expand` — and never edit from compressed views.
Known platform gap: subagent spawn does not fall back automatically on provider quota errors. On quota exhaustion, re-dispatch the same task with the fallback model set explicitly.
## 7. Skills Reference
All skills at `/opt/engine/skills/` (symlinked into the OpenCode config dir). Use `/skill ` for deep dives. Each skill directory contains a `SKILL.md` with full guidance.
| Skill | When to Load |
|-------|-------------|
| `java-developer` | Java 21 idioms, anti-patterns, quality gates (meta-skill — see companion skills below) |
| `spring-boot-enterprise` | Spring Boot 3.x REST, JPA, Security, OAuth2 (companion to java-developer) |
| `spring-boot-4x` | Spring Boot 4.x, virtual threads, GraalVM (companion to java-developer) |
| `java-streams` | Streams API, Collectors, Gatherers (companion to java-developer) |
| `java-optional` | Optional patterns, antipatterns (companion to java-developer) |
| `java-code-quality` | SOLID, clean code, anti-patterns (companion to java-developer) |
| `java-design-patterns` | GoF patterns, Java idioms (companion to java-developer) |
| `java-logging-patterns` | SLF4J, Logback, structured logging (companion to java-developer) |
| `jspecify-nullability` | JSpecify annotations, null-safety (companion to java-developer) |
| `jpa-hibernate-patterns` | N+1 prevention, @EntityGraph, batch fetching (companion to java-developer) |
| `jooq-best-practices` | jOOQ type-safe queries, DSL (companion to java-developer) |
| `postgres-table-design` | PostgreSQL schema, indexing, partitioning (companion to java-developer) |
| `pgvector-search` | pgvector semantic search, embeddings (companion to java-developer) |
| `postgres-text-search` | Full-text search, tsvector (companion to java-developer) |
| `mutation-testing` | pitest, mutation testing (companion to java-developer) |
| `coverage-kover-gradle` | Kover Gradle coverage reports (companion to java-developer) |
| `jdb-debugger` | JDB agentic debugging (companion to java-developer) |
| `software-developer` | Full-stack conventions, clean code, SOLID |
| `system-analyst` | Architecture evaluation, dependency mapping |
| `business-analyst` | Requirements gathering, acceptance criteria |
| `brainstorming` | Divergent thinking for exploring solution spaces |
| `qa-expert` | Test strategy, coverage analysis, regression prevention |
| `receiving-code-review` | Act on review feedback with technical rigor |
| `requesting-code-review` | Submit code for structured review |
| `security-expert` | Vulnerability assessment, secure coding, compliance |
| `devops-expert` | CI/CD, containerization, deployment |
| `observability-and-instrumentation` | Logging, metrics, tracing, alerting, SLI/SLO (Micrometer, Prometheus) |
| `executing-plans` | Execute implementation plans step by step |
| `release-plan` | PR templates, commit conventions, release process |
| `token-optimize` | Efficient token usage, batch operations |
| `verification-before-completion` | Verify success criteria before marking complete |
| `writing-plans` | Create structured implementation plans |
| `humanizer` | Remove AI writing patterns from output |
| `doubt-driven-development` | Validate assumptions before committing to design |
| `adaptive-solver` | Meta-cognitive uncertainty resolution — structured loop for ambiguity, low confidence, or failed attempts |
| `spec-driven-development` | Write spec first, implement after |
| `subagent-driven-development` | Delegate bounded work to specialized subagents |
| `systematic-debugging` | Structured root cause analysis |
| `test-driven-development` | Red/green/refactor cycle |
| `orchestration-template` | Contract-based multi-agent orchestration |
| `audit-observability` | State contract audit trail, orchestration observability, score analytics, cross-service consistency |
| `code-review-and-quality` | Review checklists, quality metrics |
| `simplify` | Reduce complexity without changing behavior |
| `impeccable` | Design director — critique, audit, polish UI; offline slop detector; required before UI tasks are marked done |
| `design-taste-frontend` | Art direction for frontend builds — design brief, tokens, browser self-check loop |
| `diagram-design` | 45+ branded HTML/SVG diagram types (architecture, flows, ER, journeys, deployment); Mermaid/drawio/excalidraw import |
| `design-exemplar` | Exemplar layer — awesome-design-md brand index (84 DESIGN.md files), fetch workflow, when to copy a reference look |
| `ui-ux-pro-max` | Generator layer — BM25 design-system search (192 industry rules, 79 styles, 192 palettes, 74 font pairings), `--design-system`, `--persist` (MASTER.md) |
| `banner-design` | Banner/hero art direction for social media, ads, web heroes (companion of ui-ux-pro-max) |
| `brand` | Brand voice, visual identity, messaging frameworks, asset consistency (companion of ui-ux-pro-max) |
| `design` | Umbrella design skill: logo generation, CIP mockups, HTML presentations, banners, icons, social photos (companion of ui-ux-pro-max) |
| `design-system` | Token architecture (primitive→semantic→component), CSS variables, component specs, slide generation (companion of ui-ux-pro-max) |
| `slides` | Strategic HTML presentations with Chart.js, design tokens, responsive layouts (companion of ui-ux-pro-max) |
| `ui-styling` | shadcn/ui + Tailwind + canvas-based UI styling, accessible components, theming (companion of ui-ux-pro-max) |
| `awesome-readme` | README/docs standard — badges, TOC, quick start, config tables, architecture notes (distilled from matiassingers/awesome-readme) |
| `awesome-pr` | Pull request authoring — Conventional Commit titles, description template, review checklist, small-PR sizing |
| `commit` | Conventional commits, atomic commits, commit message best practices |
| `gradle-test-runner` | Gradle test execution, filtering, parallel runs, test configuration |
| `ralph-coverage` | Ralph coverage analysis, test gap detection, coverage insights |
| `rebase-commit` | Interactive rebase, squash, commit cleanup, history management |
| `restart-spring-boot` | Spring Boot restart and hot-reload patterns for development |
| `shipping-and-launch` | End-to-end release orchestration — pre-launch verification, canary rollout, rollback planning, post-deploy smoke tests |
| `spec` | Specification writing, requirements documentation, acceptance criteria |
| `lean-ctx` | LeanCTX Context Gateway — select task-relevant context, apply controls, reuse local knowledge, inspect recorded evidence |
| `context-standards` | Discover and load project-specific pattern and standards files (context/*.md) before implementation, review, or design work |
| `dispatching-parallel-agents` | Coordinating multiple sub-agents for parallel execution — file-disjoint sharding, conflict avoidance, reconciliation |
| `tool-economy` | Tool-call discipline — batching, chaining, gate-first, spawn-tax, execution tiers, single-shot verdicts; retrieval routing table (§4.5) |
*Last updated: 2026-11-07. If you modify conventions, workflows, or config, update this file.*
(back to top)
# lean-ctx — Code Intelligence
This project is indexed by lean-ctx as **workflow-state-engine** (1675 symbols, 1666 relationships, 0 execution flows). Use the lean-ctx MCP tools to understand code, assess impact, and navigate safely.
> Index stale? Run `bash graphify --update` from the project root.
## Always Do
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows. For regression review, compare against the previous snapshot..
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `context({name: "symbolName"})`.
## Never Do
- NEVER edit a function, class, or method without first running `impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `rename` which understands the call graph.
- NEVER commit changes without running `detect_changes()` to check affected scope.
[contributors-shield]: https://img.shields.io/github/contributors/RizkiRachman/workflow-state-engine?style=for-the-badge
[contributors-url]: https://github.com/RizkiRachman/workflow-state-engine/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/RizkiRachman/workflow-state-engine?style=for-the-badge
[forks-url]: https://github.com/RizkiRachman/workflow-state-engine/network/members
[stars-shield]: https://img.shields.io/github/stars/RizkiRachman/workflow-state-engine?style=for-the-badge
[stars-url]: https://github.com/RizkiRachman/workflow-state-engine/stargazers
[issues-shield]: https://img.shields.io/github/issues/RizkiRachman/workflow-state-engine?style=for-the-badge
[issues-url]: https://github.com/RizkiRachman/workflow-state-engine/issues
[license-shield]: https://img.shields.io/github/license/RizkiRachman/workflow-state-engine?style=for-the-badge
[license-url]: https://github.com/RizkiRachman/workflow-state-engine/blob/main/LICENSE
================
File: workspace/agents/opencode/engine/opencode.json.template
================
{
"$schema": "https://opencode.ai/config.json",
"model": "sumopod/glm-5.3-flash",
"default_agent": "build",
"update": "disable",
"lsp": false,
"snapshots": false,
"watcher": {
"ignore": [
"**/node_modules/**",
"**/.runtime/**",
"**/target/**",
"**/logs/**",
"**/.git/**",
"**/graphify-out/**"
]
},
"permissions": [
{ "action": "*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" },
{ "action": "firecrawl_alexandria", "resource": "*", "effect": "ask" }
],
"plugins": ["/opt/engine/plugins/opencode-firecrawl"],
"providers": {
"sumopod": {
"name": "SumoPod",
"package": "@opencode/ai/providers/openai-compatible",
"settings": {
"baseURL": "{env:SUMOPOD_BASE_URL}",
"apiKey": "{env:SUMOPOD_API_KEY}"
},
"models": {
"deepseek-flash": {
"name": "DeepSeek Flash"
},
"glm-5.3-flash": {
"name": "GLM 5.3 Flash"
}
}
}
},
"agents": {
"build": {
"mode": "primary",
"model": "opencode/big-pickle",
"permissions": [
{ "action": "read", "resource": "*.env", "effect": "ask" },
{ "action": "read", "resource": "*.env.*", "effect": "ask" },
{ "action": "read", "resource": "*.env.example", "effect": "allow" }
]
},
"orchestrator": {
"description": "Scrum team lead and primary orchestrator — delegates to the team, runs the scoring pipeline, drives the state machine.",
"mode": "all",
"model": "sumopod/glm-5.3-flash",
"steps": 50,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "deny" },
{ "action": "grep", "resource": "*", "effect": "deny" },
{ "action": "glob", "resource": "*", "effect": "deny" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" }
]
},
"design-orchestrator": {
"description": "UI/UX specialist orchestrator — enforces 4-layer design pipeline (Exemplar > Generator > Taste > Gate).",
"mode": "all",
"model": "sumopod/glm-5.3-flash",
"steps": 50,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "deny" },
{ "action": "grep", "resource": "*", "effect": "deny" },
{ "action": "glob", "resource": "*", "effect": "deny" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "skill", "resource": "*", "effect": "allow" },
{ "action": "subagent", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" }
]
},
"product-manager": {
"description": "Owns requirements and the acceptance-criteria backlog. Turns user intent into testable Given-When-Then criteria. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/deepseek-flash",
"steps": 40,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"scrum-master": {
"description": "Process owner. Guards the contract, state machine, and workflow on every transition; unblocks BLOCKED sessions and drives retros. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/deepseek-flash",
"steps": 40,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"system-analyst": {
"description": "Analyzes requests, traces impact across the codebase, identifies edge cases, and produces a structured implementation plan or GWT spec.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 80,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"software-architect": {
"description": "Strategic architecture advisor and independent design reviewer — trade-offs, system-level debugging, YAGNI enforcement. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 60,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"developer": {
"description": "Implements the plan task by task following project conventions, writing tests alongside code.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 100,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "allow" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"devops-engineer": {
"description": "Builds and validates CI/CD, containerization, deployment, migrations, observability, and rollback plans. Read/write within scope.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 60,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "allow" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"quality-analyst": {
"description": "Independent quality reviewer — code quality, edge cases, tests, performance, and DB/DevOps operability. Cross-model reviewer. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/deepseek-flash",
"steps": 80,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"security-engineer": {
"description": "Independent security reviewer — threat modeling, vulnerability assessment, secrets/trust-boundary review. Cross-model reviewer. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/deepseek-flash",
"steps": 60,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"explorer": {
"description": "Codebase search specialist for discovering unknowns. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 30,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"librarian": {
"description": "External documentation and library research specialist. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 30,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" },
{ "action": "graphify_*", "resource": "*", "effect": "deny" }
]
},
"council": {
"description": "Multi-model consensus engine for high-stakes decisions and cross-model enforcement. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 30,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"observer": {
"description": "Visual analysis specialist for images, PDFs, and diagrams. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 30,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"fixer": {
"description": "Fast implementation specialist for well-defined bounded tasks and review fixes. Scoped edits only.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 40,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "allow" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" },
{ "action": "graphify_*", "resource": "*", "effect": "deny" }
]
},
"learner": {
"description": "Post-execution retro facilitator. Extracts lessons, patterns, and gotchas after completion to make the system smarter. Read-only — no edits.",
"mode": "subagent",
"model": "sumopod/glm-5.3-flash",
"steps": 40,
"permissions": [
{ "action": "shell", "resource": "*", "effect": "deny" },
{ "action": "edit", "resource": "*", "effect": "deny" },
{ "action": "read", "resource": "*", "effect": "allow" },
{ "action": "grep", "resource": "*", "effect": "allow" },
{ "action": "glob", "resource": "*", "effect": "allow" },
{ "action": "webfetch", "resource": "*", "effect": "deny" },
{ "action": "question", "resource": "*", "effect": "deny" },
{ "action": "context7_*", "resource": "*", "effect": "allow" },
{ "action": "memory_*", "resource": "*", "effect": "allow" },
{ "action": "lean-ctx_*", "resource": "*", "effect": "allow" }
]
},
"plan": {
"model": "sumopod/glm-5.3-flash"
},
"general": {
"model": "sumopod/glm-5.3-flash"
},
"explore": {
"model": "sumopod/glm-5.3-flash"
}
},
"instructions": [
"/opt/engine/agent.md"
],
"compaction": {
"auto": true,
"keep": {
"tokens": 20000
}
},
"mcp": {
"servers": {
"lean-ctx": {
"type": "local",
"command": [
"lean-ctx"
],
"environment": {
"LEAN_CTX_DATA_DIR": "/home/node/.local/share/opencode/lean-ctx"
}
},
"graphify": {
"type": "local",
"command": [
"sh",
"-c",
"exec graphify-mcp --graph \"${GRAPHIFY_GRAPH:-${GRAPHIFY_ROOT:-/home/node/workspace}/graphify-out/graph.json}\""
],
"disabled": true,
"environment": {
"OPENAI_API_KEY": "{env:SUMOPOD_API_KEY}",
"OPENAI_BASE_URL": "{env:SUMOPOD_BASE_URL}"
}
},
"context7": {
"type": "local",
"command": [
"context7-mcp"
],
"environment": {
"CONTEXT7_API_KEY": "{env:CONTEXT7_API_KEY}"
}
},
"memory": {
"type": "local",
"command": [
"mcp-server-memory"
]
},
"fetch": {
"type": "local",
"command": [
"mcp-server-fetch"
]
},
"everything": {
"type": "local",
"command": [
"mcp-server-everything"
]
}
}
}
}
================
File: workspace/agents/opencode/engine/package.json
================
{
"name": "workflow-state-engine",
"version": "2026.06.19.0421",
"description": "Contract-driven state machine orchestration engine for AI agent workflows",
"license": "MIT",
"author": "Workflow State Engine",
"keywords": [
"opencode",
"orchestration",
"state-machine",
"ai-agents",
"workflow"
],
"files": [
"scripts/*.sh",
"contract/*.json",
"rules/rules.json",
"agents/*.md",
"skills/*/SKILL.md",
"usage/*.md",
"agent.md",
"README.md",
"setup.sh"
]
}
================
File: workspace/agents/opencode/engine/README.md
================
# Workflow State Engine
Contract-driven state machine orchestration engine for AI agent workflows: 16 agents, 66 skills, gated scoring on every delegation.
Single source of truth for all AI agents. The canonical agent instruction file is [`agent.md`](agent.md), which is `instructions[0]` for every agent. The engine ships inside the container image (`engine/` copied to `/opt/engine/` and symlinked into the OpenCode config directory).
(back to top)
## Table of Contents
1. [About The Engine](#about-the-engine)
- [Built With](#built-with)
2. [Quick Start](#quick-start)
3. [Getting Started](#getting-started)
- [Prerequisites](#prerequisites)
- [Build](#build)
- [Configure](#configure)
- [Run](#run)
- [Verify](#verify)
4. [State Machine](#state-machine)
5. [Scoring Pipeline](#scoring-pipeline)
6. [Agents](#agents)
7. [Scripts](#scripts)
8. [Architecture](#architecture)
9. [Workflow](#workflow)
10. [Where State Lives](#where-state-lives)
11. [Roadmap](#roadmap)
12. [FAQ](#faq)
13. [Troubleshooting](#troubleshooting)
14. [Contributing](#contributing)
15. [License](#license)
16. [Acknowledgments](#acknowledgments)
(back to top)
## About The Engine
This is a reusable orchestration toolkit for AI coding agents. It codifies agent collaboration into a gated state machine (8 forward-flow states + BLOCKED escalation) that ensures:
- **Every output is scored** via a three-tier pipeline (rule checks → LLM-as-judge → combined verdict)
- **Every decision is tracked** in the shared JSON envelope (`session/contract.json`)
- **Every session learns** from the last: lessons and patterns persist cross-session via `lean-ctx ctx_knowledge`
- **Every delegation is gated**: agents only transition if scoring thresholds are met
Reference docs beyond this README:
| Resource | Content |
|---|---|
| [`doc/workflow.md`](doc/workflow.md) + [`doc/workflow/`](doc/workflow/README.md) | Canonical state machine, gates, validation rules (5 parts) |
| [`doc/health-check.md`](doc/health-check.md) | Health check dimensions and wiring |
| [`doc/project.md`](doc/project.md) | Project vision, scope, constraints |
| [`doc/service-rules-thresholds.md`](doc/service-rules-thresholds.md) | Service rules and thresholds |
| [`doc/skill-conventions.md`](doc/skill-conventions.md) | Skill packaging conventions |
| [`contract/README.md`](contract/README.md) | Envelope schema reference |
| [`usage/lean-ctx.md`](usage/lean-ctx.md), [`usage/graphify.md`](usage/graphify.md), [`usage/context7.md`](usage/context7.md), [`usage/opencode.md`](usage/opencode.md) | Tool usage guides |
| [`rules/rules.json`](rules/rules.json) | State machine transitions, scoring thresholds, cross-model review |
### Built With
This project is a documentation and configuration toolkit for AI agents. It uses:
- [OpenCode](https://www.npmjs.com/package/@opencode/cli): AI agent platform (`@opencode/cli`, npm `anomalyco/opencode`)
- [lean-ctx](https://github.com/yvgude/lean-ctx): compressed context management + code intelligence
- [Graphify](https://pypi.org/project/graphifyy/): knowledge graph builder (`graphifyy[mcp]`, PyPI)
(back to top)
## Quick Start
Engine-level tasks: run the check suite from the project root (`workspace/agents/opencode`):
```bash
bash engine/scripts/health-check.sh # full suite, 6 dimensions
bash engine/scripts/check-cross-references.sh # link/ID integrity
bash engine/scripts/check-doc-style.sh # doc style gate
bash engine/scripts/validate-toolkit.sh # toolkit integrity
bash engine/scripts/validate-contract.sh --file engine/contract/contract.template.json --score
```
Inside a running container the same scripts live at `/opt/engine/scripts/`. Session state is seeded with:
```bash
bash /opt/engine/scripts/snapshot-contract.sh # seed/refresh session/contract.json + state.md
```
(back to top)
## Getting Started
The engine ships **inside the container image**: the Dockerfile copies `engine/` to `/opt/engine/` and symlinks it into OpenCode's config directory (`~/.config/opencode/{agent.md,agents,skills,commands,rules,contract,usage,doc}` → `/opt/engine/*`).
### Prerequisites
- **Docker**: to build and run the image
- **OpenCode**: installed in the image (baked at build time)
- **lean-ctx**: compressed file reads, shell, knowledge persistence, session management; installed in the image
### Build
The Dockerfile bundles OpenCode, lean-ctx, Graphify, and Context7, then copies the engine. It installs packages from the offline store in `pre-install/` (mounted as the `preinstall` build context), so pass that context explicitly:
```bash
docker build --build-context preinstall=./pre-install -t opencode:local .
```
`engine/` changes take effect on **rebuild**.
### Configure
Configuration is **not baked into the image**; it is bind-mounted read-only from `config/`:
```bash
# config/opencode.json is V2-native; edit in place, or start from the template:
cp engine/opencode.json.template config/opencode.json
# Edit config/opencode.json, set models/providers; API keys come from compose env
```
Mount it at `~/.config/opencode/opencode.json` when running. Config changes take effect on **restart**.
### Run
```bash
docker run --rm -it \
-e OPENCODE_API_KEY="" \
-v "$PWD/config/opencode.json":/home/node/.config/opencode/opencode.json:ro \
-p 4096:4096 \
opencode:local
```
The entrypoint configures auth (writes `OPENCODE_API_KEY` to `auth.json`, then unsets it), optionally builds the Graphify index, runs an MCP readiness preflight, optionally runs engine boot checks, then probes launch forms in order: `opencode serve --hostname 0.0.0.0 --port 4096` → fallback `opencode web --hostname 0.0.0.0 --port 4096` → fallback bare `opencode serve`.
Under compose, the host binding is `127.0.0.1:4096 → container 4096` (loopback only).
### Verify
```bash
# Toolkit integrity (symlinks, contract integrity, agent consistency, cross-references)
bash /opt/engine/scripts/validate-toolkit.sh
# Verify MCP servers
bash /opt/engine/scripts/check-mcp.sh
# Load orchestration envelope
lean-ctx ctx_knowledge recall --key "orchestration-contract" --mode exact
```
(back to top)
## State Machine
*Generated by [scripts/generate_diagrams.py](scripts/generate_diagrams.py); regenerate with `python3 scripts/generate_diagrams.py` from `engine/`.*
Any state can escalate to `BLOCKED` when a combined score lands below 50 or retries are exhausted (max 3). `BLOCKED` exits to user intervention, then retries with guidance.
| Transition | Gate | Condition |
|-----------|------|-----------|
| INIT → PLAN | Session created | Always |
| PLAN → PLAN_SCORED | Plan produced | Always |
| EXECUTE → EXECUTE_SCORED | Implementation done | Always |
| EXECUTE_SCORED → REVIEW | Code review | Score ≥ 70 |
| REVIEW → REVIEW_SCORED | Review done | Always |
| REVIEW_SCORED → COMPLETE | All gates pass | Score ≥ 70 |
| Any → BLOCKED | Escalation | Score < 50 or retry ≥ 3 |
(back to top)
## Scoring Pipeline
Three-tier scoring runs after every delegation:
| Tier | Method | Deduction / Range |
|------|--------|-------------------|
| 1 | Rule-based checks (start 100) | Schema -15, permissions -40, blast radius -40, writing order -15, required fields -15, loop gate -15 |
| 2 | LLM-as-judge | Requirements 0-40, governance 0-30, completeness 0-20, edge cases 0-10 |
| 3 | **Combined verdict** | **≥ 70 PASS** · 50-69 RETRY (max 3) · < 50 BLOCKED |
(back to top)
## Agents
16 instruction files under [`agents/`](agents/orchestrator.md). Executors and reviewers run on different models, so no agent audits its own output.
| Agent | Role | Model tier | Access |
|---|---|---|---|
| [`orchestrator`](agents/orchestrator.md) | Scrum lead: delegates, scores, drives the state machine | executor | read/write |
| [`design-orchestrator`](agents/design-orchestrator.md) | UI/UX specialist orchestrator, 4-layer design pipeline | executor | read/write |
| [`product-manager`](agents/product-manager.md) | Requirements + GWT acceptance criteria | reviewer | read-only |
| [`system-analyst`](agents/system-analyst.md) | Impact analysis, specs, implementation plans | executor | read-only |
| [`software-architect`](agents/software-architect.md) | Architecture trade-offs, tech-debt assessment | reviewer | read-only |
| [`developer`](agents/developer.md) | Implements the plan, writes tests alongside code | executor | read/write |
| [`devops-engineer`](agents/devops-engineer.md) | CI/CD, containerization, deploy, migrations, observability | executor | read/write |
| [`fixer`](agents/fixer.md) | Bounded, scoped fixes for well-defined tasks | executor | read/write |
| [`quality-analyst`](agents/quality-analyst.md) | Independent 5-axis quality review | reviewer | read-only |
| [`security-engineer`](agents/security-engineer.md) | Security review: threat model, vulns, secrets | reviewer | read-only |
| [`scrum-master`](agents/scrum-master.md) | Process owner: guards contract and state machine, unblocks BLOCKED | reviewer | read-only |
| [`learner`](agents/learner.md) | Post-execution retro: extract and persist lessons | executor | read-only |
| [`explorer`](agents/explorer.md) | Fast codebase search and unknown discovery | executor | read-only |
| [`librarian`](agents/librarian.md) | External docs and library research | executor | read-only |
| [`council`](agents/council.md) | Multi-model consensus and cross-model arbitration | multi | read-only |
| [`observer`](agents/observer.md) | Visual analysis: images, PDFs, diagrams | executor | read-only |
**Cross-model review**: executors run on the executor model; reviewers (@product-manager, @software-architect, @quality-analyst, @security-engineer, @scrum-master) run on an independent reviewer model. Conflicting verdicts → `BLOCKED`, arbitrated by @council.
(back to top)
## Scripts
Automation and validation under [`scripts/`](scripts/health-check.sh), grouped by purpose:
| Group | Script | Purpose |
|---|---|---|
| Contract and session | `scripts/snapshot-contract.sh` | Envelope snapshot and archival |
| Contract and session | `scripts/persist-contract.sh` | Atomic envelope persistence |
| Contract and session | `scripts/auto-persist.sh` | Automated contract persistence |
| Contract and session | `scripts/contract-migrate.sh` | Schema version migration |
| Contract and session | `scripts/self-repair.sh` | Self-repair for corrupt contracts |
| Contract and session | `scripts/contract-enforcer.sh` | Contract rule enforcement |
| Contract and session | `scripts/state-guard.sh` | Agent state guard verification |
| Contract and session | `scripts/session-lock.sh` | Session locking |
| Contract and session | `scripts/archive-sessions.sh` | Days-based session cleanup (dry-run/restore) |
| Scoring and metrics | `scripts/validate-contract.sh` | Contract JSON validation (+ `--score`) |
| Scoring and metrics | `scripts/auto-score.sh` | Automated 3-tier scoring |
| Scoring and metrics | `scripts/metrics-aggregator.sh` | Score and cost metrics aggregation |
| Scoring and metrics | `scripts/trend-analyzer.sh` | Score trend analysis |
| Scoring and metrics | `scripts/post-mortem.sh` | Post-mortem reports |
| Scoring and metrics | `scripts/decision-journal.sh` | Decision log |
| Quality gates | `scripts/health-check.sh` | 6-dimension health verification |
| Quality gates | `scripts/validate-toolkit.sh` | Full toolkit integrity check |
| Quality gates | `scripts/check-cross-references.sh` | 8-check link/ID integrity linter |
| Quality gates | `scripts/check-doc-style.sh` | Doc style gate (em/en dashes, AI vocabulary, placeholders) |
| Quality gates | `scripts/check-conventions.sh` | Architecture convention checking |
| Quality gates | `scripts/check-plugins.sh` | V2 plugin-path rule (rejects `file://`-to-file) |
| Quality gates | `scripts/check-mcp.sh` | MCP server verification |
| Quality gates | `scripts/check-writing-order.sh` | Writing order enforcement |
| Quality gates | `scripts/check-architecture-drift.sh` | Architecture drift detection |
| Quality gates | `scripts/detect-architecture-patterns.sh` | Architecture pattern detection |
| Quality gates | `scripts/sdd-gate.sh` | SDD spec gate enforcement |
| Quality gates | `scripts/loop-validate.sh` | In-loop executor done-gate: JSON parse + `bash -n` sweep + cross-refs; mirrors CI locally before REVIEW |
| Docs and diagrams | `scripts/generate_diagrams.py` | Regenerates `doc/diagrams/*.svg` (state machine, platform pipeline) from source |
| Boot and runtime | `scripts/boot/engine-check.sh` | Boot-time engine validation |
| Boot and runtime | `scripts/boot/mcp-check.sh` | Boot-time MCP readiness probe |
| Boot and runtime | `scripts/mcp-health.sh` | MCP health probes |
| Boot and runtime | `scripts/mcp-retry.sh` | MCP retry logic |
| Boot and runtime | `scripts/timeout-watchdog.sh` | Timeout watchdog |
| Boot and runtime | `scripts/self-healing-retry.sh` | Self-healing retry loop |
| Boot and runtime | `scripts/uncertainty-router.sh` | Uncertainty routing |
| Boot and runtime | `scripts/prototype-mode.sh` | Prototype mode |
| Boot and runtime | `scripts/scheduled-runner.sh` | Scheduled tasks (drift-detect daily) |
| Boot and runtime | `setup.sh` | Bootstrap: config symlinks, MCP + plugin checks |
| Drift and coordination | `scripts/drift-detect.sh` | Knowledge vs file drift detection |
| Drift and coordination | `scripts/detect-parallel-conflicts.sh` | Parallel shard conflict detection |
| Drift and coordination | `scripts/verify-knowledge.sh` | Knowledge persistence verification |
| Support | `scripts/state-machine.ts` | TypeScript reference implementation of the state machine |
| Support | `scripts/normalize-doc-paths.py` | Normalizes deployed doc paths |
| Support | `scripts/lib/` | Shared shell libraries (`paths.sh`, `common.sh`) |
(back to top)
## Architecture
The engine source lives under `engine/` and is copied to `/opt/engine/` at container build. Edit here, then rebuild to deploy.
```text
engine/
├── agent.md ← Orchestrator instructions (instructions[0])
├── agents/ ← 16 agent instruction files
├── skills/ ← 66 engine-owned skill directories (SKILL.md + optional package.json); plugin-bundled CLI skills register at runtime and are not counted
├── commands/ ← Slash commands (impeccable)
├── contract/ ← Contract templates & schema (contract.template.json, contract.schema.json, meta-contract.template.json, state.template.md)
├── rules/ ← rules.json + rules.schema.json (state machine, scoring, cross-model review)
├── usage/ ← Tool usage guides (lean-ctx, graphify, context7, opencode)
├── doc/ ← Workflow, project, and health-check docs
├── plugins/ ← vendored first-party plugin (@firecrawl/opencode, offline)
├── scripts/ ← automation & validation scripts (see table above)
├── templates/ ← judge-prompt.md
├── .githooks/ ← pre-commit hook (bash -n sweep + cross-references + doc style)
├── opencode.json.template ← Config template (V2-native shape)
├── package.json ← Engine metadata
└── setup.sh ← Bootstrap (config symlinks, MCP + plugin checks)
```
(back to top)
## Workflow
### Orchestration Flow
Every task follows this sequence:
| Phase | Agent | Gate |
|-------|-------|------|
| **Refine** | @product-manager | Intent → GWT acceptance criteria, scope + non-goals |
| **Discuss** | Orchestrator (5-lens check) | Requirements clarified |
| **Plan** | @system-analyst (+ @software-architect review) | Impact analysis + plan produced |
| **Spec** (gate) | @system-analyst | GWT-format specs, DDD review |
| **Execute** | @developer (+ @devops-engineer, @fixer) | Implementation + tests |
| **Review** | @quality-analyst + @security-engineer (cross-model) | Code quality, security, DevOps |
| **Process** | @scrum-master | Transitions legal, DoD met, unblock BLOCKED |
| **Verify** | Quality gates | Validated JSON, conventions check |
| **Ship** | Orchestrator (+ @learner) | Post-flight protocol, lessons captured |
The definition of done for each phase is enforced by the gates in the table above and by the scoring thresholds in `rules/rules.json`; the full rules are documented in [`doc/workflow.md`](doc/workflow.md). Retrieval is advisory across these phases: agents may consult `rag_search` for domain context, project conventions, and pre-implementation discovery, verifying hits against the source before relying on them; it is deliberately not a scoring gate.
### Frugality Ladder
Before writing **any** code, run the ladder:
```text
1. Does this need to exist? → skip it (YAGNI)
2. Standard library does it? → use it
3. Native platform feature? → use it
4. Already-installed dependency? → use it
5. Can this be one line? → one line
6. Only then: minimum code that works
```
### Key Commands
```bash
# Load/save session state
lean-ctx ctx_session save
lean-ctx ctx_session load
# Persist decisions
lean-ctx ctx_knowledge remember key value
# Check blast radius before editing (see usage/lean-ctx.md)
```
(back to top)
## Where State Lives
The engine itself is stateless. Live runtime state lives in the **project-root** `session/` directory. Compose mounts the project at `/home/node/workspace`, so that folder *is* the container's global session:
- `session/contract.json`: live orchestration envelope, seeded from [`contract/contract.template.json`](contract/contract.template.json)
- `session/state.md`: append-only state log; `state.example.md` is the tracked example
- `session/archive/`: timestamped snapshots
> **LOCAL-ONLY:** `session/`, `.session/` and everything inside them are runtime state. NEVER `git add`, commit, or push them (gitignored; history grows unbounded). Archive historical state logs to `.session/` instead.
Initialize or refresh session state with:
```bash
bash /opt/engine/scripts/snapshot-contract.sh # Seed/refresh session/contract.json + state.md
```
### Sandbox (Playground)
Scratch work from any session lives in the **project-root** `sandbox/` directory, one folder per live session (`sandbox//`): probes, clones, throwaway builds, rendered tests. The recent trend is to dump this in `/tmp` or straight into repo paths, which loses track of what belongs to which session and leaves debris that looks like product code.
- Only `README.md`, `.gitignore`, and `.gitkeep` are tracked; everything else is gitignored on the `certs/` pattern, so a session folder never reaches the repository.
- Writes are limited to your own session folder, other folders are read-only, and nothing is written at `sandbox/` root.
- Policy: [`sandbox/README.md`](../../../sandbox/README.md); machine-readable: `rules/rules.json → enforcement.sandbox` (200 MB and 7-day TTL, ports 4100-4999 on `127.0.0.1`).
- Promote anything durable to its project folder before COMPLETE.
(back to top)
## Roadmap
- [x] Core state machine (8 forward-flow states + BLOCKED)
- [x] Three-tier scoring pipeline
- [x] 16 specialized agent instructions (scrum team)
- [x] Skill packaging (66 engine-owned SKILL.md directories; `package.json` where a skill needs deps; plugin-bundled CLI skills register at runtime and are not counted)
- [x] Cross-reference linting (scripts/check-cross-references.sh: 8-check link/ID integrity + context-standards + access-policy checks, wired into health-check lens-4, validate-toolkit.sh, pre-commit hook, CI)
- [x] Doc style gate (scripts/check-doc-style.sh: mechanical em/en dash, AI vocabulary, and placeholder checks on shipped docs, wired into health-check --check docs, validate-toolkit.sh, pre-commit hook, CI)
- [x] Project-context standards layer (skills/context-standards: load target-project context/*.md rules before edits, wired into agent.md Execute step, developer.md, quality-analyst.md, cross-reference check 7)
- [x] Mandatory lean-ctx access policy (identical Access policy line in all agents/*.md, access-layer rule in agent.md, enforced by cross-reference check 8)
- [x] V2 plugin-path rule (scripts/check-plugins.sh: rejects `file://`-to-file entries, wired into setup.sh, validate-toolkit.sh, engine boot checks)
- [x] MCP verification script (scripts/check-mcp.sh)
- [x] Template configuration (opencode.json.template: V2-native shape)
- [x] Architecture convention checking (scripts/check-conventions.sh: agent paths, JSON schema, forbidden patterns, state.md sync)
- [x] Toolkit integrity validation (scripts/validate-toolkit.sh: symlinks, contract integrity, agent consistency, cross-references)
- [x] Session archival (scripts/archive-sessions.sh: days-based cleanup with dry-run/restore)
- [x] JSON Schema enforcement (contract/contract.schema.json: Draft 2020-12, $defs, numeric constraints, valid states)
- [x] CI validation workflow (repository-root .github/workflows/validate.yml: JSON validity, `bash -n` sweep, cross-references, doc style gate, contract scoring, non-blocking conventions)
- [x] Multi-model council consensus for high-stakes decisions
- [x] Parallel shard execution with conflict reconciliation
- [x] Automated contract persistence (scripts/auto-persist.sh)
- [x] Automated 3-tier scoring pipeline (scripts/auto-score.sh)
- [x] Agent state guard verification (scripts/state-guard.sh)
- [x] Self-repair for corrupt contracts (scripts/self-repair.sh)
- [x] Knowledge vs file drift detection (scripts/drift-detect.sh)
- [x] SDD spec gate enforcement (scripts/sdd-gate.sh)
- [x] Knowledge persistence verification (scripts/verify-knowledge.sh)
- [x] Cross-session learning feedback loop (agents/orchestrator.md, agents/learner.md)
- [x] Session-scoped sandbox (project-root `sandbox/`, one folder per session, gitignored contents, `enforcement.sandbox` policy)
- [ ] Skill registry publishing
(back to top)
## FAQ
**Why is the UI loopback-only?**
Compose publishes host `127.0.0.1:4096` → container 4096, so the UI never exposes beyond loopback. Inside the container OpenCode always binds 4096.
**Why were the custom plugins removed?**
They target the V1 plugin API, and V2 rejects their `file://` entries (`configured plugin path must be a directory`); they never loaded. They are dropped until ported to the `@opencode/plugin` V2 API; `scripts/check-plugins.sh` now enforces the no-`file`-to-`file` rule at setup, toolkit validation, and engine boot.
**How do I add a skill?**
Create a directory under `engine/skills/` containing a `SKILL.md` with frontmatter (`name`, `description`) and rebuild the image; `skills/` is symlinked into the OpenCode config directory and the skill is picked up automatically. Add a `package.json` only if the skill needs dependency metadata (28 of the current 66 have one). Conventions: [`doc/skill-conventions.md`](doc/skill-conventions.md).
**Where do project pattern standards live?**
In the target project, not here. A `context/` directory at the project root with `*.md` files is discovered and loaded via `/skill context-standards` before edits, and review verifies the files were applied. The engine itself ships no style defaults.
**Where does RAG fit in the engine?**
As an advisory discovery layer over workspace code, docs, and engine skills, wired via the rag MCP server and the agent-facing config instructions (`config/rag-usage.md`). Agents consult it for conventions and unfamiliar-area discovery, then verify hits against the source. It is deliberately not a scoring gate: forced retrieval on every step would anchor agents on possibly stale snippets.
**Why keep the fetch MCP when Firecrawl exists?**
Different tools. `fetch` is a keyless, single-page HTTP GET with no index; Firecrawl adds an indexed primary-source search (`firecrawl_developer_search`, also keyless) plus keyed scrape/crawl/map/alexandria through the vendored plugin and `firecrawl-cli`. `fetch` stays as the zero-dependency fallback for one-off pages.
**Why does `session/` live inside the project?**
Compose mounts the project at `/home/node/workspace`, making project-root `session/` the container's global session directory. `session/contract.json` is seeded from [`contract/contract.template.json`](contract/contract.template.json). See [Where State Lives](#where-state-lives).
**Where do I put scratch files?**
`sandbox//` at the project root, never `/tmp` and never a repo path. The folder is tracked so the convention is visible, its contents are gitignored, and the rules live in [`sandbox/README.md`](../../../sandbox/README.md). See [Sandbox (Playground)](#sandbox-playground).
**What breaks without `pre-install/`?**
The Dockerfile installs apt/pip/npm/git packages from the store mounted as the `preinstall` build context. With an empty or missing store, installs fall back to the network, which fails in offline/air-gapped environments. Run the preinstall compose service first.
(back to top)
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| Container crash-loops at boot | `OPENCODE_API_KEY` missing while `OPENCODE_AUTH_REQUIRED=true`; entrypoint exits 1 | Export a real key before `up`; or set `OPENCODE_AUTH_REQUIRED=false` for keyless testing |
| `MCP FAIL ` (executable `` not found) at boot | The server's `command` in `config/opencode.json` points at a binary not installed in the image | Fix the command or add the package to the Dockerfile; `MCP_CHECK_STRICT=false` (default) continues degraded |
| Preinstall service finishes instantly, skips everything | Store already populated; repeat runs skip anything present | Expected behavior, not a bug |
| Wrong-architecture build failures | `pre-install/` store tarballs match `OPENCODE_ARCH` (default `arm64`) | Override `OPENCODE_ARCH` (e.g. `x64`) and re-run the preinstall service for that arch |
| Boot log: "continuing without a Graphify graph" | By design: `GRAPHIFY_BUILD=if-missing` skips when the graph exists; build failures/timeouts are warnings, boot continues | Set `GRAPHIFY_BUILD=always` to force a rebuild; raise `GRAPHIFY_TIMEOUT` (default 900) if builds time out |
| `firecrawl_developer_search` refused on VPN, proxy, or cloud networks | Keyless developer search can be blocked on some networks | Set `FIRECRAWL_API_KEY` in the environment, or connect a key via `/connect` in the TUI |
| Every `firecrawl_alexandria` run asks for confirmation | By design: the config permission sets `effect: ask` for alexandria | Confirm per run, or change the permission to `allow` in `config/opencode.json` if your policy allows |
(back to top)
## Contributing
The engine is developed in place in the [utility repository](https://github.com/RizkiRachman/utility); `engine/` changes deploy on image rebuild.
Gates that apply to every change:
1. The repository-root `.github/workflows/validate.yml` runs on push and pull request (JSON validity, `bash -n`, cross-references, doc style, contract score, non-blocking conventions)
2. `engine/.githooks/pre-commit` gates each commit (enable: `git config core.hooksPath engine/.githooks`)
3. The PR template structures the description
Before handing off any change, run:
```bash
bash engine/scripts/validate-toolkit.sh
```
(back to top)
## License
Internal workflow tooling. No license file ships with the repository; contents are proprietary and for internal use.
(back to top)
## Acknowledgments
- [Best-README-Template](https://github.com/othneildrew/Best-README-Template) by [@othneildrew](https://github.com/othneildrew): README structure
- [Img Shields](https://shields.io): badge generation
(back to top)
================
File: workspace/agents/opencode/engine/setup.sh
================
#!/usr/bin/env bash
# setup.sh — Create .opencode/ symlinks for flat project structure
# Run once on fresh clone: bash setup.sh
set -euo pipefail
# ── Pre-flight checks ──────────────────────────────────────────
echo "Running pre-flight checks..."
FAILED=0
# Bash >= 4 (for associative arrays used by some scripts)
if (( BASH_VERSINFO[0] < 4 )); then
echo "ERROR: bash >= 4 required (found $BASH_VERSION). Install bash 4+ and re-run."
FAILED=1
fi
# Node.js (required for OpenCode tooling)
if ! command -v node &>/dev/null; then
echo "WARNING: node not found. OpenCode tooling may not work."
echo " Install from https://nodejs.org/ or via package manager."
fi
# lean-ctx (recommended for file management)
if ! command -v lean-ctx &>/dev/null; then
echo "WARNING: lean-ctx not found. Some toolkit features will be unavailable."
echo " Install from: https://github.com/yvgude/lean-ctx"
fi
if [ "$FAILED" -eq 1 ]; then
echo "Pre-flight checks FAILED. Fix errors above and re-run."
exit 1
fi
echo "Pre-flight checks passed."
# ── End pre-flight checks ──────────────────────────────────────
echo ""
echo "── Tool Verification ──────────────────────────────────────"
source "$(dirname "$0")/scripts/check-mcp.sh" || FAILED=1
source "$(dirname "$0")/scripts/check-plugins.sh" || FAILED=1
echo "────────────────────────────────────────────────────────────"
if [ "$FAILED" -eq 1 ]; then
echo ""
echo "=============================="
echo " MISSING MANDATORY TOOLS"
echo "=============================="
echo " Install the missing tools listed above and re-run setup.sh."
exit 1
fi
echo ""
cd "$(pwd)"
echo "Creating .opencode/ symlinks..."
mkdir -p .opencode
ln -sfn ../agents .opencode/agents
ln -sfn ../skills .opencode/skills
ln -sfn ../rules .opencode/rules
ln -sfn ../contract .opencode/orchestration
ln -sfn ../usage .opencode/usage
ln -sfn ../agent.md .opencode/AGENTS.md
# ── Post-install validation ────────────────────────────────────
echo "Validating symlinks..."
VALIDATION_FAILED=0
# Expected symlinks: target -> source
declare -A SYMLINKS=(
[".opencode/agents"]="../agents"
[".opencode/skills"]="../skills"
[".opencode/rules"]="../rules"
[".opencode/orchestration"]="../contract"
[".opencode/usage"]="../usage"
[".opencode/AGENTS.md"]="../agent.md"
)
for target in "${!SYMLINKS[@]}"; do
if [ ! -L "$target" ]; then
echo "FAIL: $target is not a symlink"
VALIDATION_FAILED=1
elif [ "$(readlink "$target")" != "${SYMLINKS[$target]}" ]; then
echo "FAIL: $target points to $(readlink "$target") (expected ${SYMLINKS[$target]})"
VALIDATION_FAILED=1
else
echo " OK: $target -> $(readlink "$target")"
fi
done
if [ "$VALIDATION_FAILED" -eq 1 ]; then
echo ""
echo "WARNING: Some symlinks are incorrect or missing. Re-run setup.sh or fix manually."
echo "Expected layout:"
echo " .opencode/agents -> ../agents"
echo " .opencode/skills -> ../skills"
echo " .opencode/rules -> ../rules"
echo " .opencode/orchestration -> ../contract"
echo " .opencode/usage -> ../usage"
echo " .opencode/AGENTS.md -> ../agent.md"
exit 1
fi
echo "All symlinks valid."
echo ""
echo "NOTE: .githooks/pre-commit stays dormant unless the workspace is a git repo;"
echo " hooks activate only when the target directory is a git repository."
echo ""
echo "Setup complete. The toolkit is ready."
================
File: workspace/agents/opencode/pre-install/apt/.gitkeep
================
================
File: workspace/agents/opencode/pre-install/bin/.gitkeep
================
================
File: workspace/agents/opencode/pre-install/dist/.gitkeep
================
================
File: workspace/agents/opencode/pre-install/git/.gitkeep
================
================
File: workspace/agents/opencode/pre-install/npm/.gitkeep
================
================
File: workspace/agents/opencode/pre-install/pip/.gitkeep
================
================
File: workspace/agents/opencode/rag/test/corpus/api/users.md
================
# Users API
The users API lists accounts and their roles for the admin console.
## List users
GET /api/users returns every account with an active flag and the
account creation date.
## Rename a user
POST /api/users/:id/rename accepts a JSON body with a name field
and returns the updated account record.
================
File: workspace/agents/opencode/rag/test/corpus/src/main/java/com/example/UserService.java
================
package com.example;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Optional;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
/**
* User management service for the demo corpus. Keeps accounts, registers
* new users, and runs batch maintenance passes.
*/
@Service
@Transactional
public class UserService {
private final UserRepository repository;
private final AuditLog auditLog;
public UserService(UserRepository repository, AuditLog auditLog) {
this.repository = repository;
this.auditLog = auditLog;
}
public Optional findByEmail(String email) {
if (email == null || email.isBlank()) {
return Optional.empty();
}
return repository.findAll().stream()
.filter(u -> u.getEmail().equalsIgnoreCase(email))
.findFirst();
}
public User registerUser(String email, String displayName) {
String normalized = email.trim().toLowerCase(Locale.ROOT);
if (normalized.isEmpty()) {
throw new IllegalArgumentException("email must not be blank");
}
if (findByEmail(normalized).isPresent()) {
throw new IllegalStateException("email already registered: " + normalized);
}
User user = new User();
user.setEmail(normalized);
user.setDisplayName(displayName == null || displayName.isBlank()
? normalized.substring(0, normalized.indexOf('@'))
: displayName.trim());
user.setActive(true);
repository.save(user);
auditLog.record("register", user.getEmail());
return user;
}
public List processBatch(List emails) {
List processed = new ArrayList<>();
for (String raw : emails) {
if (raw == null || raw.isBlank()) {
continue;
}
// The marmaladeDeduplication pass keeps repeated addresses out of
// the batch so downstream sends never double-fire on one account.
String email = raw.trim().toLowerCase(Locale.ROOT);
if (processed.stream().anyMatch(u -> u.getEmail().equals(email))) {
auditLog.record("marmaladeDeduplication-skip", email);
continue;
}
Optional existing = findByEmail(email);
User user;
if (existing.isPresent()) {
user = existing.get();
if (!user.isActive()) {
user.setActive(true);
auditLog.record("reactivate", email);
}
} else {
user = registerUser(email, null);
auditLog.record("batch-register", email);
}
processed.add(user);
auditLog.record("batch-visit", email);
auditLog.record("batch-checkpoint", email);
auditLog.record("batch-verify", email);
auditLog.record("batch-summary", email);
auditLog.record("batch-note", email);
}
auditLog.record("batch-done", String.valueOf(processed.size()));
return processed;
}
}
================
File: workspace/agents/opencode/rag/test/corpus/utils/format.js
================
// Format helpers for report output.
function padLeft(value, width) {
return String(value).padStart(width, ' ');
}
function truncate(text, limit) {
if (text.length <= limit) return text;
return text.slice(0, limit - 1) + '.';
}
function joinFields(fields) {
return fields.map(function (f) { return padLeft(f, 10); }).join(' ');
}
module.exports = { padLeft: padLeft, truncate: truncate, joinFields: joinFields };
================
File: workspace/agents/opencode/rag/test/corpus/getting-started.md
================
# Getting Started
Welcome to the demo workspace. This guide walks through the
quantum marmalade pipeline that turns raw notes into indexed chunks.
## Installation
Run the setup script from the repository root. The setup script
copies configuration files and creates the data directories used
by the index.
## First search
Call the search command with any phrase. The quantum marmalade
pipeline ranks markdown sections above plain text because every
chunk carries its heading path.
================
File: workspace/agents/opencode/rag/test/java-check.js
================
'use strict';
/*
* java-check.js — assertion helper for rag/test.sh case 9.
* Opens the rag SQLite database given as argv[1] and validates the Java
* chunking invariants on UserService.java:
* - the file produced at least two chunks
* - headings carry package and class context (package com.example; > UserService)
* - the long method was hard split into more than one chunk
* - the distinctive identifier is present in some chunk text
* Prints the first processBatch chunk id as "CHUNK_ID=" so the caller
* can exercise the rag_get path on it. Exits 0 when all assertions hold.
*/
const { DatabaseSync } = require('node:sqlite');
const dbPath = process.argv[2];
if (!dbPath) {
console.error('usage: node java-check.js ');
process.exit(1);
}
const db = new DatabaseSync(dbPath);
const rows = db.prepare(
'SELECT c.id, c.heading, c.start_line, c.end_line, c.text ' +
'FROM chunks c JOIN files f ON f.id = c.file_id ' +
"WHERE f.path LIKE '%UserService.java' ORDER BY c.ordinal"
).all();
db.close();
let ok = true;
const check = (label, cond) => {
console.log(`CHECK ${label}: ${cond ? 'ok' : 'FAIL'}`);
if (!cond) ok = false;
};
check('UserService.java has at least 2 chunks', rows.length >= 2);
check('some heading carries package com.example;',
rows.some((r) => r.heading.includes('package com.example;')));
check('some heading carries the class name UserService',
rows.some((r) => /(^|> )UserService( |$)/.test(r.heading)));
const batch = rows.filter((r) => r.heading.includes('processBatch'));
check('long method hard split (>=2 processBatch chunks)', batch.length >= 2);
check('full heading shape pkg > class > method',
rows.some((r) => /^package com\.example; > UserService > \w/.test(r.heading)));
check('marmaladeDeduplication present in chunk text',
rows.some((r) => r.text.includes('marmaladeDeduplication')));
if (batch.length) console.log(`CHUNK_ID=${batch[0].id}`);
process.exit(ok ? 0 : 1);
================
File: workspace/agents/opencode/rag/test/mcp-check.js
================
'use strict';
/*
* mcp-check.js — assertion helper for rag/test.sh case 6.
* Reads the JSON-RPC response lines produced by `node server.js` (MCP stdio
* mode) from the file given as argv[1] and validates the handshake:
* id 1: initialize -> serverInfo.name === "rag"
* id 2: tools/list -> exactly 4 tools: rag_search, rag_index, rag_status, rag_get
* id 3: tools/call rag_search -> hits include getting-started.md
* Exits 0 when every assertion holds, 1 otherwise. Prints one CHECK line each.
*/
const fs = require('node:fs');
const file = process.argv[2];
if (!file) {
console.error('usage: node mcp-check.js ');
process.exit(1);
}
const lines = fs.readFileSync(file, 'utf8').split('\n').filter((l) => l.trim());
const byId = new Map();
for (const line of lines) {
let msg;
try {
msg = JSON.parse(line);
} catch (e) {
console.log(`CHECK non-JSON stdout line: FAIL (${e.message})`);
process.exit(1);
}
if (msg && msg.id !== undefined && msg.id !== null) byId.set(msg.id, msg);
}
let ok = true;
const check = (label, cond) => {
console.log(`CHECK ${label}: ${cond ? 'ok' : 'FAIL'}`);
if (!cond) ok = false;
};
const init = byId.get(1);
check('initialize serverInfo.name == rag', !!(init && init.result && init.result.serverInfo
&& init.result.serverInfo.name === 'rag'));
const list = byId.get(2);
const toolNames = list && list.result && Array.isArray(list.result.tools)
? list.result.tools.map((t) => t.name) : [];
check('tools/list has 4 tools', toolNames.length === 4);
for (const t of ['rag_search', 'rag_index', 'rag_status', 'rag_get']) {
check(`tools/list includes ${t}`, toolNames.includes(t));
}
const call = byId.get(3);
let hits = [];
if (call && call.result && call.result.structuredContent && Array.isArray(call.result.structuredContent.hits)) {
hits = call.result.structuredContent.hits;
} else if (call && call.result && Array.isArray(call.result.content)) {
const text = call.result.content.map((c) => c.text || '').join('\n');
hits = text.includes('getting-started.md') ? [{ path: 'getting-started.md' }] : [];
}
check('tools/call rag_search hits include getting-started.md',
hits.some((h) => String(h.path).includes('getting-started.md')));
process.exit(ok ? 0 : 1);
================
File: workspace/agents/opencode/rag/test/upstream-embeddings-stub.js
================
'use strict';
/*
* upstream-embeddings-stub.js — OpenAI-compatible /v1/embeddings stub for the
* rag test suite. Deterministic 8-dim vectors: for each input text, s = sum of
* char code points, value[i] = ((s + i) % 7) / 7.
*
* Endpoints:
* GET /healthz -> 200 ok (readiness probe)
* GET /last-auth -> last seen Authorization header value (default "")
* POST /v1/embeddings -> { data: [{ embedding: [8 floats] }, ...] }
* response header x-auth-ok: "true"/"false"
*/
const http = require('node:http');
const PORT = Number(process.env.STUB_PORT || 4599);
let lastAuth = '';
function embed(text) {
let s = 0;
for (const ch of String(text)) s += ch.codePointAt(0);
const vec = [];
for (let i = 0; i < 8; i++) vec.push(((s + i) % 7) / 7);
return vec;
}
const server = http.createServer((req, res) => {
if (req.method === 'GET' && req.url === '/healthz') {
res.writeHead(200, { 'content-type': 'text/plain' });
res.end('ok');
return;
}
if (req.method === 'GET' && req.url === '/last-auth') {
res.writeHead(200, { 'content-type': 'text/plain' });
res.end(lastAuth);
return;
}
if (req.method === 'POST' && req.url === '/v1/embeddings') {
lastAuth = String(req.headers.authorization || '');
let body = '';
req.on('data', (c) => { body += c; });
req.on('end', () => {
let input = [];
try {
const parsed = JSON.parse(body);
if (Array.isArray(parsed.input)) input = parsed.input;
} catch (e) {
res.writeHead(400, { 'content-type': 'application/json' });
res.end(JSON.stringify({ error: { message: 'bad JSON body' } }));
return;
}
const authOk = lastAuth === 'Bearer testkey';
res.writeHead(200, {
'content-type': 'application/json',
'x-auth-ok': String(authOk),
});
res.end(JSON.stringify({ data: input.map((t) => ({ embedding: embed(t) })) }));
});
return;
}
res.writeHead(404, { 'content-type': 'text/plain' });
res.end('not found');
});
server.listen(PORT, '127.0.0.1', () => {
console.log(`embeddings stub listening on 127.0.0.1:${PORT}`);
});
================
File: workspace/agents/opencode/rag/README.md
================
# rag
Retrieval-Augmented Generation MCP server for the opencode workspace: it indexes the workspace tree into SQLite (FTS5 keyword index plus Float32 vectors) and serves fused keyword + vector search to the agent. One file, `server.js`, zero npm dependencies, node core modules only.
It runs in two modes from the same script:
| Mode | Invocation | Transport |
|---|---|---|
| MCP server | `node /opt/rag/server.js` (no args) | JSON-RPC 2.0 over stdio, newline-delimited; only JSON-RPC messages go to stdout |
| CLI | `node server.js ` | Human-readable stdout, for indexing and inspection from a shell |
## Tools
| Tool | Arguments | Returns |
|---|---|---|
| `rag_search` | `query` (string, required), `k` (number, default 8), `glob` (string, e.g. `docs/**/*.md`) | Ranked hits as `path:startline-endline (score) heading` plus a snippet per hit; `structuredContent` carries the same hits as JSON |
| `rag_index` | `root` (string, optional) | Incremental reindex stats: files, re-chunked, purged, chunks, skipped binary/extension/name counts, dims, mode, elapsed |
| `rag_status` | (none) | JSON status: db path, root, file and chunk counts, last index time, stale file count, embedding mode, dims |
| `rag_get` | `id` (number, required) | Full chunk text with path, heading, and line range |
Search scoring fuses an FTS5 bm25 rank (weight 0.6) with cosine similarity over the vector table (weight 0.4). Candidates are the FTS top hits plus a full scan while the corpus stays under 2000 chunks; above that, the FTS top 200. A glob pattern, if given, filters candidate paths before ranking.
## CLI usage
```bash
node server.js index [rootPath] # incremental index, prints stats
node server.js search "quantum marmalade" --k 5 --glob "docs/**/*.md"
node server.js status # db path, counts, stale files, embedding mode
node server.js get 42 # full chunk text by id
```
Indexing is incremental: every file is hash checked, unchanged files are skipped, changed files are re-chunked, and files deleted from disk are purged along with their chunks, FTS rows, and vectors.
## Embeddings
| Variable | Default | Description |
|---|---|---|
| `RAG_EMBEDDINGS_URL` | (empty) | OpenAI-compatible `/v1/embeddings` endpoint; set it together with the model to leave local mode |
| `RAG_EMBEDDINGS_MODEL` | (empty) | Embeddings model id sent in the request body |
| `RAG_EMBEDDINGS_KEY` | (empty) | Bearer key sent as the `Authorization` header |
| `RAG_DB` | `~/.local/share/opencode/rag/rag.db` | SQLite database file (parent directory is created) |
Default mode is `local`: deterministic hashed n-gram vectors (word unigrams and bigrams plus char 4-grams, fnv1a-hashed into 256 buckets, L2 normalized). It needs no network, no model, and produces identical vectors on every run. With the three `RAG_EMBEDDINGS_*` variables set, indexing batches texts (32 per request) to the endpoint and stores its vectors instead; if a request fails mid-run, the run falls back to local and says so on stderr. The search side uses whichever mode the index was built with, and in remote mode embeds the query text against the same endpoint.
## Storage
One SQLite database (WAL mode) at `RAG_DB`:
| Table | Purpose |
|---|---|
| `files` | One row per indexed file: absolute path, sha256 hash, size, mtime, indexed time |
| `chunks` | One row per chunk: file id, ordinal, heading path, start and end line, text |
| `fts` | FTS5 shadow of chunk text (`porter unicode61` tokenizer) |
| `vectors` | Float32 little-endian blob per chunk |
| `meta` | `dims`, `embedding_mode`, `last_index_at`, indexed root |
## Indexing rules
Markdown, Java, and everything else each get their own chunker. Java files (`.java`) are split on method boundaries: a brace-depth scan (line comments, string literals, and char literals are respected; block comments are not tracked) closes a chunk when depth returns to the class-body level and at least 400 characters have accumulated, and any chunk above about 1400 characters is hard split at the next line boundary. Headings carry `package > class > method` context (best effort: constructors use the class name, and files without recognizable braces fall back to plain-text chunking). Chunk text stays a raw slice of the file; the heading is metadata only, same as markdown.
| Rule | Value |
|---|---|
| Markdown chunking | Split on headings; each chunk keeps its heading path |
| Java chunking | Split on method boundaries; heading carries `package > class > method` |
| Other code and plain text | Split on blank-line boundaries, about 1200 chars per chunk with about 150 chars of overlap |
| File cap | Files above 512 KB are skipped |
| Binary detection | First bytes scanned for a null byte; extension skips run after it |
| Skipped directories | `.git`, `node_modules`, `__pycache__`, `.next`, `dist`, `pre-install`, `graphify-out`, `session`, `.session`, `.venv`, and build outputs: `target`, `build`, `.gradle`, `.mvn`, `out`, `.settings`, `.idea`, `bin`, `.classpath` |
| Skipped filenames | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `gradle.lockfile`, `.DS_Store`, `Thumbs.db` |
| Skipped extensions | Archives (`jar`, `war`, `zip`, `tar`, `gz`, ...), compiled artifacts (`class`, `exe`, `dll`, `so`, `dylib`, ...), images, fonts, office and media files; `svg` stays indexed |
| Default root | `/home/node/workspace` |
## Offline guarantees
No runtime downloads: the server is a single node script in the image (`/opt/rag/server.js`) and compose bind-mounts the same directory read-only so edits activate without a rebuild. Local hashed-vector mode does no network I/O at all, and the offline pre-install store needs nothing for rag. The optional remote embeddings endpoint is the only network path, and it is used only when its three variables are set. Nothing in the opencode boot chain depends on rag: a missing or failed index leaves every other server untouched.
## Engine integration
The engine wires rag in as an advisory discovery layer. Touchpoints:
- [`../engine/agent.md`](../engine/agent.md): the Retrieval Balance section in the tool usage reference, including the guardrails against citing unverified snippets.
- [`../engine/agents/developer.md`](../engine/agents/developer.md), [`../engine/agents/explorer.md`](../engine/agents/explorer.md), [`../engine/agents/system-analyst.md`](../engine/agents/system-analyst.md): convention lookup before non-trivial edits, conceptual first-pass discovery, and a complement to impact analysis.
- [`../engine/doc/skill-conventions.md`](../engine/doc/skill-conventions.md): the paragraph that routes skill users to the index for project-specific facts.
- [`../config/rag-usage.md`](../config/rag-usage.md): agent-facing usage instructions.
Advisory only: no scoring gate, no mandatory tool call. Hits are anchors; open the cited file region before relying on them.
================
File: workspace/agents/opencode/rag/server.js
================
'use strict';
/*
* rag — Retrieval-Augmented Generation MCP server for the opencode workspace.
* Zero npm dependencies: node core modules only (node:sqlite, node:http via
* global fetch, node:crypto, node:fs, node:path, node:os).
*
* Chunking is language aware where it is cheap: markdown splits on headings,
* Java splits on method boundaries (brace depth, headings carry
* package > class > method), everything else splits on paragraph breaks.
* Build output directories (target/, build/, .gradle/, ...), lockfiles, and
* common binary extensions are excluded from the walk.
*
* Dual mode:
* CLI — `node server.js index [root] | search [--k N] [--glob P] | status | get `
* MCP — no args: JSON-RPC 2.0 over stdio (newline-delimited). Only JSON-RPC
* messages go to stdout; all diagnostics go to stderr.
*
* Storage: SQLite (node:sqlite, WAL) with FTS5 for keyword ranking plus
* Float32 vectors for cosine similarity. Embeddings default to deterministic
* local hashed n-gram vectors (fully offline); set RAG_EMBEDDINGS_URL and
* RAG_EMBEDDINGS_MODEL to upgrade to an OpenAI-compatible /v1/embeddings
* endpoint.
*/
// Attach a warning listener BEFORE loading node:sqlite so the
// ExperimentalWarning is captured instead of printed to stderr.
process.on('warning', (w) => {
if (w && w.name === 'ExperimentalWarning') return;
console.error(w && w.stack ? w.stack : String(w));
});
const fs = require('node:fs');
const path = require('node:path');
const os = require('node:os');
const crypto = require('node:crypto');
const { DatabaseSync } = require('node:sqlite');
// ---------------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------------
const SERVER_NAME = 'rag';
const SERVER_VERSION = '1.0.0';
const DEFAULT_ROOT = '/home/node/workspace';
const DEFAULT_DB = path.join(os.homedir(), '.local/share/opencode/rag/rag.db');
const MAX_FILE_SIZE = 512 * 1024;
const CHUNK_TARGET = 1200; // chars per chunk (paragraph accumulation)
const CHUNK_OVERLAP = 150; // chars carried into the next chunk
const CHUNK_HARD_SPLIT = 4000; // single paragraph above this gets windowed
const JAVA_CHUNK_MIN = 400; // closed member below this merges into the next chunk
const JAVA_CHUNK_HARD = 1400; // java chunk above this is split at the next line
const LOCAL_DIMS = 256; // hashed n-gram vector width
const EMBED_BATCH = 32; // max texts per remote embeddings request
const FTS_TOP = 50; // fts rows joined into the candidate set
const FTS_TOP_LARGE = 200; // fts rows when the corpus is too big to scan
const FULL_SCAN_LIMIT = 2000; // full vector scan below this chunk count
const FUSE_KEYWORD = 0.6; // bm25 rank weight
const FUSE_VECTOR = 0.4; // cosine weight
const SNIPPET_CHARS = 200;
const SKIP_DIRS = new Set([
'.git', 'node_modules', '__pycache__', '.next', 'dist',
'pre-install', 'graphify-out', 'session', '.session', '.venv',
// Java / JVM build outputs and IDE metadata
'target', 'build', '.gradle', '.mvn', 'out', '.settings', '.idea', 'bin',
'.classpath',
]);
// Exact filename matches: dependency lockfiles and OS metadata never index.
const NAME_SKIPS = new Set([
'package-lock.json', 'yarn.lock', 'pnpm-lock.yaml', 'gradle.lockfile',
'.DS_Store', 'Thumbs.db',
]);
// Extensions that are never worth indexing even when they decode as text.
// Checked after the null-byte binary sniff, so genuinely binary formats
// still count as skipped_binary. svg stays indexed (it is text).
const EXT_SKIPS = new Set([
'jar', 'war', 'zip', 'tar', 'gz', 'tgz', '7z', 'rar',
'class', 'exe', 'dll', 'so', 'dylib', 'a', 'o', 'obj',
'png', 'jpg', 'jpeg', 'gif', 'ico',
'pdf', 'doc', 'docx', 'xls', 'xlsx', 'ppt', 'pptx',
'woff', 'woff2', 'ttf', 'eot', 'otf',
'mp3', 'mp4', 'mov', 'avi',
'sqlite', 'db', 'wasm', 'node', 'pyc', 'pyo', 'bin', 'iso', 'img',
]);
// ---------------------------------------------------------------------------
// Small helpers
// ---------------------------------------------------------------------------
function logErr(msg) {
console.error(`[rag] ${msg}`);
}
function tokenize(text) {
return String(text || '').toLowerCase().split(/[^a-z0-9]+/).filter(Boolean);
}
function fnv1a(str) {
let h = 0x811c9dc5;
for (let i = 0; i < str.length; i++) {
h ^= str.charCodeAt(i);
h = Math.imul(h, 0x01000193);
}
return h >>> 0;
}
function sha256(buf) {
return crypto.createHash('sha256').update(buf).digest('hex');
}
function expandTilde(p) {
if (p === '~') return os.homedir();
if (p.startsWith('~/')) return path.join(os.homedir(), p.slice(2));
return p;
}
function nowIso(epoch) {
const ms = epoch === undefined ? Date.now() : epoch;
return new Date(ms).toISOString();
}
function snippetOf(text) {
return String(text || '').replace(/\s+/g, ' ').trim().slice(0, SNIPPET_CHARS);
}
/**
* Simple glob matcher supporting * (within a path segment), ** (across
* segments) and ? (single character). Anchored on the whole relative path.
*/
function globToRegExp(pattern) {
let re = '^';
for (let i = 0; i < pattern.length; i++) {
const c = pattern[i];
if (c === '*') {
if (pattern[i + 1] === '*') { re += '.*'; i++; } else { re += '[^/]*'; }
} else if (c === '?') {
re += '[^/]';
} else {
re += c.replace(/[.+^${}()|[\]\\]/g, '\\$&');
}
}
return new RegExp(re + '$');
}
// ---------------------------------------------------------------------------
// Embeddings
// ---------------------------------------------------------------------------
/**
* Local deterministic hashed n-gram embedding: word unigrams + bigrams plus
* char 4-grams of each boundary-padded word, hashed into D buckets with
* fnv1a, L2 normalized. No I/O, stable across runs and machines.
*/
function localEmbed(text, dims) {
const vec = new Float32Array(dims);
const tokens = tokenize(text);
const feats = [];
for (let i = 0; i < tokens.length; i++) {
const w = tokens[i];
feats.push(w);
if (i + 1 < tokens.length) feats.push(`${w}_${tokens[i + 1]}`);
const padded = `#${w}#`;
for (let j = 0; j + 4 <= padded.length; j++) feats.push(padded.slice(j, j + 4));
}
for (const f of feats) vec[fnv1a(f) % dims] += 1;
let norm = 0;
for (let i = 0; i < dims; i++) norm += vec[i] * vec[i];
norm = Math.sqrt(norm);
if (norm > 0) for (let i = 0; i < dims; i++) vec[i] /= norm;
return vec;
}
function embeddingsConfig() {
const url = process.env.RAG_EMBEDDINGS_URL;
const model = process.env.RAG_EMBEDDINGS_MODEL;
if (!url || !model) return null;
return { url, model, key: process.env.RAG_EMBEDDINGS_KEY || '' };
}
async function embedBatchRemote(texts, cfg) {
const res = await fetch(cfg.url, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${cfg.key}`,
},
body: JSON.stringify({ model: cfg.model, input: texts }),
signal: AbortSignal.timeout(30000),
});
if (!res.ok) throw new Error(`embeddings endpoint returned HTTP ${res.status}`);
const body = await res.json();
const data = body && body.data;
if (!Array.isArray(data) || data.length !== texts.length) {
throw new Error('embeddings endpoint returned an unexpected shape');
}
return data.map((d) => {
if (!Array.isArray(d.embedding) || d.embedding.length === 0) {
throw new Error('embeddings endpoint returned an empty embedding');
}
return Float32Array.from(d.embedding);
});
}
/** Embed all texts remotely, sequentially in batches of EMBED_BATCH. */
async function embedAllRemote(texts, cfg) {
const out = [];
for (let i = 0; i < texts.length; i += EMBED_BATCH) {
const batch = texts.slice(i, i + EMBED_BATCH);
out.push(...await embedBatchRemote(batch, cfg));
}
return out;
}
async function embedQueryRemote(text, cfg) {
const [vec] = await embedBatchRemote([text], cfg);
return vec;
}
function vecToBlob(vec) {
return Buffer.from(vec.buffer, vec.byteOffset, vec.byteLength);
}
function blobToVec(buf) {
const ab = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength);
return new Float32Array(ab);
}
function cosine(a, b) {
let dot = 0;
let na = 0;
let nb = 0;
const n = Math.min(a.length, b.length);
for (let i = 0; i < n; i++) {
dot += a[i] * b[i];
na += a[i] * a[i];
nb += b[i] * b[i];
}
if (na === 0 || nb === 0) return 0;
return dot / Math.sqrt(na * nb);
}
// ---------------------------------------------------------------------------
// Chunking
// ---------------------------------------------------------------------------
/**
* Windowed hard split for oversized single paragraphs: ~CHUNK_TARGET char
* windows with CHUNK_OVERLAP char overlap, cutting at whitespace when close
* to the window edge.
*/
function splitLongParagraph(text, startLine) {
const out = [];
let pos = 0;
let line = startLine;
while (pos < text.length) {
let end = Math.min(pos + CHUNK_TARGET, text.length);
if (end < text.length) {
const cut = text.lastIndexOf(' ', end);
if (cut > pos + CHUNK_TARGET - 200) end = cut;
}
const slice = text.slice(pos, end);
const nl = (slice.match(/\n/g) || []).length;
out.push({ text: slice, startLine: line, endLine: line + nl });
line += nl;
if (end >= text.length) break;
const next = end - CHUNK_OVERLAP;
pos = next > pos ? next : end;
}
return out;
}
/** Split plain text / code on blank-line boundaries, ~1200 chars, ~150 overlap. */
function chunkPlainText(text) {
const paras = [];
let cur = null;
const lines = text.split('\n');
lines.forEach((ln, i) => {
const n = i + 1;
if (ln.trim() === '') {
if (cur) { paras.push(cur); cur = null; }
} else {
if (!cur) cur = { startLine: n, lines: [] };
cur.lines.push(ln);
cur.endLine = n;
}
});
if (cur) paras.push(cur);
for (const p of paras) p.text = p.lines.join('\n');
const chunks = [];
let acc = [];
let accLen = 0;
const flush = () => {
if (!acc.length) return;
chunks.push({
text: acc.map((a) => a.text).join('\n'),
startLine: acc[0].startLine,
endLine: acc[acc.length - 1].endLine,
});
};
for (const p of paras) {
if (p.text.length > CHUNK_HARD_SPLIT) {
if (acc.length) { flush(); acc = []; accLen = 0; }
chunks.push(...splitLongParagraph(p.text, p.startLine));
continue;
}
if (accLen + p.text.length > CHUNK_TARGET && accLen > 0) {
flush();
const tail = [];
let tlen = 0;
for (let j = acc.length - 1; j >= 0; j--) {
if (tlen + acc[j].text.length > CHUNK_OVERLAP && tail.length) break;
tail.unshift(acc[j]);
tlen += acc[j].text.length + 1;
}
acc = tail;
accLen = tlen;
}
acc.push(p);
accLen += p.text.length + 1;
}
flush();
return chunks;
}
/** Split markdown on headings, keeping the heading path per chunk. */
function chunkMarkdown(text) {
const lines = text.split('\n');
const stack = []; // { level, title }
let cur = { startLine: 1, lines: [] };
const sections = [];
lines.forEach((ln, i) => {
const n = i + 1;
const m = /^(#{1,6})\s+(.*?)\s*#*\s*$/.exec(ln);
if (m) {
sections.push({
heading: stack.map((s) => s.title).join(' > '),
startLine: cur.startLine,
lines: cur.lines,
});
const level = m[1].length;
const title = m[2].trim();
while (stack.length && stack[stack.length - 1].level >= level) stack.pop();
stack.push({ level, title });
cur = { startLine: n, lines: [ln] };
} else {
cur.lines.push(ln);
}
});
sections.push({
heading: stack.map((s) => s.title).join(' > '),
startLine: cur.startLine,
lines: cur.lines,
});
const out = [];
for (const s of sections) {
const body = s.lines.join('\n');
if (!body.trim()) continue;
if (body.length > CHUNK_HARD_SPLIT) {
for (const piece of splitLongParagraph(body, s.startLine)) {
out.push({ heading: s.heading, ...piece });
}
} else {
out.push({
heading: s.heading,
text: body,
startLine: s.startLine,
endLine: s.startLine + s.lines.length - 1,
});
}
}
return out;
}
function isMarkdown(relPath) {
return /\.(md|markdown)$/i.test(relPath);
}
function isJava(relPath) {
return /\.java$/i.test(relPath);
}
/**
* Split Java source on member boundaries using brace depth. Naive scan:
* braces are counted outside line comments, string literals, and char
* literals; block comments are not tracked. A chunk closes when depth
* returns to the class-body level (1) and JAVA_CHUNK_MIN chars have
* accumulated; any chunk above JAVA_CHUNK_HARD chars is hard split at the
* next line boundary. Headings carry package > class > method context
* (best effort: constructors use the class name, unparseable files fall
* back to chunkPlainText). Chunk text is always a raw slice of the file;
* the heading is metadata only, same as markdown.
*/
function chunkJava(text, relPath) {
const lines = text.split('\n');
const base = path.basename(relPath || '').replace(/\.java$/i, '') || 'file';
const memberRe = /([A-Za-z_$][A-Za-z0-9_$]*)\s*\(/g;
let pkg = '';
let cls = '';
let method = '';
let depth = 0;
let sawDepth = false;
const chunks = [];
let cur = null; // { startLine, lines, heading, members, len }
const headingNow = () => {
const parts = [];
if (pkg) parts.push(`package ${pkg};`);
parts.push(cls || base);
if (method) parts.push(method);
return parts.join(' > ');
};
const flush = () => {
if (!cur) return;
const body = cur.lines.join('\n');
if (body.trim()) {
chunks.push({
heading: cur.heading,
text: body,
startLine: cur.startLine,
endLine: cur.startLine + cur.lines.length - 1,
});
}
cur = null;
};
for (let i = 0; i < lines.length; i++) {
const ln = lines[i];
const n = i + 1;
// Scan the line: brace depth plus package/class/method context.
let inStr = false;
let inChr = false;
let inComment = false;
let enteredMember = false; // depth crossed 1 -> 2 (a member body opened)
let leftMember = false; // depth returned to 1 (a member body closed)
for (let j = 0; j < ln.length; j++) {
const c = ln[j];
if (inComment) break;
if (inStr) {
if (c === '\\') j++;
else if (c === '"') inStr = false;
continue;
}
if (inChr) {
if (c === '\\') j++;
else if (c === "'") inChr = false;
continue;
}
if (c === '/' && ln[j + 1] === '/') { inComment = true; continue; }
if (c === '"') { inStr = true; continue; }
if (c === "'") { inChr = true; continue; }
if (c === '{') {
depth++;
sawDepth = true;
if (depth === 2) enteredMember = true;
} else if (c === '}') {
if (depth > 0) depth--;
if (depth === 1) leftMember = true;
}
}
if (!pkg) {
const pm = /^\s*package\s+([\w.]+)\s*;/.exec(ln);
if (pm) pkg = pm[1];
}
if (!cls) {
const cm = /^\s*(?:@\w+\s+)*(?:public\s+|protected\s+|private\s+|abstract\s+|final\s+|sealed\s+|static\s+)*(?:class|interface|enum|record)\s+([A-Za-z_$][A-Za-z0-9_$]*)/.exec(ln);
if (cm) cls = cm[1];
}
if (enteredMember) {
// The method name is the last identifier followed by "(" in the
// lines accumulated since the previous member closed.
const buf = cur ? cur.lines.concat(ln).join('\n') : ln;
let m;
let last = null;
memberRe.lastIndex = 0;
while ((m = memberRe.exec(buf))) last = m[1];
if (last) method = last;
}
if (!cur) cur = { startLine: n, lines: [], heading: headingNow(), members: 0, len: 0 };
if (enteredMember) {
cur.members++;
if (cur.members === 1) cur.heading = headingNow();
}
cur.lines.push(ln);
cur.len += ln.length + 1;
if ((leftMember && cur.len >= JAVA_CHUNK_MIN) || cur.len > JAVA_CHUNK_HARD) flush();
}
flush();
if (!sawDepth) return chunkPlainText(text);
return chunks;
}
// ---------------------------------------------------------------------------
// Database
// ---------------------------------------------------------------------------
function openDb(dbPathRaw) {
const dbPath = expandTilde(dbPathRaw);
fs.mkdirSync(path.dirname(dbPath), { recursive: true });
const db = new DatabaseSync(dbPath);
db.exec('PRAGMA journal_mode = WAL');
db.exec(`
CREATE TABLE IF NOT EXISTS files (
id INTEGER PRIMARY KEY,
path TEXT UNIQUE,
hash TEXT,
size INTEGER,
mtime INTEGER,
indexed_at INTEGER
);
CREATE TABLE IF NOT EXISTS chunks (
id INTEGER PRIMARY KEY,
file_id INTEGER,
ordinal INTEGER,
heading TEXT,
start_line INTEGER,
end_line INTEGER,
text TEXT
);
CREATE VIRTUAL TABLE IF NOT EXISTS fts USING fts5(
text, chunk_id UNINDEXED, tokenize='porter unicode61'
);
CREATE TABLE IF NOT EXISTS vectors (
chunk_id INTEGER PRIMARY KEY,
blob BLOB
);
CREATE TABLE IF NOT EXISTS meta (key TEXT PRIMARY KEY, value TEXT);
`);
return db;
}
function metaGet(db, key) {
const row = db.prepare('SELECT value FROM meta WHERE key = ?').get(key);
return row ? row.value : null;
}
function metaSet(db, key, value) {
db.prepare('INSERT OR REPLACE INTO meta (key, value) VALUES (?, ?)').run(key, String(value));
}
// ---------------------------------------------------------------------------
// Filesystem walk
// ---------------------------------------------------------------------------
function walk(root, skipStats) {
const out = [];
const visit = (dir) => {
let entries;
try {
entries = fs.readdirSync(dir, { withFileTypes: true });
} catch (e) {
logErr(`cannot read ${dir}: ${e.message}`);
return;
}
entries.sort((a, b) => a.name.localeCompare(b.name));
for (const ent of entries) {
const abs = path.join(dir, ent.name);
if (ent.isDirectory()) {
if (!SKIP_DIRS.has(ent.name)) visit(abs);
continue;
}
if (!ent.isFile()) continue;
if (NAME_SKIPS.has(ent.name)) {
if (skipStats) skipStats.skipped_name++;
continue;
}
out.push({ abs, rel: path.relative(root, abs).split(path.sep).join('/') });
}
};
visit(root);
return out;
}
function loadFile(abs, relPath) {
const st = fs.statSync(abs);
if (st.size > MAX_FILE_SIZE) return { skip: 'size', size: st.size, mtime: st.mtimeMs };
const buf = fs.readFileSync(abs);
if (buf.includes(0)) return { skip: 'binary', size: st.size, mtime: st.mtimeMs };
const ext = path.extname(relPath || abs).slice(1).toLowerCase();
if (EXT_SKIPS.has(ext)) return { skip: 'extension', size: st.size, mtime: st.mtimeMs };
return { skip: null, buf, size: st.size, mtime: st.mtimeMs };
}
// ---------------------------------------------------------------------------
// Indexing
// ---------------------------------------------------------------------------
function deleteChunksOfFiles(db, fileIds) {
if (!fileIds.length) return;
const placeholders = fileIds.map(() => '?').join(',');
const ids = db.prepare(
`SELECT id FROM chunks WHERE file_id IN (${placeholders})`
).all(...fileIds);
if (ids.length) {
const ph = ids.map(() => '?').join(',');
const arg = ids.map((r) => r.id);
db.prepare(`DELETE FROM fts WHERE chunk_id IN (${ph})`).run(...arg);
db.prepare(`DELETE FROM vectors WHERE chunk_id IN (${ph})`).run(...arg);
}
db.prepare(`DELETE FROM chunks WHERE file_id IN (${placeholders})`).run(...fileIds);
}
/**
* Incremental index of root into the database.
* Returns stats: { files, rechunked, purged, skipped_binary, skipped_ext,
* skipped_name, chunks, dims, mode, ms }.
*/
async function indexRoot(db, rootRaw) {
const t0 = Date.now();
const root = path.resolve(expandTilde(rootRaw));
if (!fs.existsSync(root) || !fs.statSync(root).isDirectory()) {
throw new Error(`root is not a directory: ${root}`);
}
const cfg = embeddingsConfig();
const stats = {
root, files: 0, rechunked: 0, purged: 0, skipped_binary: 0,
skipped_ext: 0, skipped_name: 0,
chunks: 0, dims: null, mode: null, ms: 0,
};
const disk = walk(root, stats);
const diskByPath = new Map(disk.map((f) => [f.abs, f]));
const existing = db.prepare('SELECT id, path, hash FROM files').all();
const existingByPath = new Map(existing.map((r) => [r.path, r]));
// Files deleted on disk: purge chunks + fts + vectors + file row.
const purgedIds = existing.filter((r) => !diskByPath.has(r.path)).map((r) => r.id);
if (purgedIds.length) {
db.exec('BEGIN');
try {
deleteChunksOfFiles(db, purgedIds);
const del = db.prepare('DELETE FROM files WHERE id = ?');
for (const id of purgedIds) del.run(id);
db.exec('COMMIT');
} catch (e) {
db.exec('ROLLBACK');
throw e;
}
stats.purged = purgedIds.length;
}
// Pending vectors for newly created chunks (filled during re-chunk).
let pending = [];
for (const f of disk) {
const loaded = loadFile(f.abs, f.rel);
if (loaded.skip === 'binary') { stats.skipped_binary++; continue; }
if (loaded.skip === 'extension') { stats.skipped_ext++; continue; }
if (loaded.skip === 'size') continue;
const hash = sha256(loaded.buf);
const prev = existingByPath.get(f.abs);
if (prev && prev.hash === hash) continue; // unchanged: skip
stats.rechunked++;
const text = loaded.buf.toString('utf8');
const pieces = isJava(f.rel)
? chunkJava(text, f.rel)
: isMarkdown(f.rel) ? chunkMarkdown(text) : chunkPlainText(text);
db.exec('BEGIN');
try {
let fileId;
if (prev) {
deleteChunksOfFiles(db, [prev.id]);
db.prepare('UPDATE files SET hash=?, size=?, mtime=?, indexed_at=? WHERE id=?')
.run(hash, loaded.size, Math.round(loaded.mtime), Date.now(), prev.id);
fileId = prev.id;
} else {
const ins = db.prepare(
'INSERT INTO files (path, hash, size, mtime, indexed_at) VALUES (?, ?, ?, ?, ?)'
).run(f.abs, hash, loaded.size, Math.round(loaded.mtime), Date.now());
fileId = Number(ins.lastInsertRowid);
}
const insChunk = db.prepare(
'INSERT INTO chunks (file_id, ordinal, heading, start_line, end_line, text) VALUES (?, ?, ?, ?, ?, ?)'
);
const insFts = db.prepare('INSERT INTO fts (text, chunk_id) VALUES (?, ?)');
for (let i = 0; i < pieces.length; i++) {
const p = pieces[i];
const r = insChunk.run(fileId, i, p.heading || '', p.startLine, p.endLine, p.text);
const chunkId = Number(r.lastInsertRowid);
insFts.run(p.text, chunkId);
pending.push({ id: chunkId, text: p.text });
}
db.exec('COMMIT');
} catch (e) {
db.exec('ROLLBACK');
throw e;
}
}
// --- Embedding pass -----------------------------------------------------
let mode = cfg ? 'remote' : 'local';
const prevDims = metaGet(db, 'dims');
let dims = cfg ? null : LOCAL_DIMS;
if (cfg && pending.length) {
try {
logErr(`remote embeddings: ${pending.length} texts via ${cfg.url}`);
let vectors = await embedAllRemote(pending.map((p) => p.text), cfg);
dims = vectors[0].length;
if (prevDims && Number(prevDims) !== dims) {
// Dimensionality changed: rebuild every vector, not just new chunks.
logErr(`dims changed ${prevDims} -> ${dims}; rebuilding all vectors`);
db.exec('DELETE FROM vectors');
const all = db.prepare('SELECT id, text FROM chunks ORDER BY id').all();
pending = all.map((r) => ({ id: r.id, text: r.text }));
vectors = await embedAllRemote(pending.map((p) => p.text), cfg);
dims = vectors[0].length;
}
const put = db.prepare('INSERT OR REPLACE INTO vectors (chunk_id, blob) VALUES (?, ?)');
db.exec('BEGIN');
for (let i = 0; i < pending.length; i++) put.run(pending[i].id, vecToBlob(vectors[i]));
db.exec('COMMIT');
} catch (e) {
// Fall back to local for this run: wipe stale vectors and rebuild all.
logErr(`remote embeddings failed (${e.message}); falling back to local for this run`);
mode = 'local';
dims = LOCAL_DIMS;
db.exec('DELETE FROM vectors');
const all = db.prepare('SELECT id, text FROM chunks ORDER BY id').all();
pending = all.map((r) => ({ id: r.id, text: r.text }));
}
} else if (!cfg) {
if (prevDims && Number(prevDims) !== LOCAL_DIMS) {
db.exec('DELETE FROM vectors');
const all = db.prepare('SELECT id, text FROM chunks ORDER BY id').all();
pending = all.map((r) => ({ id: r.id, text: r.text }));
}
}
if (mode === 'local' && pending.length) {
const put = db.prepare('INSERT OR REPLACE INTO vectors (chunk_id, blob) VALUES (?, ?)');
db.exec('BEGIN');
for (const p of pending) put.run(p.id, vecToBlob(localEmbed(p.text, LOCAL_DIMS)));
db.exec('COMMIT');
dims = LOCAL_DIMS;
}
stats.files = db.prepare('SELECT COUNT(*) AS n FROM files').get().n;
stats.chunks = db.prepare('SELECT COUNT(*) AS n FROM chunks').get().n;
// Only overwrite dims/mode when this run actually (re)embedded something;
// otherwise the meta keeps describing the vectors that are already stored.
if (pending.length > 0) {
metaSet(db, 'dims', dims);
metaSet(db, 'embedding_mode', mode);
stats.dims = dims;
stats.mode = mode;
} else {
const d = metaGet(db, 'dims');
stats.dims = d === null ? null : Number(d);
stats.mode = metaGet(db, 'embedding_mode') || mode;
}
metaSet(db, 'last_index_at', String(Date.now()));
metaSet(db, 'root', root);
stats.ms = Date.now() - t0;
return stats;
}
// ---------------------------------------------------------------------------
// Search
// ---------------------------------------------------------------------------
function ftsQuery(tokens, op) {
return tokens.map((t) => `"${t}"`).join(` ${op} `);
}
async function search(db, query, opts) {
const k = Math.max(1, Number(opts && opts.k) || 8);
const globRe = opts && opts.glob ? globToRegExp(opts.glob) : null;
const root = metaGet(db, 'root') || DEFAULT_ROOT;
const mode = metaGet(db, 'embedding_mode') || 'local';
const dims = Number(metaGet(db, 'dims') || LOCAL_DIMS);
const tokens = tokenize(query);
let ftsRows = [];
if (tokens.length) {
const orRows = db.prepare(
'SELECT chunk_id, bm25(fts) AS r FROM fts WHERE fts MATCH ? ORDER BY r LIMIT ?'
).all(ftsQuery(tokens, 'OR'), FTS_TOP_LARGE);
let andRows = orRows;
if (tokens.length > 1) {
const cand = db.prepare(
'SELECT chunk_id, bm25(fts) AS r FROM fts WHERE fts MATCH ? ORDER BY r LIMIT ?'
).all(ftsQuery(tokens, 'AND'), FTS_TOP_LARGE);
if (cand.length > 0) andRows = cand;
}
ftsRows = andRows.length >= orRows.length ? andRows : orRows;
if (ftsRows.length > 0) ftsRows = ftsRows.slice(0, Math.max(FTS_TOP, FTS_TOP_LARGE));
}
// Rank normalization over the fts rows kept for scoring.
const rankById = new Map();
if (ftsRows.length) {
const kept = ftsRows.slice(0, FTS_TOP_LARGE);
let rmin = Infinity;
let rmax = -Infinity;
for (const r of kept) {
if (r.r < rmin) rmin = r.r;
if (r.r > rmax) rmax = r.r;
}
for (const r of kept) {
const norm = rmax === rmin ? 1 : 1 - (r.r - rmin) / (rmax - rmin);
rankById.set(r.chunk_id, norm);
}
}
const totalChunks = db.prepare('SELECT COUNT(*) AS n FROM chunks').get().n;
// Candidate set: fts top + full scan when the corpus is small enough.
let candidates;
if (totalChunks < FULL_SCAN_LIMIT) {
candidates = db.prepare(
'SELECT c.id, c.file_id, c.heading, c.start_line, c.end_line, c.text, f.path ' +
'FROM chunks c JOIN files f ON f.id = c.file_id ORDER BY c.id'
).all();
} else {
candidates = [];
const byId = new Map();
const load = (id) => {
if (byId.has(id)) return byId.get(id);
const row = db.prepare(
'SELECT c.id, c.file_id, c.heading, c.start_line, c.end_line, c.text, f.path ' +
'FROM chunks c JOIN files f ON f.id = c.file_id WHERE c.id = ?'
).get(id);
if (row) byId.set(id, row);
return row;
};
for (const id of rankById.keys()) {
const row = load(id);
if (row) candidates.push(row);
}
}
// Query vector: remote endpoint when the index was built in remote mode.
let qvec = null;
if (mode === 'remote') {
const cfg = embeddingsConfig();
if (cfg) {
try {
qvec = await embedQueryRemote(String(query), cfg);
} catch (e) {
logErr(`remote query embedding failed (${e.message}); scoring by keyword rank only`);
}
} else {
logErr('index uses remote embeddings but RAG_EMBEDDINGS_URL/MODEL are unset; keyword rank only');
}
} else {
qvec = localEmbed(String(query), dims);
}
// Vectors for scoring.
let vecById = null;
if (qvec && totalChunks < FULL_SCAN_LIMIT) {
vecById = new Map();
for (const row of db.prepare('SELECT chunk_id, blob FROM vectors').all()) {
const buf = row.blob;
if (buf && buf.byteLength === qvec.length * 4) vecById.set(row.chunk_id, blobToVec(buf));
}
}
const scored = [];
for (const row of candidates) {
const rel = path.relative(root, row.path).split(path.sep).join('/');
if (globRe && !globRe.test(rel)) continue;
const rank = rankById.get(row.id) || 0;
let cos = 0;
if (qvec) {
let v;
if (vecById) {
v = vecById.get(row.id);
} else {
const vr = db.prepare('SELECT blob FROM vectors WHERE chunk_id = ?').get(row.id);
if (vr && vr.blob && vr.blob.byteLength === qvec.length * 4) v = blobToVec(vr.blob);
}
if (v) cos = cosine(qvec, v);
}
const score = FUSE_KEYWORD * rank + FUSE_VECTOR * cos;
scored.push({
id: row.id,
path: rel,
heading: row.heading || '',
start_line: row.start_line,
end_line: row.end_line,
score,
snippet: snippetOf(row.text),
_text: row.text,
});
}
scored.sort((a, b) => b.score - a.score || a.id - b.id);
return scored.slice(0, k).map(({ _text, ...hit }) => ({ ...hit, text: _text }));
}
// ---------------------------------------------------------------------------
// Status + get
// ---------------------------------------------------------------------------
function statusOf(db) {
const files = db.prepare('SELECT COUNT(*) AS n FROM files').get().n;
const chunks = db.prepare('SELECT COUNT(*) AS n FROM chunks').get().n;
const lastRaw = metaGet(db, 'last_index_at');
const root = metaGet(db, 'root') || DEFAULT_ROOT;
const mode = metaGet(db, 'embedding_mode') || (files ? 'local' : 'unindexed');
const dims = metaGet(db, 'dims');
let stale = 0;
let staleNote = '';
const skipCounts = { skipped_name: 0 };
let skippedExt = 0;
if (files > 0 && fs.existsSync(root)) {
const existing = new Map(
db.prepare('SELECT id, path, hash FROM files').all().map((r) => [r.path, r])
);
const disk = walk(root, skipCounts);
const diskPaths = new Set();
for (const f of disk) {
diskPaths.add(f.abs);
const ext = path.extname(f.rel).slice(1).toLowerCase();
if (EXT_SKIPS.has(ext)) { skippedExt++; continue; } // never indexed: not stale
const rec = existing.get(f.abs);
if (!rec) { stale++; continue; }
try {
const loaded = loadFile(f.abs, f.rel);
if (loaded.skip || sha256(loaded.buf) !== rec.hash) stale++;
} catch (e) {
stale++;
}
}
for (const p of existing.keys()) {
if (!diskPaths.has(p)) stale++;
}
} else if (files > 0) {
staleNote = `root missing on disk: ${root}`;
}
return {
db: expandTilde(process.env.RAG_DB || DEFAULT_DB),
root,
files,
chunks,
last_index_at: lastRaw ? nowIso(Number(lastRaw)) : null,
stale_files: stale,
stale_note: staleNote,
skipped_ext: skippedExt,
skipped_name: skipCounts.skipped_name,
embedding_mode: mode,
dims: dims === null ? null : Number(dims),
};
}
function getChunk(db, id) {
const row = db.prepare(
'SELECT c.id, c.file_id, c.ordinal, c.heading, c.start_line, c.end_line, c.text, f.path ' +
'FROM chunks c JOIN files f ON f.id = c.file_id WHERE c.id = ?'
).get(Number(id));
if (!row) return null;
const root = metaGet(db, 'root') || DEFAULT_ROOT;
return {
id: row.id,
path: path.relative(root, row.path).split(path.sep).join('/'),
ordinal: row.ordinal,
heading: row.heading || '',
start_line: row.start_line,
end_line: row.end_line,
text: row.text,
};
}
// ---------------------------------------------------------------------------
// CLI
// ---------------------------------------------------------------------------
function printIndexStats(s) {
console.log(
`index complete: files=${s.files} rechunked=${s.rechunked} purged=${s.purged} ` +
`chunks=${s.chunks} skipped_binary=${s.skipped_binary} skipped_ext=${s.skipped_ext} ` +
`skipped_name=${s.skipped_name} dims=${s.dims === null ? 'n/a' : s.dims} ` +
`mode=${s.mode} elapsed=${s.ms}ms`
);
console.log(`root: ${s.root}`);
}
function printHits(hits) {
if (!hits.length) {
console.log('no hits');
return;
}
hits.forEach((h, i) => {
const head = h.heading ? ` ${h.heading}` : '';
console.log(
`${i + 1}. ${h.path}:${h.start_line}-${h.end_line} (${h.score.toFixed(4)})${head}`
);
console.log(` ${h.snippet}`);
});
}
function printStatus(s) {
console.log(`db: ${s.db}`);
console.log(`root: ${s.root}`);
console.log(`files: ${s.files}`);
console.log(`chunks: ${s.chunks}`);
console.log(`last_index_at: ${s.last_index_at || 'never'}`);
console.log(`stale_files: ${s.stale_files}${s.stale_note ? ` (${s.stale_note})` : ''}`);
console.log(`skipped_ext: ${s.skipped_ext}`);
console.log(`skipped_name: ${s.skipped_name}`);
console.log(`embedding_mode: ${s.embedding_mode}`);
console.log(`dims: ${s.dims === null ? 'n/a' : s.dims}`);
}
async function runCli(argv) {
const cmd = argv[0];
const db = openDb(process.env.RAG_DB || DEFAULT_DB);
try {
if (cmd === 'index') {
const root = argv[1] || metaGet(db, 'root') || DEFAULT_ROOT;
printIndexStats(await indexRoot(db, root));
} else if (cmd === 'search') {
const query = argv[1];
if (!query) throw new Error('usage: search [--k N] [--glob PATTERN]');
let k = 8;
let glob = null;
for (let i = 2; i < argv.length; i++) {
if (argv[i] === '--k') { k = Number(argv[++i]); }
else if (argv[i] === '--glob') { glob = argv[++i]; }
else throw new Error(`unknown search option: ${argv[i]}`);
}
const hits = await search(db, query, { k, glob });
printHits(hits.map((h) => ({ ...h, snippet: h.snippet })));
} else if (cmd === 'status') {
printStatus(statusOf(db));
} else if (cmd === 'get') {
const id = Number(argv[1]);
if (!Number.isFinite(id)) throw new Error('usage: get ');
const c = getChunk(db, id);
if (!c) throw new Error(`no chunk with id ${id}`);
console.log(
`chunk ${c.id}: ${c.path}:${c.start_line}-${c.end_line}` +
`${c.heading ? ` heading: ${c.heading}` : ''}`
);
console.log(c.text);
} else {
throw new Error(
'usage: node server.js index [root] | search [--k N] [--glob PATTERN] | status | get | (no args for MCP stdio)'
);
}
} finally {
db.close();
}
}
// ---------------------------------------------------------------------------
// MCP stdio mode
// ---------------------------------------------------------------------------
function toolDefs() {
return [
{
name: 'rag_search',
description:
'Search the indexed workspace with keyword (FTS5) and vector ranking fused. ' +
'Returns ranked hits with path, line range, heading, score and a snippet.',
inputSchema: {
type: 'object',
properties: {
query: { type: 'string', description: 'Search text (natural language or keywords)' },
k: { type: 'number', description: 'Maximum hits to return (default 8)' },
glob: { type: 'string', description: 'Glob filter on the relative path, e.g. "docs/**/*.md" or "api/*"' },
},
required: ['query'],
},
},
{
name: 'rag_index',
description:
'Incremental reindex of the workspace root: hash-checks every file, re-chunks ' +
'changed files, purges deleted ones. Safe to call repeatedly.',
inputSchema: {
type: 'object',
properties: {
root: { type: 'string', description: 'Directory to index (default: the previously indexed root or /home/node/workspace)' },
},
},
},
{
name: 'rag_status',
description: 'Index status: database path, root, file and chunk counts, last index time, stale file count, embedding mode and dims.',
inputSchema: { type: 'object', properties: {} },
},
{
name: 'rag_get',
description: 'Fetch one indexed chunk by id: full text plus path, heading and line range.',
inputSchema: {
type: 'object',
properties: {
id: { type: 'number', description: 'Chunk id (from rag_search hits)' },
},
required: ['id'],
},
},
];
}
function textResult(text, structured) {
const out = { content: [{ type: 'text', text }] };
if (structured !== undefined) out.structuredContent = structured;
return out;
}
async function callTool(params) {
const name = params.name;
const args = params.arguments || {};
const db = openDb(process.env.RAG_DB || DEFAULT_DB);
try {
if (name === 'rag_search') {
const query = args.query;
if (typeof query !== 'string' || !query.trim()) {
return textResult('error: query must be a non-empty string', { hits: [] });
}
const hits = await search(db, query, { k: args.k, glob: args.glob });
const lines = hits.length
? hits.map((h, i) => {
const head = h.heading ? ` ${h.heading}` : '';
return `${i + 1}. ${h.path}:${h.start_line}-${h.end_line} (${h.score.toFixed(4)})${head}\n ${h.snippet}`;
})
: ['no hits'];
const payload = hits.map(({ text, ...rest }) => rest);
return textResult(lines.join('\n'), { hits: payload });
}
if (name === 'rag_index') {
const stats = await indexRoot(db, args.root || metaGet(db, 'root') || DEFAULT_ROOT);
return textResult(
`index complete: files=${stats.files} rechunked=${stats.rechunked} purged=${stats.purged} ` +
`chunks=${stats.chunks} skipped_binary=${stats.skipped_binary} skipped_ext=${stats.skipped_ext} ` +
`skipped_name=${stats.skipped_name} dims=${stats.dims === null ? 'n/a' : stats.dims} ` +
`mode=${stats.mode} elapsed=${stats.ms}ms`,
stats
);
}
if (name === 'rag_status') {
const st = statusOf(db);
return textResult(JSON.stringify(st, null, 2), st);
}
if (name === 'rag_get') {
const c = getChunk(db, args.id);
if (!c) return textResult(`error: no chunk with id ${args.id}`, { chunk: null });
const head = `chunk ${c.id}: ${c.path}:${c.start_line}-${c.end_line}${c.heading ? ` heading: ${c.heading}` : ''}`;
return textResult(`${head}\n\n${c.text}`, { chunk: c });
}
return { content: [{ type: 'text', text: `error: unknown tool: ${name}` }], isError: true };
} catch (e) {
return { content: [{ type: 'text', text: `error: ${e.message}` }], isError: true };
} finally {
db.close();
}
}
function writeMsg(msg) {
process.stdout.write(JSON.stringify(msg) + '\n');
}
async function handleMcpLine(line) {
let msg;
try {
msg = JSON.parse(line);
} catch (e) {
writeMsg({ jsonrpc: '2.0', id: null, error: { code: -32700, message: 'parse error' } });
return;
}
const { id, method, params } = msg || {};
if (!method) return; // a response or garbage without a method: ignore
try {
if (method === 'initialize') {
writeMsg({
jsonrpc: '2.0',
id,
result: {
protocolVersion: (params && params.protocolVersion) || '2025-06-18',
capabilities: { tools: {} },
serverInfo: { name: SERVER_NAME, version: SERVER_VERSION },
},
});
} else if (method === 'ping') {
writeMsg({ jsonrpc: '2.0', id, result: {} });
} else if (method === 'tools/list') {
writeMsg({ jsonrpc: '2.0', id, result: { tools: toolDefs() } });
} else if (method === 'tools/call') {
const result = await callTool(params || {});
writeMsg({ jsonrpc: '2.0', id, result });
} else if (method.startsWith('notifications/')) {
// notifications (initialized, cancelled, progress): no response
} else if (id === undefined || id === null) {
// unknown notification: stay silent
} else {
writeMsg({
jsonrpc: '2.0',
id,
error: { code: -32601, message: `method not found: ${method}` },
});
}
} catch (e) {
writeMsg({
jsonrpc: '2.0',
id,
error: { code: -32603, message: String((e && e.message) || e) },
});
}
}
function startMcp() {
let pending = '';
let chain = Promise.resolve(); // serialize line handling: responses keep request order
process.stdin.setEncoding('utf8');
process.stdin.on('data', (chunk) => {
pending += chunk;
let idx;
while ((idx = pending.indexOf('\n')) >= 0) {
const line = pending.slice(0, idx).trim();
pending = pending.slice(idx + 1);
if (line) chain = chain.then(() => handleMcpLine(line)).catch((e) => logErr(e.message));
}
});
process.stdin.on('end', () => process.exit(0));
process.stdin.resume();
}
// ---------------------------------------------------------------------------
// Entry
// ---------------------------------------------------------------------------
async function main() {
const argv = process.argv.slice(2);
if (argv.length === 0) {
startMcp();
return;
}
await runCli(argv);
}
main().catch((e) => {
logErr(e && e.stack ? e.stack : String(e));
process.exit(1);
});
================
File: workspace/agents/opencode/rag/test.sh
================
#!/usr/bin/env bash
# rag/test.sh — self-contained test suite for the rag MCP server and CLI.
# Covers: CLI index/search/glob/status/get, incremental reindex (change +
# delete), Java method-boundary chunking, build-output and lockfile skips,
# MCP stdio handshake, and the remote embeddings path via a local stub.
# Zero npm dependencies: needs node, curl, sed, grep, coreutils.
# All state lives in /tmp/rag-test-$$ and is removed on exit.
set -u
cd "$(dirname "$0")" # rag/
PASS=0
FAIL=0
WORK="/tmp/rag-test-$$"
mkdir -p "$WORK"
STUB_PID=""
cleanup() {
if [ -n "$STUB_PID" ]; then
kill "$STUB_PID" 2>/dev/null || true
fi
rm -rf "$WORK"
}
trap cleanup EXIT
pass() { echo "PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo "FAIL: $1"; FAIL=$((FAIL + 1)); }
# Extract a key=value field from the "index complete:" line.
field() { printf '%s\n' "$1" | sed -n 's/.*'"$2"'=\([0-9]*\).*/\1/p'; }
# Fresh per-case workspace: pristine corpus copy plus its own database.
new_case() {
CASE="$WORK/$1"
mkdir -p "$CASE"
cp -r test/corpus "$CASE/corpus"
DB="$CASE/rag.db"
}
idx() { RAG_DB="$DB" node server.js index "$CASE/corpus" 2>/dev/null; }
srch() { RAG_DB="$DB" node server.js search "$@" 2>/dev/null; }
echo "== syntax check =="
if node --check server.js && node --check test/upstream-embeddings-stub.js \
&& node --check test/mcp-check.js && node --check test/java-check.js; then
pass "syntax check (server.js, stub, mcp-check, java-check)"
else
fail "syntax check"
echo "passed: $PASS failed: $FAIL"
exit 1
fi
# --- Case 1: CLI index reports 4 indexed files, binary skipped, chunks > 7 ---
echo "== case 1: CLI index =="
new_case c1
OUT="$(idx)"
FILES="$(field "$OUT" files)"
CHUNKS="$(field "$OUT" chunks)"
SKIPPED="$(field "$OUT" skipped_binary)"
if [ "$FILES" = "4" ] && [ "$SKIPPED" = "1" ] && [ -n "$CHUNKS" ] && [ "$CHUNKS" -gt 7 ]; then
pass "case 1: index reports files=4, skipped_binary=1, chunks=$CHUNKS (>7)"
else
fail "case 1: index stats wrong (files=$FILES skipped=$SKIPPED chunks=$CHUNKS)"
fi
# --- Case 2: search finds the unique phrase, top hit is getting-started.md ---
echo "== case 2: CLI search unique phrase =="
OUT="$(srch "quantum marmalade")"
TOP="$(printf '%s\n' "$OUT" | head -1)"
if printf '%s' "$TOP" | grep -q "getting-started.md" \
&& printf '%s' "$OUT" | grep -q "quantum marmalade"; then
pass "case 2: top hit is getting-started.md and carries the phrase"
else
fail "case 2: unexpected result (top: $TOP)"
fi
# --- Case 3: --glob restricts hits to api/users.md ---------------------------
echo "== case 3: CLI search with glob =="
OUT="$(srch "users api rename" --glob "api/*")"
if printf '%s' "$OUT" | grep -q "api/users.md" \
&& ! printf '%s' "$OUT" | grep -q "getting-started.md" \
&& ! printf '%s' "$OUT" | grep -q "format.js"; then
pass "case 3: glob api/* limits hits to api/users.md"
else
fail "case 3: glob filter leaked other files"
fi
# --- Case 4: re-index unchanged corpus is a no-op ----------------------------
echo "== case 4: incremental reindex no-op =="
OUT="$(idx)"
RECH="$(field "$OUT" rechunked)"
CHUNKS2="$(field "$OUT" chunks)"
FILES2="$(field "$OUT" files)"
if [ "$RECH" = "0" ] && [ "$CHUNKS2" = "$CHUNKS" ] && [ "$FILES2" = "4" ]; then
pass "case 4: reindex unchanged (rechunked=0, chunks=$CHUNKS2 stable)"
else
fail "case 4: reindex not a no-op (rechunked=$RECH chunks=$CHUNKS2 files=$FILES2)"
fi
# --- Case 5: mutate one file, delete another, reindex ------------------------
echo "== case 5: incremental change + delete =="
new_case c5
idx >/dev/null
printf '\n// mutation marker for the reindex test\nfunction mutated() { return 42; }\n' \
>> "$CASE/corpus/utils/format.js"
rm "$CASE/corpus/api/users.md"
OUT="$(idx)"
RECH="$(field "$OUT" rechunked)"
PURGED="$(field "$OUT" purged)"
ST="$(RAG_DB="$DB" node server.js status 2>/dev/null)"
FILES3="$(printf '%s\n' "$ST" | sed -n 's/^files: \([0-9]*\)$/\1/p')"
GONE="$(srch "rename a user account")"
if [ "$RECH" = "1" ] && [ "$PURGED" = "1" ] && [ "$FILES3" = "3" ] \
&& ! printf '%s' "$GONE" | grep -q "api/users.md"; then
pass "case 5: changed file re-chunked, deleted file purged (files=$FILES3)"
else
fail "case 5: incremental state wrong (rechunked=$RECH purged=$PURGED files=$FILES3)"
fi
# --- Case 6: MCP stdio handshake over JSON-RPC lines --------------------------
echo "== case 6: MCP handshake =="
new_case c6
idx >/dev/null
REQ="$WORK/reqs.jsonl"
RESP="$WORK/resp.jsonl"
cat > "$REQ" <<'EOF'
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"rag-test","version":"0.0.0"}}}
{"jsonrpc":"2.0","method":"notifications/initialized"}
{"jsonrpc":"2.0","id":2,"method":"tools/list"}
{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"rag_search","arguments":{"query":"quantum marmalade"}}}
EOF
RAG_DB="$DB" node server.js < "$REQ" > "$RESP" 2>/dev/null
if node test/mcp-check.js "$RESP"; then
pass "case 6: MCP handshake (initialize, tools/list x4, tools/call hits)"
else
fail "case 6: MCP handshake assertions failed"
fi
# --- Case 7: remote embeddings via stub (dims 8, bearer key) ------------------
echo "== case 7: remote embeddings =="
node test/upstream-embeddings-stub.js >/dev/null 2>&1 &
STUB_PID=$!
READY=0
for _ in $(seq 1 50); do
if curl -sf http://127.0.0.1:4599/healthz >/dev/null 2>&1; then READY=1; break; fi
sleep 0.1
done
if [ "$READY" = "1" ]; then
new_case c7
OUT="$(RAG_EMBEDDINGS_URL=http://127.0.0.1:4599/v1/embeddings \
RAG_EMBEDDINGS_MODEL=test-embed RAG_EMBEDDINGS_KEY=testkey \
RAG_DB="$DB" node server.js index "$CASE/corpus" 2>/dev/null)"
ST="$(RAG_EMBEDDINGS_URL=http://127.0.0.1:4599/v1/embeddings \
RAG_EMBEDDINGS_MODEL=test-embed RAG_EMBEDDINGS_KEY=testkey \
RAG_DB="$DB" node server.js status 2>/dev/null)"
AUTH="$(curl -s http://127.0.0.1:4599/last-auth)"
OUT2="$(RAG_EMBEDDINGS_URL=http://127.0.0.1:4599/v1/embeddings \
RAG_EMBEDDINGS_MODEL=test-embed RAG_EMBEDDINGS_KEY=testkey \
RAG_DB="$DB" node server.js search "quantum marmalade" 2>/dev/null)"
TOP="$(printf '%s\n' "$OUT2" | head -1)"
if printf '%s' "$OUT" | grep -q "mode=remote" \
&& printf '%s\n' "$ST" | grep -q "^embedding_mode: remote$" \
&& printf '%s\n' "$ST" | grep -q "^dims: 8$" \
&& [ "$AUTH" = "Bearer testkey" ] \
&& printf '%s' "$TOP" | grep -q "getting-started.md"; then
pass "case 7: remote mode indexed (dims=8), bearer key sent, search works"
else
fail "case 7: remote mode wrong (out=$OUT auth=$AUTH top=$TOP)"
fi
else
fail "case 7: embeddings stub did not become ready"
fi
kill "$STUB_PID" 2>/dev/null || true
STUB_PID=""
# --- Case 8: rag_get returns the full chunk text -------------------------------
echo "== case 8: CLI get =="
new_case c8
idx >/dev/null
OUT="$(RAG_DB="$DB" node server.js get 1 2>/dev/null)"
LINES="$(printf '%s\n' "$OUT" | wc -l)"
if printf '%s\n' "$OUT" | grep -q "^chunk 1:" && [ "$LINES" -gt 1 ]; then
pass "case 8: get 1 returns metadata header and text ($LINES lines)"
else
fail "case 8: get 1 returned nothing usable"
fi
# --- Case 9: Java chunking (method boundaries, headings, hard split) ----------
echo "== case 9: Java chunking =="
new_case c9
idx >/dev/null
OUT="$(srch "marmaladeDeduplication")"
TOP="$(printf '%s\n' "$OUT" | head -1)"
CID=""
GOT=""
if node test/java-check.js "$DB" > "$WORK/c9.chk" 2>/dev/null \
&& printf '%s' "$TOP" | grep -q "UserService.java"; then
CID="$(sed -n 's/^CHUNK_ID=//p' "$WORK/c9.chk")"
GOT="$(RAG_DB="$DB" node server.js get "$CID" 2>/dev/null | head -1)"
fi
if printf '%s' "$TOP" | grep -q "UserService.java" \
&& grep -q ": ok" "$WORK/c9.chk" && ! grep -q "FAIL" "$WORK/c9.chk" \
&& printf '%s' "$GOT" | grep -q "processBatch"; then
pass "case 9: java chunking (top hit UserService.java, get $CID: $GOT)"
else
fail "case 9: java chunking wrong (top: $TOP get: $GOT)"
cat "$WORK/c9.chk" 2>/dev/null || true
fi
# --- Case 10: build-output dirs and lockfiles are skipped ---------------------
echo "== case 10: build dir + lockfile skips =="
new_case c10
OUT="$(idx)"
ST="$(RAG_DB="$DB" node server.js status 2>/dev/null)"
FILES4="$(printf '%s\n' "$ST" | sed -n 's/^files: \([0-9]*\)$/\1/p')"
NOJUNK="$(srch "compiled leftover")"
if [ "$FILES4" = "4" ] \
&& printf '%s\n' "$OUT" | grep -q "skipped_name=1" \
&& printf '%s\n' "$ST" | grep -q "^skipped_name: 1$" \
&& ! printf '%s' "$NOJUNK" | grep -q "junk.txt"; then
pass "case 10: target/ and package-lock.json skipped (files=$FILES4, no junk.txt hits)"
else
fail "case 10: skip accounting wrong (files=$FILES4 out=$OUT junk=$NOJUNK)"
fi
echo "----"
echo "passed: $PASS failed: $FAIL"
if [ "$FAIL" -eq 0 ]; then
exit 0
fi
exit 1
================
File: workspace/agents/opencode/session/state.example.md
================
# STATE — Session Log (Example)
> Worked example of the `engine/contract/state.template.md` structure, filled with plausible values.
> Copy this structure to `session/state.md` (local state — never committed) and let
> `scripts/snapshot-contract.sh` append transition rows.
## Current Focus
State: **EXECUTE** — implementing plan task 3 of 5 (adapter layer + mapper tests). Developer shard A on `*InPort.java`/`*Service.java`, shard B on `*Adapter.java`; files provably disjoint.
## Known Blockers
None.
## Transition Log
| Timestamp | Session | State | Score | Hash | Summary |
|-----------|---------|-------|-------|------|---------|
| 2026-11-01T09:14:02Z | sess-20261101-a1b2 | INIT | — | 3f9c1a2 | Session initialised from contract.template.json |
| 2026-11-01T09:20:37Z | sess-20261101-a1b2 | PLAN | — | 7d21e8b | Plan produced by @system-analyst (5 tasks, 2 parallel-eligible) |
| 2026-11-01T09:24:10Z | sess-20261101-a1b2 | PLAN_SCORED | 84 | 91ac04f | Plan accepted (score ≥ 70), spec gate passed |
| 2026-11-01T09:31:55Z | sess-20261101-a1b2 | EXECUTE | — | c48b771 | Developer shards A/B dispatched (max_parallel_agents: 2) |
Append-only. Each transition row is written by `scripts/snapshot-contract.sh`.
================
File: workspace/agents/opencode/.dockerignore
================
.git
.gitignore
.idea
session
.session
pre-install
.env*
node_modules
docker-compose.example.yaml
================
File: workspace/agents/opencode/.env.template
================
# .env.template for docker compose (copy to .env and fill in)
# All values are optional: the compose files carry working defaults.
# Required for OpenCode auth (compose default is a placeholder)
OPENCODE_API_KEY=REPLACE_ME
# Model provider (compose defaults exist for all three)
SUMOPOD_API_KEY=
SUMOPOD_BASE_URL=
SUMOPOD_BASE_MODEL=
# Context7 docs MCP (compose default is a placeholder)
CONTEXT7_API_KEY=
# Optional Firecrawl key for the vendored @firecrawl/opencode plugin.
# Leave empty: firecrawl_developer_search works keyless and CLI-free.
# firecrawl_alexandria and the firecrawl-cli skills require it (or OAuth).
FIRECRAWL_API_KEY=
# Key for OpenCode's own auth (pairing screen / HTTP basic auth on the opencode server). Leave empty to disable.
OPENCODE_SERVER_PASSWORD=
# Optional real embeddings endpoint for the rag MCP server. All three must be
# set to leave local hashed-vector mode; local mode is deterministic, needs no
# network, and works fully offline (the pre-install store needs nothing for it).
# RAG_EMBEDDINGS_URL points at an OpenAI-compatible /v1/embeddings endpoint,
# RAG_EMBEDDINGS_MODEL is the model id, RAG_EMBEDDINGS_KEY its bearer key.
RAG_EMBEDDINGS_URL=
RAG_EMBEDDINGS_MODEL=
RAG_EMBEDDINGS_KEY=
================
File: workspace/agents/opencode/.gitattributes
================
*.sh text eol=lf
Dockerfile text eol=lf
*.json text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
================
File: workspace/agents/opencode/.gitignore
================
.idea/
.session/
session/*
!session/state.example.md
pre-install/**
!pre-install/**/
!pre-install/**/.gitkeep
================
File: workspace/agents/opencode/docker-compose.example.yaml
================
name: opencode
services:
opencode:
container_name: opencode
image: opencode:local # tag applied to the build below; scheduler service reuses it
labels:
bpg.group: infra
bpg.type: agents
build:
context: .
additional_contexts:
preinstall: ./pre-install
args:
OPENCODE_VERSION: "2.2.5" # pinned for reproducible builds
OPENCODE_ARCH: "arm64" # matches pre-install store tarballs; x64 builds override
GRAPHIFY_VERSION: "0.9.100"
LEAN_CTX_VERSION: "3.12.0"
CONTEXT7_VERSION: "4.4.4"
MCP_VERSION: "2.3.0"
MCP_SERVER_FETCH_VERSION: "2026.10.28"
init: true
mem_limit: 3g
cpus: 2
restart: unless-stopped
ports:
- "127.0.0.1:4096:4096"
environment:
LANG: C.UTF-8
LC_ALL: C.UTF-8
TERM: xterm-256color
OPENCODE_API_KEY: ${OPENCODE_API_KEY:-REPLACE_ME}
OPENCODE_AUTH_REQUIRED: "true"
OPENCODE_SERVER_PASSWORD: ${OPENCODE_SERVER_PASSWORD:-}
OPENCODE_CONFIG: /home/node/.config/opencode/opencode.json
SUMOPOD_API_KEY: ${SUMOPOD_API_KEY:-REPLACE_ME}
SUMOPOD_BASE_URL: ${SUMOPOD_BASE_URL:-REPLACE_ME}
SUMOPOD_BASE_MODEL: ${SUMOPOD_BASE_MODEL:-REPLACE_ME}
SUMOPOD_BASE_MODEL_NAME: ${SUMOPOD_BASE_MODEL_NAME:-REPLACE_ME}
CONTEXT7_API_KEY: ${CONTEXT7_API_KEY:-REPLACE_ME}
# Optional: keyless developer search works without it; alexandria + firecrawl-cli skills need it
FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY:-}
# Optional rag embeddings upgrade; empty keeps offline hashed-vector mode
RAG_EMBEDDINGS_URL: ${RAG_EMBEDDINGS_URL:-}
RAG_EMBEDDINGS_MODEL: ${RAG_EMBEDDINGS_MODEL:-}
RAG_EMBEDDINGS_KEY: ${RAG_EMBEDDINGS_KEY:-}
# Graphify disabled but config preserved
GRAPHIFY_ROOT: /home/node/workspace
GRAPHIFY_BUILD: if-missing
GRAPHIFY_TIMEOUT: "900"
volumes:
- .:/home/node/workspace
- ./config/opencode.json:/home/node/.config/opencode/opencode.json:ro
- ./config/opencode-model-fallback.json:/home/node/.config/opencode/opencode-model-fallback.json:ro
- ./rag:/opt/rag:ro
- opencode-data:/home/node/.local/share/opencode
- /var/run/docker.sock:/var/run/docker.sock
group_add:
- "${DOCKER_GID:-101}"
opencode-preinstall:
image: node:22-bookworm-slim
profiles: ["tools"]
user: root
volumes:
- .:/repo
- ./pre-install:/repo/pre-install # explicit store mount (project level)
working_dir: /repo
command: >-
sh -c "apt-get update -qq && apt-get install -y -qq --no-install-recommends
curl ca-certificates python3 python3-pip git && sh /repo/populate-preinstall.sh"
opencode-scheduler:
container_name: opencode-scheduler
image: opencode:local
profiles: ["scheduler"]
depends_on: [opencode]
restart: unless-stopped
init: true
working_dir: /home/node/workspace/engine # run_task invokes scripts/... relative to CWD
volumes:
- .:/home/node/workspace
environment:
GRAPHIFY_ROOT: /home/node/workspace
GRAPHIFY_BUILD: if-missing
command: >-
sh -c "scripts/scheduled-runner.sh --task drift-detect --schedule daily
--pid-file /tmp/scheduled-runner.pid && exec tail -F scheduled-runner.log"
volumes:
opencode-data:
================
File: workspace/agents/opencode/Dockerfile
================
FROM node:22-bookworm-slim
ARG OPENCODE_VERSION=2.2.5
# Platform package suffix used by the offline store tarball glob
# (@opencode+cli-linux--.tgz). Override for x64 builds:
# OPENCODE_ARCH=x64 (or x64-baseline depending on store contents).
ARG OPENCODE_ARCH=arm64
ARG GRAPHIFY_VERSION=0.9.100
ARG LEAN_CTX_VERSION=3.12.0
ARG CONTEXT7_VERSION=4.4.4
ARG MCP_MEMORY_VERSION=2.0.1
ARG MCP_EVERYTHING_VERSION=2.1.0
ARG MCP_VERSION=2.3.0
ARG MCP_SERVER_FETCH_VERSION=2026.10.28
# Vendored first-party plugin (@firecrawl/opencode) + its runtime dep
ARG ZOD_VERSION=4.1.8
ARG FIRECRAWL_CLI_VERSION=1.26.24
# Corporate CA certificates.
# Place PEM certificates with the .crt extension under certs/.
COPY certs/ /usr/local/share/ca-certificates/
# Base packages. The optional `preinstall` additional build context (/pi, see
# docker-compose files) pre-seeds cached .deb files so repeat builds skip most
# downloads; the apt cache mounts keep downloaded state out of the image layer.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=from=preinstall,src=/,dst=/pi,ro \
set -eux; \
cp /pi/apt/*.deb /var/cache/apt/archives/ 2>/dev/null || true; \
apt-get update; \
apt-get install -y --no-install-recommends \
git \
curl \
sudo \
iproute2 \
jq \
ripgrep \
ca-certificates \
python3 \
pipx \
python3-venv \
coreutils \
procps; \
update-ca-certificates; \
git config --system --add safe.directory '*'
# Configure Node.js and Python to trust the corporate CA.
ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt \
SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
PIP_CERT=/etc/ssl/certs/ca-certificates.crt
# Locale + terminal defaults so the TUI and ncurses apps render correctly in
# `docker exec -it` sessions (which inherit the container env, not the host's).
ENV LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
TERM=xterm-256color
# Install OpenCode and the Context7 MCP server.
# Context7 is baked (not npx) so library docs lookup works on
# offline / CA-restricted corporate networks without a runtime download.
# Tarball-first from the offline store (/pi/npm), network as fallback.
RUN --mount=type=cache,target=/root/.npm,sharing=locked \
--mount=from=preinstall,src=/,dst=/pi,ro \
set -eux; \
if ls /pi/npm/@opencode+cli-*.tgz >/dev/null 2>&1 \
&& ls /pi/npm/@opencode+cli-linux-${OPENCODE_ARCH}-*.tgz >/dev/null 2>&1; then \
npm install -g --offline --no-audit --no-fund \
/pi/npm/@opencode+cli-*.tgz /pi/npm/@opencode+cli-linux-${OPENCODE_ARCH}-*.tgz \
|| npm install -g --no-audit --no-fund \
/pi/npm/@opencode+cli-*.tgz /pi/npm/@opencode+cli-linux-${OPENCODE_ARCH}-*.tgz; \
else \
npm install -g --no-audit --no-fund "@opencode/cli@${OPENCODE_VERSION}"; \
fi; \
if ls /pi/npm/@upstash+context7-mcp-*.tgz >/dev/null 2>&1; then \
npm install -g --offline --no-audit --no-fund /pi/npm/@upstash+context7-mcp-*.tgz \
|| npm install -g --no-audit --no-fund /pi/npm/@upstash+context7-mcp-*.tgz; \
else \
npm install -g --no-audit --no-fund "@upstash/context7-mcp@${CONTEXT7_VERSION}"; \
fi; \
opencode --version; \
command -v context7-mcp
# lean-ctx binary: from the offline store when present, else upstream installer.
RUN --mount=from=preinstall,src=/,dst=/pi,ro \
set -eux; \
store_ok=false; \
if [ -f /pi/bin/lean-ctx ]; then \
install -m 0755 /pi/bin/lean-ctx /usr/local/bin/lean-ctx; \
lean-ctx --version >/dev/null 2>&1 && store_ok=true || rm -f /usr/local/bin/lean-ctx; \
fi; \
if [ "$store_ok" != true ]; then \
echo "store lean-ctx missing or wrong-arch — fetching from network"; \
curl -fsSL \
"https://raw.githubusercontent.com/yvgude/lean-ctx/v${LEAN_CTX_VERSION}/install.sh" \
-o /tmp/leanctx-install.sh; \
chmod +x /tmp/leanctx-install.sh; \
LEAN_CTX_NO_ONBOARD=1 sh /tmp/leanctx-install.sh --download; \
install -m 0755 /root/.local/bin/lean-ctx /usr/local/bin/lean-ctx; \
rm -rf /root/.local /tmp/leanctx-install.sh; \
fi; \
lean-ctx --version
# Docker CLI (client only - talks to the host daemon via the compose socket mount).
ARG TARGETARCH
ARG DOCKER_CLI_VERSION=27.3.1
RUN case "${TARGETARCH}" in \
arm64) DC_ARCH=aarch64 ;; \
amd64) DC_ARCH=x86_64 ;; \
*) DC_ARCH=x86_64 ;; \
esac \
&& curl -fsSL "https://download.docker.com/linux/static/stable/${DC_ARCH}/docker-${DOCKER_CLI_VERSION}.tgz" -o /tmp/docker-cli.tgz \
&& tar -xzf /tmp/docker-cli.tgz -C /tmp \
&& install -m 0755 /tmp/docker/docker /usr/local/bin/docker \
&& rm -rf /tmp/docker /tmp/docker-cli.tgz
# Install Graphify and its MCP server (wheel-first from the offline store).
ENV PIPX_HOME=/opt/pipx \
PIPX_BIN_DIR=/usr/local/bin
RUN --mount=type=cache,target=/root/.cache/pip,sharing=locked \
--mount=from=preinstall,src=/,dst=/pi,ro \
set -eux; \
if ls /pi/pip/graphifyy-*.whl >/dev/null 2>&1; then \
PIP_FIND_LINKS=/pi/pip PIP_NO_INDEX=true pipx install /pi/pip/graphifyy-*.whl; \
PIP_FIND_LINKS=/pi/pip PIP_NO_INDEX=true pipx inject graphifyy /pi/pip/mcp-*.whl; \
else \
pipx install "graphifyy[mcp]==${GRAPHIFY_VERSION}"; \
pipx inject graphifyy "mcp==${MCP_VERSION}"; \
fi; \
graphify --help >/dev/null; \
command -v graphify-mcp
# Memory + everything MCP servers and the fetch MCP server,
# tarball/wheel-first from the offline store, network as fallback.
RUN --mount=type=cache,target=/root/.npm,sharing=locked \
--mount=type=cache,target=/root/.cache/pip,sharing=locked \
--mount=from=preinstall,src=/,dst=/pi,ro \
set -eux; \
if ls /pi/npm/@modelcontextprotocol+server-memory-*.tgz >/dev/null 2>&1; then \
npm install -g --offline --no-audit --no-fund /pi/npm/@modelcontextprotocol+server-memory-*.tgz \
|| npm install -g --no-audit --no-fund /pi/npm/@modelcontextprotocol+server-memory-*.tgz; \
else \
npm install -g --no-audit --no-fund "@modelcontextprotocol/server-memory@${MCP_MEMORY_VERSION}"; \
fi; \
if ls /pi/npm/@modelcontextprotocol+server-everything-*.tgz >/dev/null 2>&1; then \
npm install -g --offline --no-audit --no-fund /pi/npm/@modelcontextprotocol+server-everything-*.tgz \
|| npm install -g --no-audit --no-fund /pi/npm/@modelcontextprotocol+server-everything-*.tgz; \
else \
npm install -g --no-audit --no-fund "@modelcontextprotocol/server-everything@${MCP_EVERYTHING_VERSION}"; \
fi; \
if ls /pi/pip/mcp_server_fetch-*.whl >/dev/null 2>&1; then \
PIP_FIND_LINKS=/pi/pip PIP_NO_INDEX=true pipx install /pi/pip/mcp_server_fetch-*.whl; \
else \
pipx install "mcp-server-fetch==${MCP_SERVER_FETCH_VERSION}"; \
fi; \
# Bake readabilipy's Readability.js deps: without node_modules in the
# package dir, the fetch server shells out to `npm install` per request,
# which fails air-gapped. Small tree; network-at-build is acceptable
# (same trade-off as the graphify/firecrawl fallbacks above).
JS_DIR="/opt/pipx/venvs/mcp-server-fetch/lib/python3.11/site-packages/readabilipy/javascript"; \
npm install --prefix "$JS_DIR" --no-audit --no-fund; \
test -d "$JS_DIR/node_modules/@mozilla/readability"
# Prepare OpenCode configuration and persistent-data directories.
RUN mkdir -p \
/home/node/.config/opencode \
/home/node/.local/share/opencode \
/home/node/workspace \
&& chown -R node:node /home/node
# Web uploads stage into /tmp/opencode/uploads — must be writable by the node
# user (root runtime processes can otherwise create it root-owned first).
RUN mkdir -p /tmp/opencode && chown node:node /tmp/opencode
# lean-ctx shell allowlist extension: merged on top of the built-in defaults
# (shell_allowlist_extra per lean-ctx docs config-keys reference) so agent sessions
# can run docker/sh/bash/sudo/ss through ctx_shell without refusals. Baked at build
# time because lean-ctx deliberately blocks agents from changing their own policy
# at runtime — the user approved this expansion explicitly.
ARG LEAN_CTX_CONFIG_DIR=/home/node/.config/lean-ctx
RUN mkdir -p ${LEAN_CTX_CONFIG_DIR} \
&& printf 'shell_allowlist_extra = ["docker", "sh", "bash", "sudo", "ss"]\n' \
> ${LEAN_CTX_CONFIG_DIR}/config.toml \
&& chown node:node ${LEAN_CTX_CONFIG_DIR}/config.toml
# Install the entrypoint and normalize Windows CRLF line endings.
# (Placed before COPY engine/ so engine edits don't invalidate these layers.)
COPY entrypoint.sh /usr/local/bin/opencode-entrypoint.sh
RUN sed -i 's/\r$//' /usr/local/bin/opencode-entrypoint.sh \
&& chmod 755 /usr/local/bin/opencode-entrypoint.sh
USER node
# NOTE: the removed custom plugins (model-fallback, dcp, websearch-cited)
# targeted the V1 plugin API and V2 also refuses the old file://
# plugin entries ("configured plugin path must be a directory"), so they
# silently never loaded. Remove this note once they are ported to the V2
# @opencode/plugin API and re-added to config/opencode.json. The first-party
# @firecrawl/opencode plugin is separate: it is V2-native and vendored
# offline at engine/plugins/opencode-firecrawl (see the RUN block below).
# Copy the shared workflow engine. (Kept LAST: engine edits only invalidate
# the layers below, not the package installs above.)
ENV ENGINE_ROOT=/opt/engine
# Engine transform needs root: removes root-owned files, chowns, sets exec bits
USER root
COPY engine/ /opt/engine/
# RAG MCP server: zero-dependency node script (rag/server.js -> /opt/rag).
# Image-baked as the base; compose additionally bind-mounts ./rag:/opt/rag:ro
# so server edits activate without a rebuild (the mount wins when present).
COPY rag/ /opt/rag/
# Vendored @firecrawl/opencode plugin: install its single runtime dependency
# (zod) from the offline store into the plugin dir, and firecrawl-cli
# (binary: firecrawl) globally. Tarball-first from /pi/npm, network fallback,
# same style as the npm blocks above. Runs before the engine transform below
# so the final chown covers the installed node_modules.
RUN --mount=type=cache,target=/root/.npm,sharing=locked \
--mount=from=preinstall,src=/,dst=/pi,ro \
set -eux; \
rm -rf /tmp/zod-install; \
mkdir -p /tmp/zod-install; \
if ls /pi/npm/zod-*.tgz >/dev/null 2>&1; then \
npm install --no-save --no-package-lock --offline --no-audit --no-fund \
--prefix /tmp/zod-install /pi/npm/zod-*.tgz \
|| npm install --no-save --no-package-lock --no-audit --no-fund \
--prefix /tmp/zod-install /pi/npm/zod-*.tgz; \
else \
npm install --no-save --no-package-lock --no-audit --no-fund \
--prefix /tmp/zod-install "zod@${ZOD_VERSION}"; \
fi; \
rm -rf /opt/engine/plugins/opencode-firecrawl/node_modules; \
mv /tmp/zod-install/node_modules /opt/engine/plugins/opencode-firecrawl/node_modules; \
rm -rf /tmp/zod-install; \
if ls /pi/npm/firecrawl-cli-*.tgz >/dev/null 2>&1; then \
npm install -g --offline --no-audit --no-fund /pi/npm/firecrawl-cli-*.tgz \
|| npm install -g --no-audit --no-fund /pi/npm/firecrawl-cli-*.tgz; \
else \
npm install -g --no-audit --no-fund "firecrawl-cli@${FIRECRAWL_CLI_VERSION}"; \
fi; \
test -d /opt/engine/plugins/opencode-firecrawl; \
test -d /opt/engine/plugins/opencode-firecrawl/node_modules/zod; \
test ! -d /opt/engine/plugins/opencode-firecrawl/node_modules/@opencode
RUN set -eux; \
rm -f /opt/engine/.env /opt/engine/.env.template; \
find /opt/engine -type f \( \
-name '*.sh' \
-o -name '*.md' \
-o -name '*.json' \
-o -name '*.js' \
-o -name '*.mjs' \
-o -name '*.ts' \
-o -name '*.py' \
\) -exec sed -i 's/\r$//' {} +; \
if [ -d /opt/engine/.githooks ]; then \
find /opt/engine/.githooks \
-type f \
-exec sed -i 's/\r$//' {} + \
-exec chmod a+x {} +; \
fi; \
find /opt/engine -name '*.md' -not -path '/opt/engine/plugins/*' \
-exec sed -E -i \
-e 's#\.opencode/(agents|skills|rules|usage|contract)/#/opt/engine/\1/#g' \
{} +; \
python3 /opt/engine/scripts/normalize-doc-paths.py /opt/engine /opt/engine; \
find /opt/engine -name '*.sh' -exec chmod a+x {} +; \
chown -R node:node /opt/engine; \
ln -sfn /opt/engine/agent.md \
/home/node/.config/opencode/agent.md; \
ln -sfn /opt/engine/agents \
/home/node/.config/opencode/agents; \
ln -sfn /opt/engine/skills \
/home/node/.config/opencode/skills; \
ln -sfn /opt/engine/commands \
/home/node/.config/opencode/commands; \
ln -sfn /opt/engine/rules \
/home/node/.config/opencode/rules; \
ln -sfn /opt/engine/contract \
/home/node/.config/opencode/contract; \
ln -sfn /opt/engine/usage \
/home/node/.config/opencode/usage; \
ln -sfn /opt/engine/doc \
/home/node/.config/opencode/doc
# Drop back to the runtime user; nothing below needs root.
USER node
WORKDIR /home/node/workspace
ENTRYPOINT ["/usr/local/bin/opencode-entrypoint.sh"]
# entrypoint.sh probes and prefers `opencode serve`; CMD kept in sync with
# that path and serves as the bare fallback.
CMD ["opencode", "serve", "--hostname", "0.0.0.0", "--port", "4096"]
================
File: workspace/agents/opencode/entrypoint.sh
================
#!/bin/sh
set -eu
# Web uploads stage into /tmp/opencode/uploads. The image bakes /tmp/opencode
# node-owned; this guard re-creates it if absent (no-op if root-owned).
mkdir -p /tmp/opencode/uploads 2>/dev/null || true
# -----------------------------------------------------------------------------
# OpenCode authentication
# -----------------------------------------------------------------------------
OPENCODE_AUTH_REQUIRED="${OPENCODE_AUTH_REQUIRED:-true}"
OPENCODE_AUTH_DIR="${OPENCODE_AUTH_DIR:-$HOME/.local/share/opencode}"
OPENCODE_AUTH_FILE="${OPENCODE_AUTH_FILE:-$OPENCODE_AUTH_DIR/auth.json}"
configure_opencode_auth() {
if [ -z "${OPENCODE_API_KEY:-}" ]; then
if [ "$OPENCODE_AUTH_REQUIRED" = "true" ]; then
echo "[entrypoint] ERROR: OPENCODE_API_KEY is required" >&2
exit 1
fi
echo "[entrypoint] OPENCODE_API_KEY is not set; skipping authentication"
return
fi
if ! command -v node >/dev/null 2>&1; then
echo "[entrypoint] ERROR: node executable not found" >&2
exit 1
fi
mkdir -p "$OPENCODE_AUTH_DIR"
chmod 700 "$OPENCODE_AUTH_DIR"
export OPENCODE_AUTH_FILE
node <<'NODE'
const fs = require("fs");
const path = require("path");
const authFile = process.env.OPENCODE_AUTH_FILE;
const apiKey = process.env.OPENCODE_API_KEY;
if (!authFile) {
console.error("[entrypoint] ERROR: OPENCODE_AUTH_FILE is missing");
process.exit(1);
}
if (!apiKey) {
console.error("[entrypoint] ERROR: OPENCODE_API_KEY is missing");
process.exit(1);
}
let auth = {};
if (fs.existsSync(authFile)) {
try {
const content = fs.readFileSync(authFile, "utf8").trim();
if (content) {
auth = JSON.parse(content);
}
} catch (error) {
console.error(
`[entrypoint] ERROR: cannot parse ${authFile}: ${error.message}`
);
process.exit(1);
}
}
if (
auth.opencode &&
auth.opencode.type === "api" &&
auth.opencode.key === apiKey
) {
console.log("[entrypoint] OpenCode authentication already configured");
process.exit(0);
}
auth.opencode = {
type: "api",
key: apiKey
};
fs.mkdirSync(path.dirname(authFile), {
recursive: true,
mode: 0o700
});
const temporaryFile = `${authFile}.tmp`;
fs.writeFileSync(
temporaryFile,
`${JSON.stringify(auth, null, 2)}\n`,
{
encoding: "utf8",
mode: 0o600
}
);
fs.renameSync(temporaryFile, authFile);
fs.chmodSync(authFile, 0o600);
console.log("[entrypoint] OpenCode authentication configured");
NODE
}
configure_opencode_auth
# OpenCode reads the credential from auth.json.
# Remove the original token from the environment inherited by plugins and MCPs.
unset OPENCODE_API_KEY
# -----------------------------------------------------------------------------
# Graphify initialization
# -----------------------------------------------------------------------------
GRAPHIFY_ROOT="${GRAPHIFY_ROOT:-/home/node/workspace}"
GRAPHIFY_GRAPH="${GRAPHIFY_GRAPH:-$GRAPHIFY_ROOT/graphify-out/graph.json}"
GRAPHIFY_BUILD="${GRAPHIFY_BUILD:-if-missing}"
GRAPHIFY_TIMEOUT="${GRAPHIFY_TIMEOUT:-900}"
case "$GRAPHIFY_BUILD" in
always)
build=1
;;
never)
build=0
;;
if-missing)
if [ -f "$GRAPHIFY_GRAPH" ]; then
build=0
else
build=1
fi
;;
*)
echo "[entrypoint] ERROR: invalid GRAPHIFY_BUILD=$GRAPHIFY_BUILD" >&2
echo "[entrypoint] Valid values: if-missing, always, never" >&2
exit 1
;;
esac
if [ "$build" = "1" ]; then
if [ ! -d "$GRAPHIFY_ROOT" ]; then
echo "[entrypoint] WARNING: GRAPHIFY_ROOT not found: $GRAPHIFY_ROOT" >&2
echo "[entrypoint] continuing without building Graphify" >&2
elif ! command -v graphify >/dev/null 2>&1; then
echo "[entrypoint] WARNING: graphify executable not found" >&2
echo "[entrypoint] continuing without building Graphify" >&2
else
echo "[entrypoint] building Graphify graph"
echo "[entrypoint] root=$GRAPHIFY_ROOT"
echo "[entrypoint] graph=$GRAPHIFY_GRAPH"
echo "[entrypoint] timeout=${GRAPHIFY_TIMEOUT}s"
if (
cd "$GRAPHIFY_ROOT" &&
timeout "$GRAPHIFY_TIMEOUT" nice -n 10 graphify update .
); then
if [ -f "$GRAPHIFY_GRAPH" ]; then
echo "[entrypoint] Graphify graph ready: $GRAPHIFY_GRAPH"
else
echo "[entrypoint] WARNING: Graphify completed without creating:" >&2
echo "[entrypoint] $GRAPHIFY_GRAPH" >&2
fi
else
status=$?
if [ "$status" = "124" ]; then
echo "[entrypoint] WARNING: Graphify timed out after ${GRAPHIFY_TIMEOUT}s" >&2
else
echo "[entrypoint] WARNING: Graphify failed with status $status" >&2
fi
echo "[entrypoint] continuing without a Graphify graph" >&2
fi
fi
else
echo "[entrypoint] Graphify graph present or build disabled; skipping build"
fi
# -----------------------------------------------------------------------------
# MCP readiness checks
# -----------------------------------------------------------------------------
#
# OpenCode launches every `local` MCP server from opencode.json as a stdio
# process. Before starting OpenCode, probe each enabled server with a JSON-RPC
# `initialize` handshake so a dead MCP is detected at boot instead of
# mid-session.
#
# Env:
# MCP_CHECK_ENABLED true|false enable the preflight (default: true)
# MCP_CHECK_TIMEOUT seconds per-server timeout (default: 20)
# MCP_CHECK_STRICT true|false exit 1 if any server is not ready (default: false)
MCP_CHECK_ENABLED="${MCP_CHECK_ENABLED:-true}"
MCP_CHECK_TIMEOUT="${MCP_CHECK_TIMEOUT:-20}"
MCP_CHECK_STRICT="${MCP_CHECK_STRICT:-false}"
MCP_CHECK_CONFIG="${OPENCODE_CONFIG:-$HOME/.config/opencode/opencode.json}"
# Function body lives in scripts/boot/mcp-check.sh (moved verbatim); source it
# conditionally so a missing engine degrades to a skip, matching the in-body
# fallback semantics (warn + continue, never fail the boot chain).
if [ -f /opt/engine/scripts/boot/mcp-check.sh ]; then
. /opt/engine/scripts/boot/mcp-check.sh
mcp_check_servers
else
echo "[entrypoint] WARNING: /opt/engine/scripts/boot/mcp-check.sh not found; skipping MCP readiness checks" >&2
fi
# -----------------------------------------------------------------------------
# Engine boot self-check
# -----------------------------------------------------------------------------
#
# Optional preflight that validates the bundled workflow-state-engine before
# OpenCode starts, using the same scripts a developer would run by hand.
#
# Env:
# ENGINE_BOOT_CHECK true|false enable the self-check (default: false)
# ENGINE_BOOT_STRICT true|false exit 1 if any check fails (default: false)
# ENGINE_BOOT_TIMEOUT seconds per-check timeout (default: 30)
ENGINE_BOOT_CHECK="${ENGINE_BOOT_CHECK:-false}"
ENGINE_BOOT_STRICT="${ENGINE_BOOT_STRICT:-false}"
ENGINE_BOOT_TIMEOUT="${ENGINE_BOOT_TIMEOUT:-30}"
# Function body lives in scripts/boot/engine-check.sh (moved verbatim); source
# it conditionally. The in-body /opt/engine guard is preserved; this elif only
# covers the module file itself missing (same warning text as the in-body guard).
if [ -f /opt/engine/scripts/boot/engine-check.sh ]; then
. /opt/engine/scripts/boot/engine-check.sh
engine_boot_checks
elif [ "$ENGINE_BOOT_CHECK" = "true" ]; then
echo "[entrypoint] WARNING: /opt/engine not found; skipping engine boot checks" >&2
fi
# -----------------------------------------------------------------------------
# Start OpenCode
# -----------------------------------------------------------------------------
#
# v2 CLI subcommands/flags are not fully verified yet; probe each launch form
# with --help before exec so an unsupported flag set degrades gracefully
# instead of crash-looping the container.
if opencode serve --help >/dev/null 2>&1; then
echo "[entrypoint] starting: opencode serve --hostname 0.0.0.0 --port 4096"
exec opencode serve --hostname 0.0.0.0 --port 4096
elif opencode web --help >/dev/null 2>&1; then
echo "[entrypoint] starting: opencode web --hostname 0.0.0.0 --port 4096"
exec opencode web --hostname 0.0.0.0 --port 4096
else
echo "[entrypoint] WARN: neither serve nor web accepted; falling back to bare serve" >&2
exec opencode serve
fi
================
File: workspace/agents/opencode/populate-preinstall.sh
================
#!/bin/sh
# Populate the offline package store (pre-install/) used by the Docker build.
#
# pre-install/ is fully gitignored (except its .gitkeep files): its contents
# live only on the local disk / inside the workspace volume and are NEVER
# committed or pushed. The store is portable - copy the workspace to another
# machine and the store comes with it.
#
# Idempotent: every artifact follows the same skip-if-exists pattern - when
# its marker (tarball, wheel, binary, deb, clone dir, zip) is already in the
# store, the fetch is skipped, so re-runs fetch only what is missing. Root is
# required only for the apt section. Store sections: npm pip bin apt git dist.
set -eu
# Versions - must match the Dockerfile ARG defaults (env-overridable).
OPENCODE_VERSION="${OPENCODE_VERSION:-2.2.5}"
OPENCODE_ARCH="${OPENCODE_ARCH:-arm64}" # @opencode/cli-linux-${OPENCODE_ARCH}
OPENCODE_PLATFORM_VERSION="${OPENCODE_PLATFORM_VERSION:-2.2.5}"
CONTEXT7_VERSION="${CONTEXT7_VERSION:-4.4.4}"
GRAPHIFY_VERSION="${GRAPHIFY_VERSION:-0.9.100}"
LEAN_CTX_VERSION="${LEAN_CTX_VERSION:-3.12.0}"
MCP_MEMORY_VERSION="${MCP_MEMORY_VERSION:-2.0.1}"
MCP_EVERYTHING_VERSION="${MCP_EVERYTHING_VERSION:-2.1.0}"
MCP_VERSION="${MCP_VERSION:-2.3.0}"
MCP_SERVER_FETCH_VERSION="${MCP_SERVER_FETCH_VERSION:-2026.10.28}"
# @firecrawl/opencode runtime dependency + CLI companion (offline store pins)
ZOD_VERSION="${ZOD_VERSION:-4.1.8}"
FIRECRAWL_CLI_VERSION="${FIRECRAWL_CLI_VERSION:-1.26.24}"
STORE_DIR=$(cd "$(dirname "$0")" && pwd)/pre-install
mkdir -p "$STORE_DIR/npm" "$STORE_DIR/pip" "$STORE_DIR/bin" "$STORE_DIR/apt" \
"$STORE_DIR/git" "$STORE_DIR/dist"
ADDED=0
SKIPPED=0
mark_added() {
ADDED=$((ADDED + 1))
echo " added: $1"
}
mark_skipped() {
SKIPPED=$((SKIPPED + 1))
echo " skipped: $1"
}
# Generic skip-if-exists guard for one store artifact.
#
# fetch_if_missing MARKER CMD [ARG...]
#
# MARKER is a file (e.g. a downloaded zip) or a directory (e.g. a git clone);
# it counts as present when it exists as a file or holds at least one entry
# as a directory. Present -> return 1 so the caller can report a skip;
# absent -> run CMD with the remaining arguments. Multi-step fetches pass a
# shell function as CMD (POSIX sh has no arrays, so commands are dispatched
# through "$@"). A failed fetch stays fatal: set -e is suspended in an
# if-condition, so a nonzero CMD status is turned into an explicit exit.
fetch_if_missing() {
marker=$1
shift
if [ -e "$marker" ] && { [ -f "$marker" ] || [ -n "$(ls -A "$marker" 2>/dev/null)" ]; }; then
return 1
fi
"$@" || exit $?
}
# --------------------------------------------------------------------------
# npm - one tarball per package.
# --------------------------------------------------------------------------
# True when a tarball for the package (any version) is already in the store.
# npm pack writes scoped packages as '@scope+name-.tgz'.
# $glob is expanded unquoted on purpose (paths in this workspace have no spaces).
npm_tarball_present() {
pkg=${1%@*}
case "$pkg" in
@*/*) glob="$STORE_DIR/npm/${pkg%%/*}+${pkg#*/}-"*.tgz ;;
*) glob="$STORE_DIR/npm/$pkg-"*.tgz ;;
esac
for f in $glob; do
if [ -e "$f" ]; then
return 0
fi
done
return 1
}
npm_maybe_pack() {
if npm_tarball_present "$1"; then
mark_skipped "$1 (tarball already in store)"
else
npm pack "$1" --pack-destination "$STORE_DIR/npm"
mark_added "$1"
fi
}
echo "==> npm tarballs"
npm_maybe_pack "@opencode/cli@${OPENCODE_VERSION}"
npm_maybe_pack "@opencode/cli-linux-${OPENCODE_ARCH}@${OPENCODE_PLATFORM_VERSION}"
npm_maybe_pack "@upstash/context7-mcp@${CONTEXT7_VERSION}"
npm_maybe_pack "@modelcontextprotocol/server-memory@${MCP_MEMORY_VERSION}"
npm_maybe_pack "@modelcontextprotocol/server-everything@${MCP_EVERYTHING_VERSION}"
npm_maybe_pack "zod@${ZOD_VERSION}"
npm_maybe_pack "firecrawl-cli@${FIRECRAWL_CLI_VERSION}"
# NOTE: V1-API custom plugin tarballs (opencode-ai/plugin, model-fallback,
# dcp, websearch-cited) are no longer packed - the plugins are removed from
# the image until they are ported to the @opencode/plugin V2 API.
# --------------------------------------------------------------------------
# pip - full wheel closure.
# --------------------------------------------------------------------------
# True when at least one wheel is already in the store.
pip_wheels_present() {
for f in "$STORE_DIR"/pip/*.whl; do
if [ -e "$f" ]; then
return 0
fi
done
return 1
}
# The container ships pipx but NOT python3-pip; prefer the graphifyy venv's
# pip and fall back to python3 -m pip (e.g. on a dev host).
fetch_pip_wheels() {
if [ -x /opt/pipx/venvs/graphifyy/bin/pip ]; then
/opt/pipx/venvs/graphifyy/bin/pip download --dest "$STORE_DIR/pip" \
"graphifyy[mcp]==${GRAPHIFY_VERSION}" "mcp==${MCP_VERSION}" "mcp-server-fetch==${MCP_SERVER_FETCH_VERSION}"
else
python3 -m pip download --dest "$STORE_DIR/pip" \
"graphifyy[mcp]==${GRAPHIFY_VERSION}" "mcp==${MCP_VERSION}" "mcp-server-fetch==${MCP_SERVER_FETCH_VERSION}"
fi
}
echo "==> python wheels (full closure)"
if pip_wheels_present; then
mark_skipped "pip wheels (already in store)"
else
fetch_pip_wheels
mark_added "pip wheels: graphifyy[mcp]==${GRAPHIFY_VERSION}, mcp==${MCP_VERSION}, mcp-server-fetch==${MCP_SERVER_FETCH_VERSION}"
fi
# --------------------------------------------------------------------------
# bin - release binaries.
# --------------------------------------------------------------------------
# Fetch the lean-ctx release binary via its install script.
fetch_lean_ctx() {
HOME=$(mktemp -d)
curl -fsSL \
"https://raw.githubusercontent.com/yvgude/lean-ctx/v${LEAN_CTX_VERSION}/install.sh" \
-o "$HOME/inst.sh"
LEAN_CTX_NO_ONBOARD=1 sh "$HOME/inst.sh" --download
cp "$HOME/.local/bin/lean-ctx" "$STORE_DIR/bin/lean-ctx"
chmod 0755 "$STORE_DIR/bin/lean-ctx"
rm -rf "$HOME"
}
echo "==> lean-ctx binary"
if fetch_if_missing "$STORE_DIR/bin/lean-ctx" fetch_lean_ctx; then
mark_added "lean-ctx ${LEAN_CTX_VERSION}"
else
mark_skipped "lean-ctx ${LEAN_CTX_VERSION} (binary already in store)"
fi
# --------------------------------------------------------------------------
# apt - .deb closure (root only).
# --------------------------------------------------------------------------
# True when at least one .deb is already in the store.
apt_debs_present() {
for f in "$STORE_DIR"/apt/*.deb; do
if [ -e "$f" ]; then
return 0
fi
done
return 1
}
fetch_apt_debs() {
mkdir -p "$STORE_DIR/apt/partial"
apt-get update
apt-get install --reinstall --download-only -y --no-install-recommends \
-o Dir::Cache::archives="$STORE_DIR/apt" \
git curl jq ripgrep ca-certificates python3 pipx python3-venv coreutils procps
}
echo "==> apt debs (root only)"
if [ "$(id -u)" -eq 0 ]; then
if apt_debs_present; then
mark_skipped "apt debs (already in store)"
else
fetch_apt_debs
mark_added "apt debs"
fi
else
echo " apt section skipped - re-run with: docker compose exec -u root opencode sh populate-preinstall.sh"
fi
# --------------------------------------------------------------------------
# git/ - shallow clones of git-sourced artifacts (reserved for future use).
#
# Pattern: the marker is the clone directory; the clone is skipped while it
# holds at least one entry (an empty or partial clone refetches):
#
# fetch_if_missing "$STORE_DIR/git/" \
# git clone --depth 1 "$STORE_DIR/git/"
#
# Working example (uncomment and fill in when a git-sourced artifact is
# needed; wrap with if/else + mark_added/mark_skipped to keep counts):
#
# echo "==> git clones"
# # fetch_if_missing "$STORE_DIR/git/example-repo" \
# # git clone --depth 1 https://github.com/example/example-repo.git \
# # "$STORE_DIR/git/example-repo"
# --------------------------------------------------------------------------
# --------------------------------------------------------------------------
# dist/ - zips, tarballs, and other direct-URL downloads (reserved for
# future use).
#
# Pattern: the marker is the downloaded file itself; the download is skipped
# while the file exists:
#
# fetch_if_missing "$STORE_DIR/dist/.zip" \
# curl -fsSL -o "$STORE_DIR/dist/.zip"
#
# Working example (uncomment and fill in when a direct-download artifact is
# needed):
#
# echo "==> direct downloads"
# # fetch_if_missing "$STORE_DIR/dist/example-1.0.0.zip" \
# # curl -fsSL https://example.com/example-1.0.0.zip \
# # -o "$STORE_DIR/dist/example-1.0.0.zip"
# --------------------------------------------------------------------------
echo ""
echo "==> done: ${ADDED} added, ${SKIPPED} skipped - store at ${STORE_DIR}"
echo " sections: npm pip bin apt git dist"
================
File: workspace/agents/opencode/README.md
================
# opencode
Self-hosted OpenCode v2 stack in Docker: workflow-state-engine, lean-ctx, Graphify, MCP servers, version-pinned and offline-ready.
[](https://github.com/RizkiRachman/utility/actions/workflows/validate.yml)
This folder packages OpenCode v2 (`@opencode/cli`, npm `anomalyco/opencode`) as a reproducible Docker stack: the workflow-state-engine (16 agents, 66 skills, contract-driven scoring), lean-ctx, Graphify, and Context7 / memory / fetch / everything / rag MCP servers, all version-pinned and pre-populated for offline builds. OpenCode v2 is early software, so this runs as a **beta experience**. The engine itself is documented in [engine/README.md](engine/README.md).
## Table of Contents
1. [Quick Start](#quick-start)
2. [Configuration](#configuration)
3. [Ports and Volumes](#ports-and-volumes)
4. [MCP Servers](#mcp-servers)
5. [Health Checks](#health-checks)
6. [Offline Pre-Install Store](#offline-pre-install-store)
7. [Components](#components)
8. [Project Structure](#project-structure)
9. [Architecture](#architecture)
10. [Enforcement and Quality Gates](#enforcement-and-quality-gates)
11. [Troubleshooting](#troubleshooting)
12. [Known Unknowns](#known-unknowns)
13. [Local State](#local-state)
14. [Contributing and License](#contributing-and-license)
## Quick Start
From the repository root:
```bash
cp workspace/agents/opencode/.env.template .env # fill in your keys
docker compose -f docker-compose.example.yaml up -d --build
open http://127.0.0.1:4096 # OpenCode pairing screen, enter your key once
```
Standalone build and run, without the repository compose file:
```bash
docker build --build-context preinstall=./pre-install -t opencode:local .
docker run --rm -it -p 127.0.0.1:4096:4096 \
-e OPENCODE_API_KEY="" \
-v "$PWD/config/opencode.json":/home/node/.config/opencode/opencode.json:ro \
opencode:local
open http://127.0.0.1:4096 # OpenCode's own pairing screen, enter your key once
```
Both the compose stack and the standalone run above publish OpenCode on `127.0.0.1:4096` (loopback only). The first visit shows OpenCode's own pairing prompt; enter the key there.
The entrypoint boot chain: write `OPENCODE_API_KEY` to `auth.json` and unset it, optionally build the Graphify index, run the MCP readiness preflight, optionally run engine boot checks, then probe launch forms in order: `opencode serve --hostname 0.0.0.0 --port 4096` → `opencode web --hostname 0.0.0.0 --port 4096` → bare `opencode serve`.
## Configuration
Compose file: [`docker-compose.example.yaml`](../../../docker-compose.example.yaml) at the repository root is the canonical entry point; the sibling [`docker-compose.example.yaml`](docker-compose.example.yaml) in this folder runs the same stack standalone. Both publish OpenCode serve's `4096` on `127.0.0.1:4096` (loopback only).
### Runtime environment
| Variable | Required | Default | Description |
|---|---|---|---|
| `OPENCODE_API_KEY` | Yes | `REPLACE_ME` | OpenCode auth; written to `auth.json`, then unset from the env |
| `OPENCODE_SERVER_PASSWORD` | No | (empty) | Auth key for OpenCode's own HTTP basic auth / pairing screen |
| `SUMOPOD_API_KEY` | Yes | `REPLACE_ME` | Model-provider API key |
| `SUMOPOD_BASE_URL` | Yes | `REPLACE_ME` | Model-provider endpoint |
| `SUMOPOD_BASE_MODEL` | Yes | `REPLACE_ME` | Executor model id |
| `SUMOPOD_BASE_MODEL_NAME` | Yes | `REPLACE_ME` | Executor model display name |
| `CONTEXT7_API_KEY` | Yes | `REPLACE_ME` | Context7 docs MCP key |
| `FIRECRAWL_API_KEY` | No | (unset) | Firecrawl key; developer search works keyless, alexandria + firecrawl-cli skills need it |
| `RAG_EMBEDDINGS_URL` | No | (empty) | OpenAI-compatible `/v1/embeddings` endpoint for the rag server; all three `RAG_EMBEDDINGS_*` set switches it to remote embeddings |
| `RAG_EMBEDDINGS_MODEL` | No | (empty) | Embeddings model id at the endpoint above |
| `RAG_EMBEDDINGS_KEY` | No | (empty) | Bearer key for the embeddings endpoint |
| `OPENCODE_AUTH_REQUIRED` | No | `true` | Fail boot when `OPENCODE_API_KEY` is missing |
| `OPENCODE_CONFIG` | No | `/home/node/.config/opencode/opencode.json` | Config path inside the container |
| `GRAPHIFY_ROOT` | No | `/home/node/workspace` | Graphify index root |
| `GRAPHIFY_BUILD` | No | `if-missing` | `if-missing` / `always` / `never` |
| `GRAPHIFY_TIMEOUT` | No | `900` | Graph build timeout (seconds) |
| `MCP_CHECK_ENABLED` | No | `true` | Boot-time MCP readiness preflight |
| `MCP_CHECK_TIMEOUT` | No | `20` | Per-server probe timeout (seconds) |
| `MCP_CHECK_STRICT` | No | `false` | Exit 1 when a server is not ready |
| `ENGINE_BOOT_CHECK` | No | `false` | Validate the bundled engine before launch |
| `ENGINE_BOOT_STRICT` | No | `false` | Exit 1 when a boot check fails |
| `ENGINE_BOOT_TIMEOUT` | No | `30` | Per-check timeout (seconds) |
### Build args
| Arg | Default | Description |
|---|---|---|
| `OPENCODE_VERSION` | `2.2.5` | OpenCode CLI pin |
| `OPENCODE_ARCH` | `arm64` | Platform package (`@opencode/cli-linux-arm64`); x64 builds override |
| `GRAPHIFY_VERSION` | `0.9.100` | `graphifyy[mcp]` pin (PyPI) |
| `LEAN_CTX_VERSION` | `3.12.0` | lean-ctx pin |
| `CONTEXT7_VERSION` | `4.4.4` | `@upstash/context7-mcp` pin |
| `MCP_VERSION` | `2.3.0` | MCP Python SDK pin |
| `MCP_SERVER_FETCH_VERSION` | `2026.10.28` | `mcp-server-fetch` pin |
| `ZOD_VERSION` | `4.1.8` | zod pin (runtime dep of `@firecrawl/opencode`, resolved once) |
| `FIRECRAWL_CLI_VERSION` | `1.26.24` | `firecrawl-cli` pin (binary `firecrawl`) |
Config files: `config/opencode.json` (V2-native schema: `agents`, `permissions` (incl. a `firecrawl_alexandria` ask entry), `plugins` (vendored `/opt/engine/plugins/opencode-firecrawl`), `mcp.servers` + `disabled`, `providers.package`, `instructions` (loads `/opt/engine/agent.md` plus `opencode/config/rag-usage.md`, which tells agents to search the rag index before writing code), `update`, `compaction.keep.tokens`; mounted read-only) and `config/opencode-model-fallback.json` (dormant, kept for a future `@opencode/plugin` port).
## Ports and Volumes
| Host | Container | Notes |
|---|---|---|
| `127.0.0.1:4096` | `4096` | OpenCode web UI (serve); loopback only |
| `./workspace` | `/home/node/workspace` | Working tree mounted into the container (repo compose) |
| `config/opencode.json` | `/home/node/.config/opencode/opencode.json` | Read-only bind mount; restart to apply edits |
| `config/opencode-model-fallback.json` | `/home/node/.config/opencode/opencode-model-fallback.json` | Read-only, dormant |
| `opencode-data` (named volume) | `/home/node/.local/share/opencode` | Auth, session store, memory MCP file |
| `pre-install/` | build context `preinstall` | Offline package store, consumed at build time |
## MCP Servers
Defined in `config/opencode.json`:
| Server | Command | Purpose |
|---|---|---|
| `lean-ctx` | `lean-ctx` | Compressed reads/search/shell, code intelligence, knowledge and session persistence |
| `graphify` | `sh -c "… graphify-mcp …"` | Knowledge graph MCP; configured but `disabled: true` by default, flip the flag and restart to enable |
| `context7` | `context7-mcp` | Up-to-date library docs lookups; needs `CONTEXT7_API_KEY` |
| `memory` | `mcp-server-memory` | Persistent memory graph, writes to `MEMORY_FILE_PATH` in the `opencode-data` volume |
| `fetch` | `mcp-server-fetch` | Keyless single-page HTTP GET with readability extraction (deps baked at build) |
| `everything` | `mcp-server-everything` | MCP diagnostics and test fixtures |
| `rag` | `node /opt/rag/server.js` | Workspace retrieval over SQLite FTS5 + vectors: `rag_search`, `rag_index`, `rag_status`, `rag_get`; java-aware method-boundary chunking with `package > class > method` headings; offline hashed-vector embeddings, optional remote endpoint via `RAG_EMBEDDINGS_*`; agents get usage rules from the `opencode/config/rag-usage.md` instructions entry |
| `@firecrawl/opencode` (plugin) | vendored at `engine/plugins/opencode-firecrawl` | Keyless `firecrawl_developer_search`; `firecrawl_alexandria` is ask-gated; keyed scrape/crawl via `firecrawl-cli` |
## Health Checks
Run from this folder (`workspace/agents/opencode`); every script exits 0 on a healthy tree:
```bash
bash engine/scripts/health-check.sh # aggregates the full suite (6 dimensions)
bash engine/scripts/check-cross-references.sh # link/ID integrity across engine docs
bash engine/scripts/validate-toolkit.sh # symlinks, contract integrity, agent consistency
bash engine/scripts/check-conventions.sh # architecture conventions (non-blocking)
bash engine/scripts/check-doc-style.sh # doc style gate: dashes, vocabulary, placeholders
bash engine/scripts/check-plugins.sh # V2 plugin path rule (no file:// to file)
bash engine/scripts/check-mcp.sh # MCP server binaries present
bash engine/scripts/boot/engine-check.sh # boot-time engine validation
bash engine/scripts/boot/mcp-check.sh # boot-time MCP readiness probe
```
Details: [engine/doc/health-check.md](engine/doc/health-check.md).
## Offline Pre-Install Store
`pre-install/` holds apt `.deb` files, pip wheels, npm tarballs, git sources, and prebuilt binaries keyed to the pinned versions and `OPENCODE_ARCH` (default `arm64`). The compose `opencode-preinstall` service (profile `tools`) is a throwaway `node:22-bookworm-slim` container that fills it; repeat runs skip everything already present. The image build consumes the store via the `preinstall` build context; without it, installs fall back to the network and fail in offline environments. The store is gitignored except `.gitkeep` placeholders.
## Components
| Component | Pin | Source |
|---|---|---|
| OpenCode v2 CLI | `@opencode/cli@2.2.5` + platform pkg `@opencode/cli-linux-${OPENCODE_ARCH}@2.2.5` | npm (`anomalyco/opencode`) |
| Context7 MCP | `@upstash/context7-mcp@4.4.4` | npm |
| Memory MCP | `@modelcontextprotocol/server-memory@2.0.1` | npm |
| Everything MCP | `@modelcontextprotocol/server-everything@2.1.0` | npm |
| Graphify | `graphifyy[mcp]==0.9.100` | PyPI |
| MCP Python SDK | `mcp==2.3.0` | PyPI |
| Fetch MCP | `mcp-server-fetch==2026.10.28` | PyPI |
| lean-ctx | `v3.12.0` | [GitHub release](https://github.com/yvgude/lean-ctx) |
| @firecrawl/opencode | `1.4.0` (ISC, first-party) | vendored at `engine/plugins/opencode-firecrawl` ([upstream](https://github.com/firecrawl/opencode-firecrawl)) |
| firecrawl-cli | `1.26.24` (ISC, binary `firecrawl`) | npm |
**Custom plugins removed** (`@razroo/opencode-model-fallback@0.3.2`, `@tarquinen/opencode-dcp@3.2.0`, `opencode-websearch-cited@1.2.0`): they target the V1 plugin API and V2 additionally rejects their old `file://` entries (`configured plugin path must be a directory`), so they never loaded. They are dropped from the image and `config/opencode.json` until ported to the `@opencode/plugin` V2 API; no fallback executor→reviewer resilience is active until then. Contrast: the first-party V2-native `@firecrawl/opencode` plugin is adopted offline-vendored at `engine/plugins/opencode-firecrawl` (developer search works keyless out of the box); the community V1-era plugins above remain removed.
## Project Structure
```text
opencode/
├── Dockerfile # Image build; consumes the preinstall context
├── entrypoint.sh # Boot chain: auth → graphify → MCP preflight → serve
├── docker-compose.example.yaml # opencode + preinstall + scheduler services
├── rag/ # RAG MCP server (zero npm dependencies)
│ ├── server.js # CLI + MCP stdio; SQLite FTS5 + vectors
│ ├── README.md
│ ├── test.sh # 10-case suite (CLI, Java chunking, MCP, remote embeddings)
│ └── test/ # corpus fixtures, embeddings stub, MCP checks
├── populate-preinstall.sh # Offline-store bootstrap (run inside preinstall service)
├── .env.template # Copy to repository-root .env
├── config/ # opencode.json (V2-native), rag-usage.md agent guidance, dormant fallback config
├── certs/ # TLS certs placeholder (currently empty)
├── pre-install/ # Offline package store (gitignored)
├── session/ # Live orchestration contract + state log (gitignored)
├── sandbox/ # Session playground, one folder per session (contents gitignored)
└── engine/ # Workflow-state-engine → copied to /opt/engine
├── agent.md # Orchestrator instructions (instructions[0])
├── agents/ # 16 agent instruction files
├── skills/ # 66 skill directories
├── commands/ # 1 slash command (impeccable)
├── rules/ # rules.json + rules.schema.json
├── contract/ # Contract templates + JSON Schema
├── scripts/ # Automation & validation scripts (incl. boot/)
├── usage/ # 4 tool guides (lean-ctx, graphify, context7, opencode)
├── doc/ # Workflow + health-check docs
├── templates/ # judge-prompt.md
└── opencode.json.template # Config template (V2-native shape)
```
CI lives at the repository root: [.github/workflows/validate.yml](../../../.github/workflows/validate.yml) runs JSON validity, a `bash -n` sweep, cross-reference and doc style gates, and contract scoring on every push.
## Architecture
*Generated by [engine/scripts/generate_diagrams.py](engine/scripts/generate_diagrams.py); regenerate with `python3 engine/scripts/generate_diagrams.py` from the repository root.*
```text
Host browser → 127.0.0.1:4096 (loopback, published) → opencode serve :4096
entrypoint.sh boot chain:
1. auth: OPENCODE_API_KEY to auth.json, then unset
2. Graphify build: if-missing / always / never, timeout 900
3. MCP preflight: JSON-RPC initialize per enabled local server
4. engine boot checks: validate-contract --score, check-plugins
5. launch probe: serve, then web, then bare serve
opencode serve attaches to:
- MCP servers: lean-ctx, context7, memory, fetch, everything, rag (graphify disabled by default)
- Firecrawl plugin: keyless developer search, alexandria ask-gated
- workflow-state-engine: 16 agents, 66 skills, session/, sandbox/
```
## Enforcement and Quality Gates
| Gate | Checks | Wired into |
|---|---|---|
| `engine/scripts/check-cross-references.sh` | 8-check link/ID integrity + context-standards + access-policy verification | `health-check.sh` lens-4, `validate-toolkit.sh`, pre-commit hook, CI |
| `engine/scripts/check-doc-style.sh` | Doc style gate: em/en dashes, AI vocabulary, placeholders in shipped docs | `health-check.sh --check docs`, `validate-toolkit.sh`, pre-commit hook, CI |
| `engine/scripts/check-plugins.sh` | Rejects `file://`-to-file plugin paths (V2 rule) | `setup.sh`, `validate-toolkit.sh`, engine boot checks |
| `engine/scripts/validate-contract.sh --score` | 3-tier contract scoring (PASS ≥ 70) | CI, engine boot checks |
| `engine/scripts/check-conventions.sh` | Rule 1 FAILs stale contract paths; INFOs the V2 chain | `validate-toolkit.sh`, CI (non-blocking) |
| `engine/rules/rules.schema.json` | `store_in` target schema + required top-level keys for rules | `validate-toolkit.sh` |
| `engine/.githooks/pre-commit` | `bash -n` sweep + cross-references + doc style | `git config core.hooksPath engine/.githooks` |
| `.github/workflows/validate.yml` (repository root) | JSON validity, `bash -n`, cross-refs, doc style, contract score, conventions | Runs on every push and pull request |
| humanizer gate (`/skill humanizer`) | Strips AI writing patterns (em dashes, rule-of-three, promotional tone) from shipped user-facing docs; internal state files (`session/state.md`, contract JSON, code comments) exempt | `agent.md` ship flow, `engine/skills/release-plan`, `engine/skills/shipping-and-launch` |
| context gate (`/skill context-standards`) | Loads target-project pattern standards (`context/*.md`) before edits and verifies them in review | `agent.md` Execute step, `developer.md`, `quality-analyst.md`, cross-reference check 7 |
| `engine/scripts/loop-validate.sh` | In-loop executor done-gate: JSON parse, `bash -n` sweep, cross-references; mirrors CI locally | developer / fixer / system-analyst rule 7, orchestrator scoring table (Tier 1) |
| Agent permission tiers (`config/opencode.json`) | Read-only agents deny edit, shell, webfetch, question; global edit-deny on `**/opencode.json` and `**/contract.json` | OpenCode permission checks on every tool call |
| Sandbox policy (`rules.json → enforcement.sandbox`) | Scratch work confined to `sandbox//`; own folder only, root never written, contents gitignored, promote before COMPLETE | `engine/agent.md` section 6, writer agents' Permissions blocks, `sandbox/README.md` |
## Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| Build fails on x64 with missing platform package | The `pre-install/` store matches `OPENCODE_ARCH` (default `arm64`) | Set `OPENCODE_ARCH=x64`, re-run the preinstall profile, rebuild |
| Forgot the key | OpenCode's pairing screen expects `OPENCODE_SERVER_PASSWORD` from the repository root `.env` | Run `grep OPENCODE_SERVER_PASSWORD .env` from the repository root; change it there and restart the `opencode` service |
| Web UI unreachable, port already allocated | Host port 4096 is taken by another process | Change the host side of the `ports:` mapping in the compose file |
| Config edits have no effect | `config/opencode.json` is mounted read-only | Restart the container after editing; the mount itself never changes |
| `fetch` MCP fails to extract pages | Image predates the baked readabilipy dependencies; older builds installed them per request and needed network | Rebuild from this Dockerfile; the readability node modules are baked in |
| Memory disappears between sessions | Memory MCP persists to `MEMORY_FILE_PATH` inside the `opencode-data` volume | Removing the volume removes memory; inspect `docker volume` before wiping |
| `rag_search` returns nothing | The index is empty or the server reads a different database than expected | Run `rag_index` (or `node /opt/rag/server.js index`), then check `rag_status`: it prints the db path, the indexed root, and a stale file count; make sure `RAG_DB` matches the path you expect |
| rag embedding mode looks wrong | An index keeps the embedding mode it was built with; `rag_search` uses that mode, and switching `RAG_EMBEDDINGS_*` only takes hold on the next `rag_index` | Re-run `rag_index` after changing the env; `rag_status` prints `embedding_mode` and `dims` |
## Known Unknowns
- **V2 config schema** (resolved as of 2026-11-01): `config/opencode.json` migrated to the V2-native shape; deprecated keys (`agent`, `permission`, `mcp.`, `autoupdate`, `compaction.reserved`, `bash`) must not be reintroduced.
- **Plugin API**: the three custom plugins were removed (see [Components](#components)); port them to `@opencode/plugin@2.x` before re-adding.
- **`opencode web` flags**: v2 CLI subcommands are not fully verified. `entrypoint.sh` probes at boot: `opencode serve --hostname … --port …` → fallback `opencode web --hostname … --port …` → fallback bare `opencode serve`.
## Local State
Never committed or pushed:
- `pre-install/`: offline package store (gitignored except `.gitkeep` placeholders).
- `session/`: **project-local runtime state**. Compose mounts the workspace at `/home/node/workspace`, so this `session/` *is* the container's global session directory: `contract.json` (seeded from `engine/contract/contract.template.json` via `snapshot-contract.sh`), `state.md`, `state.example.md` (tracked), `archive/`. Gitignore policy: `session/*` ignored, `state.example.md` negated.
- `sandbox/`: **session playground**. One folder per live session (`sandbox//`) for probes, clones, and rendered tests. Tracked: `README.md`, `.gitignore`, `.gitkeep`; everything else is gitignored on the `certs/` pattern. Rules: [sandbox/README.md](../../../sandbox/README.md).
The engine reference ([engine/README.md](engine/README.md)) covers internals.
## Contributing and License
The stack is internal and not open for external contribution. Changes are made in place; the CI workflow at the repository root and a PR template ([.github/PULL_REQUEST_TEMPLATE.md](../../../.github/PULL_REQUEST_TEMPLATE.md)) apply the usual gates automatically. For local commits, enable the pre-commit hook with `git config core.hooksPath engine/.githooks`. No license file ships with this repository.
================================================================
End of Codebase
================================================================