`knowledge-layer/SKILL.md`

101 lines
---
name: knowledge-layer
description: "Build and maintain an AI Knowledge Layer — a plain-directory, LLM-compiled wiki (the Karpathy / gBrain pattern) that agents read FIRST before any task. Instead of RAG or a vector database, the LLM reads curated sources once and compiles them into typed entry folders (people, companies, concepts, ideas, projects, operations, media) with per-fact provenance, cross-references, and a contradictions log, then refreshes incrementally as sources change. Use this when the user wants a persistent, read-first shared brain / knowledge base / memory layer for a project, company, domain, or fleet of agents; when onboarding agents that must wake up with full context; when turning transcripts, chats, docs, campaigns, threads, or research into a durable queryable wiki without vector infrastructure; or when they say knowledge layer, gBrain, second brain for my agents, compile my notes into a wiki, Karpathy knowledge base, read-first memory, or /knowledge-layer. Not for one-off Q&A over a single file, and not a RAG or embeddings pipeline."
---

# Knowledge Layer

## Overview

A Knowledge Layer turns curated raw material into a structured wiki that a fleet
of agents reads before acting and writes durable context back to. It implements
Andrej Karpathy's LLM-knowledge-base idea (popularized as "gBrain"): **read the
sources once, compile a maintained wiki** — summaries, cross-references, and
contradictions handled by the LLM — instead of re-retrieving raw chunks through
RAG on every question. There is no vector database; the directory structure is
the index.

## When to use

Use this skill to **create** a new knowledge layer, **compile** sources into it,
**query** it read-first, or **maintain** its health. Reach for it whenever
someone wants a persistent shared brain for a project/company/domain or agent
fleet — not for single-file Q&A and not to stand up an embeddings pipeline.

## Tools in this skill

All scripts are stdlib-only Python 3 (no installs) and cost **$0** to run.
Run any with `-h` for full flags.

| Script | Purpose |
|--------|---------|
| `scripts/kl_init.py [path]` | Scaffold a new layer (typed folders + SCHEMA/INDEX/CONTRADICTIONS/SOURCES + manifest). |
| `scripts/kl_scan.py [root]` | Content-hash `sources/` vs. the manifest → the incremental compile work-list (NEW/CHANGED/REMOVED). `--commit` after compiling. |
| `scripts/kl_query.py "<q>" [root]` | Read-first retrieval (keyword + frontmatter + cross-ref neighbours, no vectors) → the entries an agent should load. `--print` to dump bodies. |
| `scripts/kl_status.py [root]` | Health report: pending sources, open contradictions, orphan cross-refs, entries missing provenance. |

## Workflow

### 1. Create the layer
```bash
python3 scripts/kl_init.py <path>      # default ./knowledge-layer
```
This drops the typed folders and the four control docs (`SCHEMA.md`, `INDEX.md`,
`CONTRADICTIONS.md`, `SOURCES.md`). Read `SCHEMA.md` — it defines the entry
format everything else depends on.

### 2. Add sources
A human curates what goes in `sources/` (transcripts, chats, docs, contracts,
research, saved threads). Quality in, quality out — the LLM compiles; it does not
decide what is worth ingesting.

### 3. Compile
```bash
python3 scripts/kl_scan.py <root>      # shows NEW / CHANGED / REMOVED
```
Then follow **`references/compile-protocol.md`** exactly: read the work-list
sources, extract atomic facts, route them into one-entity-per-file typed entries
with per-fact provenance, cross-reference with `[[wikilinks]]`, and — critically
— **log contradictions instead of silently overwriting**. Finish with
`kl_scan.py <root> --commit`, then `kl_status.py <root>` and clear what it flags.

Only work the changed sources — incremental compilation is what keeps a large
layer maintainable.

### 4. Use it (read-first / write-back)
Before an agent does a task, load context:
```bash
python3 scripts/kl_query.py "renewal terms for our SaaS vendors" <root> --print
```
The agent reads the returned entries first, acts, then writes durable new
decisions back as fresh source notes → recompile. That read-first / write-back
loop is what makes the layer compound.

### 5. Keep it healthy
Run `kl_status.py <root>` on a cadence. When it flags drift (orphan links,
missing provenance, a growing contradiction backlog), do a consolidation pass
per **`references/wiki-principles.md`** — merge duplicates, prune dead
cross-refs, re-summarize.

## Core rules (see references for detail)

- **No RAG, no vectors** — the structure is the index.
- **Provenance on every claim** — an unsourced fact is not allowed.
- **Surface contradictions, never resolve them silently** — a human decides.
- **One entity per entry; narrow beats broad.**
- **Incremental compile** — recompile only what changed.
- **Summary-first, read-first, write-back.**

## References

- `references/compile-protocol.md` — the exact LLM compile procedure. Read it
  every compile pass.
- `references/wiki-principles.md` — the "layer you keep vs. abandon" disciplines;
  read before a consolidation pass.

## Attribution

Pattern credit: Andrej Karpathy's LLM knowledge-base workflow, and Shann
Holmberg (@shannholmberg), who popularized the "gBrain" / AI-Knowledge-Layer
framing this skill is built from.