log
Brain - chronological log
Append-only. One dated entry per source ingested or milestone worth remembering.
| Date | Source | Entry |
|---|---|---|
| 2026-07-24 | brain (kit) | Created the Brain kit: agent-driven compounding learning convention for YouTube / blog / paper sources. Built on three ideas (layered lazy context, close-the-loop, ground-every-claim); adds multimodal ingest (agent view = the vision model) and the corroboration gate. Personas: curator, fact-checker, synthesizer, mentor, architect. |
| 2026-07-24 | brain (kit) | Extended to a fourth source type: code repositories (clone-and-learn, not build). Added the code-explorer persona, MAP.md + git-ignored repo/, generalized the corroboration gate to evidence leg ↔ claim leg (visual↔text / code↔docs) with docs↔code divergence as a first-class finding, path:line @sha citations, and an inferencing seed topic. |
| 2026-07-25 | 260725_closed-loop-evals-multimodal-agent | First real source ingested (video, Uber "Building Closed-Loop Evals for a Multimodal Agent at Scale", AI Engineer WF 2026). Captured auto-captions + 11 scene-deduped slides; gated 15 nodes (12 corroborated, 3 single-leg). Created new topic evals (emerging) and promoted claims into it + agents; 10 rows to claims.md, 6 terms to glossary.md. |
| 2026-07-25 | 260725_12-factor-agents | Ingested Dex Horthy's 12-Factor Agents (AI Engineer WF 2025). Gated 18 nodes (16 corroborated, 2 single-leg) + 2 divergences. New method: phash dedup over-collapsed slides sharing a template, so switched to transcript-anchored frame extraction triaged via contact sheets (2 view calls instead of 17) - reusable for slide-heavy talks. First external corroboration in the brain: fetched the canonical humanlayer/12-factor-agents README to verify the factor numbering (the talk delivers them out of order), matched exactly. Created new topic context-engineering (emerging, ADR-0001); promoted into it + agents, which reached established on the one cross-source claim (small scoped LLM steps inside deterministic software - S1 and S2 converging from practice and theory). 13 rows to claims.md, 8 terms to glossary.md, ramp-up report to reports/. |
| 2026-07-25 | brain (kit) | Added an optional deep-research stage (ADR-0002): external-evidence pass between the gate and distillation, triggered only on "deep research". Targets gated node IDs (not topics), returns supports/contradicts/refines/no-evidence, weighs sources by tier T1-T5 under a hard independence rule, and writes a permanent note to sources/<id>/context/. Read Copilot CLI's /research docs first - borrowed its autonomous stance + Confidence assessment section, inverted its throwaway session storage. Contract in AGENTS.md; Claude Code wrapper in .claude/commands/research.md. |
| 2026-07-25 | 260725_12-factor-agents | First deep-research pass (R1) - the new stage's shakedown run. 5 searches / 8 fetches. Closed both open questions: context degradation is now measured (Lost in the Middle/TACL T1; Chroma Context Rot, 18 models, T2; Anthropic's n² "attention budget" T2) and the micro-agent claim is quantified (+13.1 to +41.5 pp reliability from decomposition across 10 models, T3 preprint). Found an independent third party (Anthropic) matching S2 near-verbatim on frameworks obscuring prompts. Two findings S2 does not contain: naive memory scaffolds hurt 6 of 10 models (decompose, don't remember), and factors 3/5/6/12 are Event Sourcing (Fowler 2005) with three known failure modes the talk never mentions. Claims 14/17/22/23 upgraded to externally corroborated; claims 24-25 added. |
| 2026-07-25 | brain (kit) | Made the visual leg optional (ADR-0003) - the mirror of ADR-0002's opt-in research: frames are analysed by default, skipped when the user says "don't analyze video" (podcast / webcam interview - the picture never changes, so the tokens buy nothing). Added a free static-video probe: scene-detect + phash dedup is already the first shell step, so <= 3 distinct frames auto-degrades to transcript-only. The cost is recorded, not discovered - a skipped visual leg makes every node single-leg by construction (a transcript agreeing with itself is not two legs), tracked in a new SOURCE.md Visual leg field. Deep research is the designated way back to a second leg. |
| 2026-07-25 | brain (kit) | Added validate.py - a type checker for the prose contract (ADR-0004). The kit's strength (capabilities ship as paragraphs) has one structural weakness: prose has no compiler, so drift accumulates silently. 10 checks enforce what AGENTS.md already requires - INDEX integrity both ways, legal statuses, every frame cited, every claim cited, log chronology, unique ADRs, resolving links, balanced mermaid, no em dashes. Stdlib only; CI on every push and PR. Form, never judgement - it cannot tell you a claim is corroborated, and encoding that would launder judgement as a green check. Mutation-tested against a corrupted copy; the first run found 3 bugs, all in the validator rather than the repo. |
| 2026-07-25 | brain (kit) | Froze the mechanical ingest steps as tools/ingest.py (ADR-0005) - a toolbox, not a pipeline: transcript, probe, frames, sheet, composed per source, no do-everything entrypoint. Overrides prd.md §10's "wait for 5-10 ingests" trigger on one narrow ground: ADR-0003 created a contract rule with no canonical implementation - "<= 3 distinct frames" depends on a scene threshold and a hash distance that lived only in prose, so two agents would reach different verdicts on the same podcast. Principle: generate what should vary, freeze what should not. Judgement stays in prose (same line validate.py draws). Verified: transcript reproduces the 12-factor hand-run byte-for-byte; probe discriminates static (0 distinct) from rich (5). Testing caught a silent-failure bug - run_ffmpeg ignored ffmpeg's exit code, so any error looked like a static video and would have skipped the visual leg without saying why. |
| 2026-07-25 | brain (kit) | Every diagram now carries a walkthrough - a hard rule in AGENTS.md, enforced by validate.py. Generated diagrams were being dropped into LEARNING.md and reports with no explanation, which teaches nobody: the reader who understands it does not need it, the reader who does not is no better off. Required shape, written as a mentor ramping up an engineer: orientation (how to read it, legend for colour), the crux (one bold sentence - if you cannot name it, delete the diagram), why it is shaped this way (rationale + what a different shape would break - the part that transfers judgement), provenance (synthesized from nX so generated material never reads as sourced). Named anti-pattern: narrating the arrows. Validator found 5 undocumented diagrams; all backfilled. Templates now model the scaffold. |
| 2026-07-25 | brain (kit) | Wired the diagram-walkthrough rule into the personas - the gap found by auditing the trigger chain. The rule was in AGENTS.md, the templates and validate.py, but not in the four personas that actually draw diagrams (curator, mentor, synthesizer, code-explorer) - i.e. absent at the exact moment of action. mentor.md had said only "validate it parses". Each now carries a short pointer (not a duplicated contract) framed for that persona's failure mode. Reinforcement now at 4 points: contract -> persona at point of action -> template scaffold -> validator. |
| 2026-07-25 | brain (kit) | Reconciled the docs with the kit (v0.16). Five commits of contract work (v0.10-v0.15) had landed in AGENTS.md, the ADRs and this log but never reached the three files a new reader opens: validate.py appeared in none of README.md / how_to_use_this.md / prd.md, tools/ingest.py in prd only, deep research in neither user-facing doc. Added prd §4.1 (the two frozen scripts and the form-vs-judgement line), the missing ADR rows to the §10 decisions table, R8-R10, resolutions for OQ1/OQ2/OQ5, and changelog v0.10-v0.16. Corrected three statements the repo had outgrown: the Agent-Skills revisit trigger (superseded by ADR-0005), "where packaging would belong" (it landed in-tree), and "script-free kit". Fixed a numbering defect in how_to_use_this.md (a headingless section made §6.5 jump to §8) by writing the missing deep-research section. |
| 2026-07-25 | 260725_oauth2-oidc-plain-english | Ingested Nate Barbettini's OAuth 2.0 and OpenID Connect (in plain English) (Okta, 2018). First non-AI source - flagged against the domain guardrail and ingested deliberately as prerequisite substrate: an agent calling a tool on a user's behalf is the delegated-authorization problem. Gated 20 nodes (14 corroborated, 5 single-leg, 1 divergence). First topic taken from seed to emerging: agent-security (previously 0 sources), promoted on three transferable primitives - scopes as least privilege enforced at the resource server, consent generated from the requested scopes, and channel separation. Best transferable idea: make the untrusted leg carry only useless material - the authorization code crosses the browser precisely because stealing it achieves nothing. 5 rows to claims.md (26-30, the brain's first non-agent claims), 8 terms to glossary.md. First recorded temporal divergence (n17): the talk's implicit-flow recommendation for SPAs has been reversed by the field - both of the source's own legs agree, so it passes the gate as a 2018 statement, but internal consistency is not currency. Marked do not apply, excluded from promotion as guidance, and flagged as the prime deep-research target. The "What has aged" section is explicitly labelled commentary, not sourced evidence - it rests on agent background knowledge, not on any verified external source. Deliberately did not promote into mcp.md: the belief that MCP auth builds on OAuth 2.1 is uncited, and is recorded as an open question instead. |
| 2026-07-25 | 260725_harness-design-long-running-apps | First blog ingest - Anthropic Labs, Harness Design for Long-Running Application Development (2026-03-24). Closes one third of R10 (prd.md §5.2 was prose that had never run). Gated 24 nodes: 6 corroborated (table), 18 single-leg. Visual leg skipped - all 8 figures are outcome screenshots of generated apps, no architecture diagram exists (confirmed by a targeted second fetch), so the pixels carry no second leg; skipping also avoids copying a vendor's product screenshots into the repo. Best transferable idea: every harness component encodes an assumption about what the model cannot do alone, and those assumptions expire - the article deletes its own sprint construct on a stronger model. Two topics promoted to established: context-engineering (S4 needs S2's serialisation artifact to make context resets work - convergence from opposite directions) and evals (S1's QA gates and S4's evaluator both land on an independent checker that can fail the work). agents gained a third source. Refines claim 24 rather than contradicting it: decomposition helps until the capability boundary moves past your task - recorded with the evidence asymmetry noted (10-model study vs vendor n=1). 7 rows to claims.md (31-37), 5 terms to glossary.md. Shakedown findings for the blog flow: (1) web_fetch declines verbatim reproduction, which is the right outcome - raw/ now holds a derived structural outline, clearly marked, and citations point at the original by section, not at the outline; (2) prd.md §5.2's "download meaningful figures" needs the ADR-0003 escape hatch spelled out for blogs, since outcome screenshots are the blog analogue of a static video; (3) a blog's own tables act as a genuine but weak second leg for numeric claims - one author agreeing with himself in two renderings, which raises extraction confidence only. |
| 2026-07-26 | 260726_dont-ship-skills-without-evals | Ingested Philipp Schmid's Don't Ship Skills Without Evals (Google DeepMind, AI Engineer WF 2026). Run with architect + mentor on the analysis leg by request, fact-checker at the gate. Gated 29 nodes (24 corroborated, 5 single-leg) - the strongest two-leg source yet, because the deck is ~20 dense slides narrated closely. First topic taken from seed to emerging with third-party evidence: skills (previously 0 sources) - SkillsBench is a public benchmark, not the speaker's assertion. evals reached 3 sources and agents 4. 10 claims promoted (38-47), 6 terms to glossary.md. Claim 17 gained its second ingested source - a talk about skills landing on the same script-not-inference boundary as a talk about agents, which is the more interesting convergence because the two authors were not answering the same question. First case of one source supplying the method for another's principle: claims 44/45 instrument S4's claim 31 (harness assumptions expire) with ablation, and add what S4 lacks - keep the eval after removing the component, so a reversal is detectable. Toolbox defect found, second occurrence of its class: tools/ingest.py probe returned candidates=3 -> STATIC on a continuously-changing 20-slide deck, because a heavy shared template (fixed background, logo, speaker inset, footer) suppresses whole-frame scene delta. 260725_12-factor-agents hit the same root cause at the phash-dedup stage. Probe overridden for 1 view call (9 spread frames, one contact sheet); the override recovered n11, n13 and n28, whose numbers exist only on the slides. Not fixed in the tool - a constants change breaks cross-source comparability and is an architect decision, raised for a call rather than taken. New curation rule recorded: keep a frame only when it adds data the transcript lacks; 11 of ~20 kept, 6 pruned and cited by slide title instead, which keeps the gate honest without retaining files that teach nothing. |
| 2026-07-26 | brain (kit) | Made the static probe's STATIC verdict advisory (ADR-0006) and reconciled the docs with five ingests (v0.17). The defect, found the hard way an hour earlier: scene detection measures whole-frame delta, so a templated conference deck - fixed background, logo, speaker inset, footer, slide body a minority of the pixels - reads as static while every slide changes. Two occurrences at two stages: 260726_dont-ship-skills-without-evals (candidates=3 on ~20 dense slides) and 260725_12-factor-agents (pHash over-collapse). The fix is additive, and that is the whole design: on STATIC only, probe writes a 9-frame chronological confirmation sheet to view first. The three pipeline constants are untouched, so no past verdict needs re-measuring - confirming a verdict is not recomputing it, which is what lets ADR-0006 coexist with ADR-0005 instead of eroding it. Rejected: lowering SCENE_THRESHOLD (breaks comparability, trades one false class for another), --crop on probe (you cannot know the crop region without looking first - circular), and leaving it to prose (it already failed twice that way; the check belongs in the tool's output, at the point of action). Justified by asymmetry: a false RICH wastes one view; a false STATIC destroys the source's second leg and cannot be undone, since the degrade rule forbids retro-marking nodes corroborated. Verified against a synthetic templated deck built for the purpose - 9 distinct slides, candidates=0 -> STATIC reproducing the bug, differences plainly visible on the generated sheet. Docs reconciled: R10 corrected (was "two ingests, both video"; now 5 ingests, 4 video + 1 blog, blog flow exercised, paper and code still unrun), new R11 for composited recordings, OQ3's stale "170 lines" refreshed to ~250 across 4 sources with the scaffolding thread named as the likely split line, plus AGENTS.md § "The visual leg", the toolbox table, the degrade table, prd §5.1 and the decisions table. |
| 2026-07-31 | 260731_chatgpt-memory-dreaming | Ingested OpenAI's Dreaming: Better memory for a more helpful ChatGPT (2026-06-04). Second blog ingest, and the first source where the visual leg is product screenshots - worth recording as a distinct class: a screenshot is not the author restating himself in a second rendering (the weakness noted for S4's tables), it is a photograph of the artifact the prose describes, so it genuinely corroborates that a mechanism shipped - while proving nothing about whether it works. Gated 11 nodes (9 corroborated, 2 single-leg) plus 3 recorded drops. New topic memory (ADR-0007), status emerging; claims 48-55 and 5 terms promoted. Capture note: the live URL 403s to every user agent tried (Cloudflare); captured from the Internet Archive snapshot, section headings as the citation anchor. The defining limitation, recorded three times over: the article's three eval charts are client-rendered and render as Loading… in any static capture, so every performance claim in the source reduces to the word "improves" - the topic ships with mechanism and zero measurement, and nodes.md d1 exists to stop "improves" hardening into a number later. Best transferable idea: staleness is not irrelevance - memory written during a conversation keeps that conversation's tense, so it decays into confident wrongness; a missing fact degrades an answer, a stale fact poisons it. Corollaries: revision beats expiry (age is information, not invalidity), and representation is a maintenance decision - a flat append-only list makes the required edit inexpressible. Deliberately left unresolved: claims 48-55 sit against claim 24, the only measured memory evidence in this brain (naive episodic scaffolds hurt 6 of 10 models). S6 argues that was the broken design; S6 has no numbers. Recorded as a live conflict in claims.md, topics/memory.md and topics/context-engineering.md rather than resolved in the vendor's favour. A second capture was in flight the whole time, and I misread it - recorded because the mistake generalises. sources/260731_agent-memory-and-dreaming/ (Anthropic talk on memory + dreaming in Claude Managed Agents) was sitting untracked with an unfilled SOURCE.md, which I read as "interrupted last session" and handled per the degrade table. It was actually another agent working the source concurrently - file mtimes showed it writing a fresh transcript-anchored frame set minutes into this pass. I had renamed and then moved its working directory out from under it. Restored to its original path; nothing was lost (video, both VTT tracks, transcript, all frames intact). The tell I should have checked and did not: mtimes. An unfilled template says nothing about whether a process is live; ls -la does. Consequence for whoever finishes that source: memory already exists as a topic - merge into it and take it to established, do not create a second note - and claims 48-55, label S6 and ADR-0007 are already taken. Left as HANDOFF.md in that folder. |
| 2026-07-31 | 260731_chatgpt-memory-dreaming | Recovered the eval charts I had recorded as unrecoverable - the pass's biggest correction, and the technique generalises. nodes.md d1 said the article's three charts were client-rendered Loading… placeholders and that S6 therefore contributed "mechanism and zero measurement". Wrong: they are Vega-Lite components (reactComponent: "VegaChart"), and a Vega-Lite spec carries its own data inline, so the full data.values arrays were sitting in the Next.js RSC flight payload (self.__next_f) of the HTML I had already downloaded. Extracted to chart_data.json. Reusable rule, now in that source's SOURCE.md: when a chart will not render in a static capture, grep the payload for the charting library before declaring the numbers lost - Vega/Vega-Lite, Plotly and Chart.js all embed data in the spec, and reading a spec is less lossy than viewing a rendered chart, because you get the publisher's exact values instead of pixel estimates. What the numbers say (task success %, 2024 saved memories -> 2025 + Dreaming V0 -> 2026 Dreaming V3): factual recall 41.5 -> 67.9 -> 82.8; preference adherence 31.4 -> 55.3 -> 71.3; staying correct over time 9.4 -> 52.2 -> 75.1. Three readings the article never states: staleness was catastrophic (a 9.4% baseline is wrong nine times in ten - the only quantitative backing the write-once diagnosis has), introducing dreaming beat refining it (2024->2025 exceeds 2025->2026 on every objective), and the ceiling is low (71-83% in 2026, so memory still fails ~1 task in 5 on the vendor's own measure). Added nodes n12-n14, claims 56-57. The claim-24 "tension" was sharpened and largely dissolved: with numbers in hand it is clear the two results measure different things - claim 24 tests naive episodic append-and-retrieve on agent long-horizon reliability; S6 tests a maintained synthesized user model in a chat assistant on recall/adherence/freshness. S6 agrees with claim 24's premise. What no source here measures is whether a maintained memory helps an agent - now the topic's headline open question, and the best thing the parallel Anthropic ingest could settle. Caveat retained everywhere: exact figures, undisclosed methodology - "task success" is never defined and no n, eval set or CI is published. |
| 2026-07-31 | 260731_claude-memory-dreaming | Ingested Anthropic's Memory and dreaming for self learning agents ("Code w/ Claude", 2026-05-21) - the independent second leg for memory, deliberately paired with S6 ingested hours earlier. Gated 23 nodes (19 corroborated, 4 single-leg) + 2 recorded gaps. Static probe candidates=85 distinct=45 -> RICH; transcript-anchored extraction, 50 candidates triaged in 7 view calls via contact sheets, 13 frames kept. memory promoted emerging -> established (ADR-0008); claims 58-64, 5 glossary terms. The promotion is narrower than the status suggests, and the ADR exists to say so: two vendors independently shipped the same architecture under the same name, which is strong evidence the design is the natural answer and no evidence at all that it works - convergence would look identical if both were wrong. Confidence on individual claims therefore does not move; only claim 58 has two legs. The finding that mattered more than the promotion: S7 was the source that could have closed claim 24's gap - first source on the right side of both axes (maintained memory, agent platform, long-horizon, multi-agent) - and it does not, supplying design conviction plus three customer testimonials with no baseline, n or eval set. The evidence bar moved down, not up: S6's figures at least came from the publisher's own chart specs; S7's are marketing quotes. Recorded as still-open rather than letting "97% fewer first-pass errors" stand in for a result. Best new mechanism: the live demo is stronger evidence than any slide - a running console supplying content_sha256 write preconditions and version/attribution rows that the narration never states. Best finding the speaker passes over: agents write instructions to their successors ("Next agent: skip dep checks") and successors comply, making a shared store a coordination channel rather than a knowledge base - which turns memory.md's theoretical injection-sink note into a concrete propagation path with forensics (attribution) but no admission control (claim 63, labelled commentary since the source never discusses it). Also named the category skills.md lacked: a skill is procedural memory (claim 64). Process note: this folder is a clean re-capture after a concurrent session moved the original mid-ingest; the transcript reproduces byte-for-byte and the superseded capture is parked in git-ignored staging/. |
| 2026-07-31 | brain (kit) | Added a dream stage - the global reconciliation pass over brain/ itself (ADR-0009), plus a concurrency convention in AGENTS.md. Both come from the kit turning its own sources on itself. S6 and S7 independently found that memory written during work is locally optimal and globally suboptimal - each write defensible, the aggregate duplicating and drifting, because the writer was busy doing something else (claim 59). That is exactly how brain/ is written: ingest merges into the topic note in front of it and never asks whether the claim contradicts one promoted three sources ago. validate.py does not catch it - it checks form, this is drift, and the line between them is ADR-0004's. The gap already produced a defect: memory.md shipped saying "this topic has no measurements at all", which the chart recovery falsified hours later; it survived a full ingest and needed a dedicated fix: commit. Karpathy's LLM Wiki gist (reviewed, not yet ingested) sharpened the framing - it names ingest / query / lint and the kit has all three; what it lacks is the maintenance pass both vendors argue is load-bearing. Design follows the sources point for point: decoupled from ingest (never a compound step - an agent finishing a source holds two objectives and trades the second away silently, same argument as claim 33's generator/evaluator split); reads the whole brain with no budget cap, since affording that is the entire point of being out of band; applies clear defects but proposes judgement calls and asks; writes a permanent note per pass to brain/dreams/. Eight classes: contradiction, duplication, stale confidence, stale status, orphans, closed open questions, superseded framing, drift from source. "Nothing found" is defined as a valid result, to remove the incentive to manufacture findings. Rejected: extending validate.py (would launder judgement as a green check - form is code, judgement is prose); running it at the end of every compound (recreates the objective conflict it exists to remove); scheduling it (this brain gains a source every few days; nightly passes would mostly print "nothing found" and train the reader to ignore them - revisit if ingest rate rises). Also recorded the concurrency convention this session learned the hard way: two agent sessions wrote the brain concurrently, one renaming the other's source folder mid-ingest. Git already supplies claim 61's machinery (commits are attribution, history is versioning, conflicts are arbitration) so nothing was built - only the convention that source-local files are always safe, shared files must be re-read from disk before compounding, and claim / S<n> / ADR numbers come from disk and never from context. Not yet run - the first pass is brain/dreams/0001-*.md and should be expected to find real drift across seven uncompounded-against-each-other sources. |
| 2026-07-31 | 260731_llm-wiki | Ingested Andrej Karpathy's LLM Wiki gist (2026-04-04, revision ac46de1) as S8. Gated on the source's own terms before topics/memory.md was opened, by request, so that agreement with the brain's memory synthesis would be observed rather than reverse-engineered - the disclosure of what was unavoidably in context at gate time is in nodes.md. 18 nodes, all single-leg, all needs-check, and that is a property of the artifact rather than a degrade decision: ~1,960 words of prose with no figures, no code, no data, no implementation, so Visual leg: n/a and the source cannot produce an internally corroborated claim. One divergence found anyway (d1), which is notable for a one-legged source because it is internal to the document: §Why this works claims LLMs "don't forget to update a cross-reference" while §Lint instructs you to hunt for missing ones. Believe §Lint - the six-item list is an admission that integrate-on-ingest leaves defects behind, and the honest form of the economic claim is cheap enough to be worth doing repeatedly, not so reliable that doing it once is enough. The finding of the pass: ADR-0009 had this source backwards. It cited the gist as "reviewed, not yet ingested" and said the kit lacked the out-of-band maintenance pass that S6 and S7 argue for. §Lint is that pass - periodic, separately invoked, and four of its six defect classes are verbatim members of the dream stage's eight (contradictions, superseded claims, orphans, missing cross-references); a fifth is the architect's create-vs-merge call and the sixth is the deep-research trigger. The error was matching the word "lint" to validate.py and stopping - the two share nothing but the name, since validate.py checks form and every item on Karpathy's list is judgement. Corrected in ADR-0010; ADR-0009's reasoning left in place with a forward pointer, because an ADR records what was decided at the time. What the third source is worth, stated narrowly: it is dated seven weeks before S7 and two months before S6 and sells nothing, so it removes "the vendors copied each other" as the explanation for their convergence, and it generalises the pattern from agent memory to any maintained knowledge store. It corroborates the practice of claim 59 and not the rationale - S8 never says why periodic beats at-ingest. And three unmeasured sources are not more measured than two. Topic work: rag goes seed -> emerging on its first source in the brain's history, with its scope widened to cover maintained knowledge layers as the alternative to query-time retrieval - the architect call was to merge rather than spawn a knowledge-bases note, which would have put two halves of one argument in two places. The rag/memory boundary from ADR-0007 needed a qualifier and got one in both notes: "who authored the corpus" does not separate them (an LLM Wiki is system-authored about someone else's documents), so the refined line is what the knowledge is about - external sources vs the system's own experience. Claims 65-74, 4 glossary terms. Incidental defect fixed: context-engineering.md named S4 in its status line and cited it in two claims but had no row in "Sources feeding this topic" - an orphan of exactly the class ADR-0009 exists to catch, found while editing the note for an unrelated reason. The lesson worth keeping is about where drift comes from: this defect was a claim made about a source that had not been ingested, wrong the day it was written, in a file nothing re-reads - and a dream pass could not have caught it, because reading all of brain/ would show a perfectly coherent story with the contradicting evidence sitting outside. Citing a source you have only skimmed creates a claim with no gate behind it. |
| 2026-07-31 | brain (kit) | The brain got a reader (ADR-0011): tools/build_site.py renders INDEX.md + brain/ + every LEARNING.md into a phone-first, offline-capable site, published to GitHub Pages on every push by .github/workflows/pages.yml. A third frozen script that renders and never authors - it adds no claims, drops no citations, and site/ is git-ignored and reproducible from scratch. Nothing needed re-authoring: the lessons were already there as each source's ## TL;DR + ## Key claims and each topic's **Status:** + ## What this covers. Four presentation transforms earn their keep - &t=NNNs citations become tappable YouTube deep links, claims.md's 74-row five-column table becomes filterable cards, the landing page is the lessons, and each note's agent-directed preamble is collapsed rather than stripped because some preambles carry real trust caveats (the skills note bounds how far to trust its own numbers) and deleting those would be authoring by omission. Verified at 390px: no body overflow, mermaid fits, zero broken internal links. validate.py now gates the deploy, and BUILD.md ships the reader so the open-source bundle matches the contract it carries. |
| 2026-08-01 | 260801_agent-framework-layered-sdk | Ingested Inside the Microsoft Agent Framework: How we designed a layered SDK (Shawn Henry, 2026-05-28) as S9. Third blog ingest, and the first source whose figures are richer than its prose - worth recording as a class alongside S6's product screenshots and S4's tables. Three of four authored architecture diagrams carry named content the text never states: the model/tool/hosting inventory (n3), the harness columns naming skills, todo and tool selection plus preset harnesses per task archetype (n7), and the non-stacked arrangement of the three layers (n9). That inverts the kit's usual leg assignment - normally the text is primary and the visual confirms; here the gate ran the other way, and n4 is single-leg on a diagram tile with the prose as the weaker leg. 9 nodes (6 internally corroborated, 1 single-leg, 1 split, 1 implicated in the divergence), 2 drops, claims 75-81, 6 glossary terms, Visual leg: analysed (4 figures kept) - nothing pruned, because a four-figure blog post has nothing to pre-filter. What the source is actually worth, stated narrowly: a vocabulary, not a finding. Loop / workflows / harness as three separable purchases rather than a three-tier stack (the summary figure draws the two surrounds as peers that do not touch, so each is separately declinable - claim 17 one level up); five named orchestration patterns; and the only enumerated harness inventory in this brain. The brain's first flat cross-source contradiction (d1): S2 says own all four parts of the loop and blames the 70-80% wall on a framework owning one; S9 says the loop is fiddly plumbing the SDK should absorb. Kept and flagged, not resolved - both unmeasured, both authors sell a framework, and they may be answering different questions (where the debuggable seam sits at 80% vs how much plumbing you write to reach it). What would settle it is what happens at the 80% wall with this SDK, which the article never discusses. Two placements matter more than the inventory: skills filed under Context beside prompts and memory by a vendor with no stake in S7's argument and no reference to it - independent weak evidence for claim 64's family assignment, which is the scarce kind - and todo named a Planning primitive, making the todo list architectural rather than a prompting trick. The absence worth naming: claim 31 has no counterpart here. S9 lists what a harness may contain; S4 built one and deleted half of it. A catalog invites you to take the whole shelf and an SDK vendor has a structural reason never to suggest subtraction - recorded as an open question, since the two sources have opposite incentives to raise it. Two deliberate restraints. mcp stays seed despite MCP appearing as one of exactly two tool-integration standards: that is evidence about MCP's position and none about its mechanics, so the sighting is recorded in that note's open questions and not counted as its first source - one box in one diagram does not populate a topic. And skills stays emerging for the same reason. agents reaches 6 sources, context-engineering 4. Fixed in passing: agents.md's status line read "3 sources" while its own Sources section listed five - stale since S5 and S7 landed, exactly the class the dream pass exists to catch, found because this ingest had to edit the line anyway. The standing caveat: T2 vendor design post about its own SDK - no benchmark, no baseline, no comparison to another framework, no report of what building with it is like. Every claim is emerging at best, and where any meets a measured claim (24, 32) the measurement wins. |
| 2026-08-01 | brain (kit) | Closed the loop on S9 with ADR-0012: a mention is not a source. The ingest forced the same taxonomy call twice in one pass and the kit had no written rule for it. S9 names MCP as one of exactly two tool-integration standards and files Skills under Context beside prompts and memory - on a literal reading of the status ladder that takes mcp from seed (zero sources since the brain started) to emerging, and arguably skills toward established. Both readings inflate the brain. What S9 supplies in each case is one box in one diagram: it never defines MCP, never describes a server, tool, resource or transport, never defines a skill, never discusses writing or evaluating one, and mentions neither in its prose at all. mcp.md would have gone emerging with a literally empty Synthesis section. The rule: a source advances a topic's status only when it teaches something within that topic's scope; a mention, citation or labelled box is a sighting, recorded in the note's Open questions without advancing status or appearing under "Sources feeding this topic". The test: would a reader arriving from the INDEX find what the status promises? Scope is the note's own "What this covers", which makes it checkable rather than taste. Decided by asymmetry, the same argument ADR-0006 makes about the probe: under-counting is self-correcting (the sighting sits there cited, waiting to be promoted with the next real source), while over-counting is invisible - a populated status row over an empty note reads as "not written up yet", not "never had a source" - and it corrupts what the ladder measures, since established would come to mean "this string appeared three times". Rejected: counting any citing source and letting confidence carry the weakness (conflates the two axes, in the opposite direction from ADR-0008 - there the area genuinely recurred, here only the word does); a numeric threshold (ADR-0004's line - form is code, judgement is prose; a count invites gaming and breaks on a source that teaches a lot in two sentences); and advancing mcp and writing its Synthesis from general knowledge, which is the tempting one and would produce an uncited topic note - exactly the failure S8's ingest caught, a claim written about a source never ingested, wrong the day it was written, in a file nothing re-reads. Also fixed on review: S9's SOURCE.md named evals in its Topics row while nothing was promoted to evals.md - the same defect class the ADR describes, caught by cross-checking the row against which notes actually cite S9. Dropped evals rather than padding that note to justify the metadata. New watch item for the dream pass: sightings accumulate in Open questions and nothing sweeps them - three sightings and no source is itself evidence an area is real and under-covered. Recorded as a watch item, not a rule, since it has not happened yet. |
| 2026-08-01 | 260801_tool-search-toolboxes | Ingested Tool search: Finding the right tool at the right time (Lisa Brown Jaloza, Microsoft, 2026-07-29) as S10. The brain's first real MCP source, and it is an article about tool retrieval - MCP is the substrate it explains on the way past. Promoted claims 82-90 across mcp, rag, context-engineering and agents. Visual leg analysed, 3 of 3 figures kept and all three earned it: Figure 1 corroborated the architecture prose, Figure 2 carried the headline measurement and said more than the prose (the tool-search curve is roughly flat in catalog size, not merely "savings scale"), and the portal screenshot - shipped with an empty alt and nearly skipped - turned out to carry the load-bearing MCP mechanic the prose never states: the toolbox is itself an MCP server fronting other MCP servers (n7, figure-only). The source also has a code snippet, so the docs-vs-code check the contract writes for repositories applied to a blog post and caught d1: four different names for one class inside one section, in a snippet that cannot run as printed. Two further findings recorded rather than resolved - d2, the Recall@10 head-to-head borrows its baselines from another paper while the challenger column is a self-run under an admittedly different protocol; and d3, auto-pinning (per-user, aging) contradicting "deterministic pinning keeps the prompt prefix stable" one sentence later. The gap the source leaves is the interesting one: Recall@10 of 39-46% against a default shortlist of 5, never placed beside its own closing line "the shortlist has to be good" - promoted as claim 87 and labelled as this brain's reading, not the source's claim. Closed context-engineering's standing caveat that nothing measured tool selection (S9's figure-only box), and partially closed rag's "zero sources on retrieval mechanics" - precisely, since chunking, embeddings, vector stores and grounding evals remain at zero and S10 retrieves over tool schemas rather than prose. rag therefore stays emerging on two sources that barely corroborate each other. Independence recorded: S9 and S10 are both Microsoft and are not two sources where they overlap. |
| 2026-08-01 | brain (kit) | ADR-0013: secondary but substantial - a source advances a topic it is not about. ADR-0012 (one source earlier) closed by naming the case it had not produced: a source "substantial on a topic but entirely secondary to its main subject". S10 produced it immediately. mcp advances seed to emerging on a tool-retrieval article, because five cited, corroborated claims about MCP mechanics fill a Synthesis section that was literally empty since the brain started. The rule: being secondary is not disqualifying, being thin is - and ADR-0012's guard is unchanged and now does all the work, since the Synthesis section must be written from gated nodes before the status moves. S9's earlier mention stays a sighting and is not retroactively promoted. New drift watch item for the dream pass, mirroring ADR-0012's: a topic with no primary source is scoped by accident, and mcp.md is now in that state by construction and says so. |
| 2026-08-02 | 260802_agent-data-stack | Ingested LangChain's How we built LangChain's agent-first data stack (Emily Hawkins, 2026-07-27) as S11, with a deep-research pass (R2) requested at ingest. Run curator + fact-checker, architect for the topic call. Gated 12 nodes (7 corroborated, 5 single-leg) and 4 divergences, two of which exist only because of the visual leg - 23 of the page's 26 images were chrome, one of the three body figures was a decorative title card (viewed, then pruned), and the remaining two carried claims the prose does not make. d1 is the pass's best find: the figure captioned "LangChain's data stack architecture" is a stock ELT pipeline with no agent, no semantic model, no trust signals and no feedback loop in it, while the prose calls the migration "a big architectural shift" - so what changed was the reporting tier and the English written around the warehouse, not the architecture. d2: the prose's feedback loop has two human review steps and the architecture figure has no human anywhere, which matters because the endorsement write-guard is the one control making the design safe. 10 claims promoted (91-100), plus claim 72 raised needs-check -> emerging on a second historical precedent. 5 terms to glossary.md. Claim 93 is the pass's headline: metadata written for humans becomes a control surface the moment an agent reads it - third independent instance (S5 a skill's description is its trigger, S10 a tool's description is a ranking feature, S11 a column's description is a default policy) - and claims 93 and 95 are the first in this brain to reach corroborated on independent external evidence rather than two internal legs. R2 (4 searches, 4 fetches of <= 8/<= 12, stopped early on agreement) found the source's mechanism measured by parties with no stake in it: MotherDuck's query-log-informed descriptions (T2) bought +2.0pp on BIRD-Dev and +16pp on a real warehouse - a benchmark-vs-production gap promoted as claim 94 - arXiv:2408.04691 (T3) measured +20% on uninformative column names, Power BI endorsement (T1) is feature-for-feature prior art for n6 with two tiers where S11 has one, and Feigenbaum's knowledge acquisition bottleneck (1977) is a second precedent for claim 72 alongside Memex (1945), refined by claim 98: the LLM collapsed the encoding cost of expert knowledge and left the elicitation cost untouched. Nothing was contradicted, and nothing was corroborated about results: every figure the source reports measures adoption while its thesis is trustworthiness (d4), the 40x compares agent conversations against a team's estimated capacity to field requests (d3), and Spider 2.0-DBT - the public setting closest to this stack - tops out at 65.6%. no-evidence on the question that matters most: nobody has measured whether a trust flag changes an agent's source selection, only a human's - now rag.md's cheapest high-value open experiment. ADR-0014: no new topic for organisational context - a real create-vs-merge call, since S11 is the first source about an agent reading a company's proprietary non-code knowledge and it arrived with its own vocabulary (semantic layers, endorsements, knowledge engineering). Declined because nothing in S11 is specific to data: swap "table" for "API endpoint" and every claim holds, so a new note would have collected the disposable vocabulary while the transferable claims stayed elsewhere. context-engineering 5 -> 6 sources, rag 2 -> 3 (stays emerging - the three sources still do not corroborate each other's machinery, and S11 is silent on S8's self-maintaining premise because every write in its loop is a human's), evals 3 -> 4 (S11 enters as a counter-example - five instruction artifacts shipped company-wide with no evals, conceded by the authors). Stale status fixed: skills.md read "1 source" while three were listed and INDEX.md said three, stale since S7; corrected to 4 with an explicit note that the count rose 1 -> 4 while the evidence did not move at all, since S7, S9 and S11 each contribute one peripheral observation and none studies skills. S11's MCP appearances (two bullets in a list of access surfaces) dropped under ADR-0012 - mcp stays at 1 source. |
| 2026-08-02 | brain (kit) | First dreaming pass (dream 0001) - run 11 sources and 100 claims after ADR-0009 created the stage, so it reconciles six ingests of in-band compounding that never had a pass behind them. Six findings applied, and five are one structural bug: a fact about note A written inside note B, gone stale when A changed. agent-security.md said 1 source with two listed; agents.md called agent-security "still a seed" and skills.md said the same of mcp; rag.md measured this brain at 8 sources; INDEX.md said "the six below" above nine rows. The sixth is the one that justifies the pass: memory.md cited claim 33 for generator/evaluator separation when 33 is ablation and the claim is 34 - the same mis-citation fixed in AGENTS.md hours earlier, and precisely the defect the new validate.py check cannot catch, because 33 exists and is merely wrong. Zero belief drift: no contradiction, no confidence moved, no topic split, and every conflict the brain holds was already kept-both-sides and labelled. Two checks proposed, not applied (status line vs INDEX row; cross-note status claims). |
| 2026-08-02 | 260802_gcp-multi-tenant-agentic-ai | Ingested Google Cloud's Multi-tenant agentic AI system reference architecture (Awasthi + Bhardwaj, reviewed 2026-06-18) as S12. Run curator + mentor, fact-checker at the gate, architect on two topic calls. The brain's first source about deploying agents across an organisation rather than building one, and the first from the reference architecture genre - which is worth naming as an evidence class, because it documents a shape and reports no result: 5,000 words, 21 contributors beyond the two authors, a Terraform implementation, and no latency figure, cost figure, incident, named deployment, or comparison against any alternative. Gated 18 nodes (9 corroborated, 9 single-leg), 3 divergences and 3 recorded gaps. Claims 101-109, 8 glossary terms. Visual leg: analysed (1 figure kept) - the page has exactly one content image and everything else is site chrome, so there was nothing to pre-filter; it was rendered at full size with headless Chrome after qlmanage squashed a 1188x1370 viewBox into a square thumbnail and silently cropped half the diagram away, and then cropped into two teaching views. The gate is structurally weaker here than the verdicts look, and nodes.md says so up front: both legs are the same author team on the same page, so corroborated means the document is internally consistent and nothing more - unlike a figure that is a benchmark chart (S10) or a console recording (S7). The organising finding (n14, claim 106): the four "design alternatives" in the second half are not four decisions, they are one trade made four times - shared model endpoint, shared MCP server, one Model Armor instead of two, private ingress - and the trade is not cost-for-isolation but a change in what kind of thing the guarantee is, from a property of the topology into a claim about an implementation. The figure corroborates this by omission: it draws the maximum-isolation branch of every single fork. The asymmetry is the teaching point - the per-tenant cost is countable and lands in a budget, the shared cost is a class of defect that surfaces in someone else's incident, which is why the cheap branch wins arguments it should lose. Best divergence (d2), and both sides are in one paragraph: the security section deploys Model Armor at two layers for "a zero-trust posture", the cost section then recommends deploying it "exclusively in the routing hub" - deleting the tenant-local leg and keeping the shared one, so every tenant's PII masking moves into shared infrastructure, while the same sentence concedes the two-layer design is what "helps to ensure data sovereignty". The missing artifact is the condition under which a tenant may take the cheap option, and it is one sentence long. d3 is the same species: "even if an agent identity is compromised, the agent can't access unauthorized Google Cloud resources" is stated in the Use case and holds only for the drawn topology, with its qualifications three sections away and uncross-referenced - the characteristic failure of this genre, and the reason to read a reference architecture alternatives-first. d1: the org-level VPC Service Controls perimeter the prose calls the strict security boundary is drawn nowhere in the figure. Three absences recorded and deliberately not promoted (g1-g3): the word "eval" does not appear anywhere in 5,000 words about running enterprise agents (observability is entirely infrastructural - this architecture tells you the agent is up, never that it is wrong); there is no cross-tenant path and no discussion of wanting one, so this is n independent agents behind one door rather than a multi-agent system; and there is no memory tier at all, session context appearing only as a cost to summarise away. S12 is not counted as an evals source - an absence is not a source (ADR-0012), and reading silence as a claim is exactly that ADR's laundering failure. ADR-0015 settles two calls with one test: a source advances a topic when it teaches within the scope, not when it depends on it. S12 does not trigger agent-security.md's standing identity split despite being drenched in IAM, IAP, principal boundaries and identity propagation, because it teaches no identity mechanics whatever - no protocol, token, flow or lifetime - and at the one point where mechanics are needed it states the requirement and stops. A source that needs a subject and cannot supply it is evidence the subject matters, not a source on it. No multi-tenancy topic either, on ADR-0014's swap test: replace the agent with a microservice and the isolation machinery survives intact, so the note would have collected the generic vocabulary while the four agent-specific claims scattered elsewhere. mcp advances 1 -> 2 sources and stays emerging, with the reason written into its status line: S10 and S12 corroborate nothing of each other's, so established is no nearer - counting sources is not measuring agreement. S12 contributes the deployment side the note entirely lacked (MCP as the mandatory data seam; local-vs-shared placement as an isolation decision) and partially answers its standing open question about what an aggregating server owes the servers behind it, supplying the auth-propagation requirement and no more of an answer than S10 did. Two independent sources have now stopped at the same missing piece from opposite directions, which promoted it to a question of its own in both mcp.md and agent-security.md - the cheapest high-value research target in the brain, and the concrete landing point for the owner's stated identity track. agents 7 -> 8, context-engineering 6 -> 7, agent-security 2 -> 3. Four of the nine claims are this brain's synthesis of statements the source makes separately (101, 106, 107, and the bound on 103) - and they are the four most transferable, labelled as such in every cell of claims.md and inline in LEARNING.md, because a reader lifting them into a design document should know they are not the vendor's assertions. |
| 2026-08-03 | brain (kit) | ADR-0016: /verify reads the frames it checks. The stage's first run anywhere found a defect in itself - check 6 asks whether a kept frame's what it teaches matches what the frame shows, while the scope rule capped the read list at three text files and no images, so the stage could not run one of its own six checks. That is the S7 d4 failure (a verification step with no stated mechanism) reproduced by the stage built out of it. Decision: check 6 runs by default, the read list gains the source's visuals/, and a skip is recorded rather than discovered in a Frames field on every pass entry - the same shape as SOURCE.md's Visual leg row, reusing a pattern the kit already has instead of inventing a flag. The cost objection was measured and did not survive: 81 kept frames across 12 sources, median 4, maximum 15, and zero where the visual leg was skipped. Two alternatives rejected on the record - opt-in (a default-off check never runs, because nobody asks about frames they have not seen) and drop it (the caption's author is the ingesting curator, and after ingest nobody looks at that image and that sentence together again, so the visual half would have no independent reader at all - claim 34). validate.py proves a frame is cited; only a reader with the image open can tell whether the citation is true. |
| 2026-08-03 | brain (kit) | foundations/ exercised for the first time, and a reading list created because the contract promised one and no file existed. Six personally commissioned research modules (2026-07-11, agent-generated, roughly T5) arrived in staging/ in two versions each. The _v2 set was rejected wholesale: architecture decision documents written for a specific commercial platform, which is "making claims about the world" under the contract's triage table, so the primaries they cite were taken and the documents were not. Every v1 module was split, since all six were "Both": the definitional half became a foundation, the prescriptive half (17 numbered best practices, 7 failure modes, all quantified results) was dropped rather than filed, because a quantified claim inside an uncited file is precisely what this layer exists to prevent, and several would have shadowed claims the brain already holds gated. Exercises dropped - this kit learns from sources, it is not a course. Four foundations filed, not six, on the test does the brain genuinely lack this background: tool-use-and-mcp, memory-taxonomy-and-lifecycle, grounding-and-retrieval, agent-threat-model. Evals rejected (evals.md is established on four sources and the module's benchmarks are things to ingest); knowledge-representation rejected (half its bibliography is product documentation, and knowledge graphs are off this brain's path). The real yield was the gap analysis, not the background: agents.md is established across eight sources and has no primary behind the agent loop - ReAct has never been ingested; rag.md has no foundational retrieval paper at all; agent-security.md has one threat source. Twelve terms promoted to glossary.md under a new F attribution that declares them background rather than findings. Specifications are deliberately not on the reading list - MCP, W3C, NIST, the EU AI Act are reference consulted on demand, and ingesting one would produce a worse copy of something authoritative and permanently online. |
| 2026-08-03 | brain (kit) | validate.py was linting the git-ignored inbox. markdown_files() skipped raw/ and repo/ but not staging/ or site/, so six staged modules produced 1,485 style errors, not one of them about a kit file. That inverts the contract's own rule that material becomes part of the kit when it is filed, not when it is copied in - the build went red for the crime of having received something. Both directories added to the skip set. Found by exercising foundations/ for the first time, which is the argument for exercising a stage rather than shipping it. |
| 2026-08-03 | 260803_autoresearch | Ingested karpathy/autoresearch (Andrej Karpathy, code, snapshot 228791f, repo created 2026-03-06) as S13 - the brain's first code source, and its first source about an agent that runs experiments unattended rather than one that answers or builds. Run code-explorer + mentor, fact-checker at the gate, architect on the topic call. Gated 18 nodes (13 corroborated, 5 single-leg), 2 divergences, 3 recorded gaps. Claims 110-119, 6 glossary terms. Owner scoped the ingest at the start: the research harness, not the model - the transformer, MuonAdamW and the FA3 kernel path in train.py are recorded as deliberately-not-traced, since train.py is the artifact under optimization rather than the subject. The clone was declined mid-ingest, so the 10-file repo was fetched file-by-file at the pinned SHA into git-ignored repo/; consequence recorded rather than discovered - no git history leg was available, and no code was executed (it needs an NVIDIA GPU; the owner has none), so nothing here is a reproduction. gh was present but unauthenticated, so metadata came from the public REST API and the license had to be read out of the README by hand - MIT is declared in prose with no LICENSE file and a null license from the API, which is recorded in SOURCE.md rather than smoothed over. The organising finding (n1-n5, claims 110-113): the repository is a permissions diagram, and every file's real identity is the answer to "who may change this?" Four things are frozen before the agent starts - the editable surface (one file), the resource budget (wall-clock seconds, not steps and not tokens), the metric's units (bits per byte, evaluated at a fixed sequence length), and the holdout (a pinned shard excluded inside the read-only file) - after which the loop is nine steps of shell commands and needs no framework. The repo contains no agent code at all. The derivation is the transferable part and is this brain's: fixing steps rewards shrinking the model, fixing tokens makes efficiency invisible, fixing time puts a faster kernel, a better optimizer and a longer schedule on one axis, so efficiency becomes part of the objective without being part of the metric. Generalised as claim 111: anti-Goodhart is a code-layout problem, not a prompt problem - none of the three defences is an instruction to the agent, all three are properties of a module it is told not to open. Two weaknesses, and both sit on the decision path rather than the computation path, which is the pattern worth carrying. n5/claim 113: the frozen scorer's result is imported, formatted and printed by the file the agent rewrites, and the agent reads its score by grepping that print, with nothing comparing the two - claim 34 arriving as plumbing rather than prompting, which is the sharper form, since generator and evaluator can be perfectly separated at the function level while the evaluator's output still travels through the generator's hands. n11/claim 114 is the pass's headline and the best find in the source's own evidence: the accept rule is a bare scalar comparison with no repetition, seed averaging, threshold or error bar - and the fifteenth and final kept improvement in the author's own published 83-experiment run is a change of random seed, sitting in the repo's teaser image. Not a failure of the agent's judgement; the rule executing correctly on an input it cannot recognise. That experiment also measures the loop's noise floor for free, and by that bound at least three other accepted changes are unresolved (n12, claim 115) - gated needs-check and deliberately not promoted harder, because the deltas are read off a rendered PNG by eye at a scale where the quantity of interest is about a pixel, and the floor rests on n=1. Compounding consequence recorded as g3: every accept permanently moves the baseline and nothing re-tests a kept change, so a lucky accept raises the bar for every subsequent real one. n7/claim 116 is the sharpest small idea: the results ledger is deliberately untracked, because discard is git reset and a committed ledger would lose the row describing the experiment that just failed - generalised as in any loop whose failure mode is rollback, the audit trail must not be rollback-able. The source gives the instruction and no reason; the derivation is this brain's and is labelled. Price paid in the same breath and recorded twice: the author's published results cannot be reproduced from the repository, which is why the whole empirical half of this source is gated weakly. Visual leg unusual for a code source and analysed anyway - the contract expects generated diagrams, and this repo ships a real 2382x1180 results chart which is its only evidence; viewed at full size and cropped into 2 teaching views, 3 frames kept and all 3 cited in LEARNING.md. Reading analysis.ipynb against the chart caught n15: the figure is filtered by the notebook that draws it (crashes dropped, only experiments at or below baseline + 0.0005 plotted) while its title counts all 83, so the visible near-miss cloud is not the failure population. Second divergence d2: evaluate_bpb sizes its eval set by integer division on an agent-editable batch size, so the exam moves 0.6% against a docstring promising comparability across configs - an order of magnitude below the noise floor and recorded for the principle, not the magnitude. d1: the crash sentinel 0.000000 sorts as best under a lower-is-better metric; the shipped notebook filters on status so nothing in the repo is wrong, but the schema is. ADR-0017 creates autonomous-research-loops (emerging, 1 source) - the fourth create-vs-merge call in four ingests and the first to say yes, after ADR-0013/0014/0015 all declined. Two reasons, both recorded: the swap test cuts both ways here (swap the domain and every claim survives, but swap the agent for a random-search script and the context budget, the suppressed check-in, the published exchange rates and the reporting-path leak all die), and the area is single-primary, not single-source - claims 7, 10, 31, 34 and 59 already bear on it and currently sit scattered across agents and evals as members of other arguments. They are cross-referenced, not re-homed, and their Topic column is unchanged. Deliberately not named research-agents: this kit's /research stage means literature search, and a note by that name beside .claude/commands/research.md would read as the opposite of its contents. Merge-back trigger written into the note rather than left to be re-derived. Claims distributed across four notes rather than hoarded in the new one: 110/112/116/117 to autonomous-research-loops, 111/113/114/115 to evals (they are measurement claims reusable well outside a research loop), 118 to context-engineering, 119 to agents. evals 4 -> 5 and gains the half it was missing - every prior source there takes the metric as given and asks how to use it; S13 is the first to ask how to design one an optimizer cannot game, and pairs with S5 as the closest thing that note has to a corroborated position on variance, arrived at from opposite ends (S5 prescribes up to six trials per case; S13 demonstrates what omitting that costs when nobody is watching until morning). agents 8 -> 9 on claim 119 alone - autonomy as a deliberately suppressed default, which points opposite to claim 16 (contacting a human is a tool call); kept as a tension rather than resolved, with the two reconciling conditions (the check-in has no information to offer; the blast radius is bounded) labelled as this brain's reading and flagged for the next source on unattended loops. context-engineering 7 -> 8 on claim 118, the first multiplier this note has had to reason about - and its gap noted in the same breath: S13 solves the per-iteration input cost and ignores cross-iteration memory entirely. skills 4 -> 5 under ADR-0012, mention-only: the author calls program.md "essentially a super lightweight 'skill'", and the artifact stretches this brain's definition in a direction nothing else has - not a capability invoked when relevant, but an entire operating procedure loaded once by a human - which makes S5's cost-ladder and description-as-trigger analysis inapplicable to it. A second live counter-example on evals beside S11, and the trap in that note's status line now has a fifth data point: the count rose 1 -> 5 while the evidence did not move at all. No deep-research pass was requested or run, so S13 has no external corroboration whatsoever - every citation in this ingest points inside one repository, and the note says so in three places. |
| 2026-08-04 | 260804_cs329a-self-improving-agents | Ingested Stanford CS329A lecture 1 as S14 - the brain's first academic source, and the source that gave self-improvement its own note. Human-authored captions (no Whisper), and the richest visual leg yet measured: probe returned candidates=154 distinct=44 against a STATIC threshold of 3 and S1's previous high of 19. 32 transcript-anchored candidates triaged through four contact sheets (4 view calls, not 32); 8 frames kept, two of them cropped out of room-camera cuts because the recording alternates angles and the clean cut was not available for those slides. Gated 14 nodes (7 corroborated, 7 single-leg) plus 2 divergences. The payload is 40 minutes out of 70 - the first 17 are field 101 for this reader and were gated but deliberately not promoted under the never-101 calibration, and the last 11 are course logistics. Created self-improvement (emerging, ADR-0018) on the finding that self-improvement is loop closure rather than a technique: one arrow from test-time back into fine-tuning, and every turn of the loop passes through a verifier. Its headline claim (124) is that verification, not generation, sets the ceiling, which is corroborated on the pattern while the causal join is this brain's - the lecture states the domain gradient and the bottleneck in separate places and never connects them. The sharpest thing the compound produced belongs to neither source alone: claim 124 joined to claim 114 gives having a verifier is not the same as having one that works, since S13's loop had a real, cheap, automatic verifier and still banked a random-seed change. That also partly closed a standing open question in autonomous-research-loops (where does this shape stop working), which is why the two notes are cross-referenced rather than merged - ADR-0018 records the merge-back trigger. Three ADR-0018 calls worth remembering: evals took claims 125 and 126 rather than the new note hoarding them, agents took 127, and inferencing was deliberately left at zero sources because test-time scaling buys accuracy while that note's scope is serving - filing it there would have made an empty seed note look populated. The gate's most useful catch is against the source's own best slide (d1): it is titled "Models Improve Drastically with Just Repeated Sampling!" and plots coverage, not accuracy, with an oracle verifier on half its panels - the narration is more honest than the slide, which is the general shape worth carrying, since the overstatement lives in the artifact that gets shared and the correction lives in speech that does not. A glossary collision surfaced and was recorded rather than overwritten: S1 and S14 both say pass@k and mean different things (sequential retries with feedback versus independent samples without). evals 5 -> 6 and now holds two independent assertions of self-preference and zero measurements (claims 34 and 125); agents 9 -> 10. 8 rows to claims.md (120-127), 7 terms to glossary.md, 2 primaries to reading-list.md (Large Language Monkeys, The AI Scientist - the latter being the obvious second primary for autonomous-research-loops). No deep-research pass was requested, so S14 has no external corroboration at all, which matters more than usual here because its two strongest slides are non-independent in two different ways. |
| 2026-08-04 | 260804_cs329a-test-time-compute | Ingested Stanford CS329A lecture 2, Test-Time Compute Scaling, as S15 - tracked separately from S14 at the owner's request, which is also exactly what self-improvement.md had already required in writing. Run curator + mentor, fact-checker at the gate, architect on the routing calls. Human-authored captions again; probe returned candidates=83 distinct=33 -> RICH, 35 transcript-anchored candidates triaged through four contact sheets (4 view calls, not 35), 9 frames kept, all from the clean slide cut so no cropping was needed. Gated 31 nodes (21 corroborated, 10 single-leg) and 4 divergences. Claims 128-134, 6 glossary terms. The playlist warning worked, same day it was written. S14's note predicted that further CS329A lectures would raise the raw source count without adding evidence, and told a future agent to record them as S15+ and keep the counts separate. That is precisely what happened, with no re-derivation needed at ingest time: self-improvement 1 -> 2 raw and 1 -> 1 independent, status unchanged, and none of claims 120-124 was re-confidenced even where lecture 2 repeats lecture 1 nearly verbatim. A warning written before the event is the only kind that gets obeyed; this is the first time this brain has demonstrated that on itself. Capture nearly degraded silently and was caught by a tool rather than by eye: the 720p AV1 stream 403'd and the fallback format 18 downloaded cleanly at 640x360, which is fine as video and useless for slides carrying eight-panel charts and per-fit error bars. ffprobe caught it; a second pull of the video-only 720p H.264 stream fixed it, since audio was never needed. A citation convention was needed and is recorded rather than fudged: many nodes cite a slide viewed in triage and deliberately not kept, citing it by title plus timestamp rather than file path - the gate is about what the evidence is, and the prune rule governs what is stored. Inflating visuals/ to twenty frames so every citation resolved locally would have honoured the letter of the citation rule by breaking the rule that actually matters. 9 frames slightly exceeds the 3-8 guidance, deliberately, because the ninth is the second leg for n9 and dropping it would have traded a hard rule for a soft one. The pass's headline finding belongs to the gate, not to the lecture (claim 132): the source overstates its own headline three times in one direction (d1, d2, d3 - a slide titled "outperforms" over an axis reading coverage, with an oracle verifier on half its panels, and SWE-bench coverage drawn against real systems' resolution rates) and then refutes itself sixteen minutes later with n10, the generation-verification gap. S14 carried the identical defect, so it is a property of how this course writes slides rather than a one-off. What turns it from a complaint into a reading rule is n31: the same lecturer reports pass@1 correctly the moment her own lab's artifact emits a single answer. The reporting follows the artifact, so this is a reporting incentive with a predictable direction. Note the inverted evidential shape and it is why the claim survives a weak source - S15 is T4 and non-independent, which makes it poor evidence about the world and perfectly good evidence about its own behaviour. d4 runs the other way (chart honest, narration lossy) and is recorded for the symmetry. The two strongest pieces of content: n5/claim 129, the only peer-reviewed result in either lecture (ICML 2025), showing a power law in average pass@k is necessarily evidence of a long tail of very hard problems - which inverts what a scaling exponent is a fact about, from the model to the benchmark, and is why it was filed to evals rather than kept; and n25/claim 133, fusion beating oracle selection, which is impossible under the selection frame and is therefore the evidence that the frame was wrong. Claim 133 is simultaneously the most reusable and the least corroborated thing here (authors' own result, one unnamed benchmark) and is now self-improvement's highest-value research target. n20/claim 131 bounds claim 123 rather than supporting it - FLOPs-matched, the gain flips sign on two variables, reaching -37.2%, so the regime you would most want test-time compute for is the one where it performs worst. Bounding is the only move a non-independent source may make. inferencing deliberately left at zero sources for the second time, against the hardest case it will ever face - a lecture literally titled "Test-Time Compute Scaling" - because ADR-0015's test is whether a source teaches within the scope (prefill/decode, KV cache, batching, serving stacks) rather than depends on it. This ingest's routing was corrected mid-pass after initially planning to file claim 131 there. evals 6 -> 7 raw (6 independent), agents 10 -> 11 raw (10 independent); both counts now split, because S15 is the source that made the distinction load-bearing across three notes at once. No deep-research pass was requested, so S15 has no external corroboration whatsoever, which matters more than usual given that three of its four papers are the presenter's own. |
| 2026-08-04 | brain (kit) | Surveyed the CS329A syllabus and promoted three of its papers to reading-list.md, rather than more of the lecture series. Run architect. The owner's goal is a self-improving agent around MCP and agent security, so the question asked was whether continuing the playlist serves it. cs329a.stanford.edu was fetched and read in full: ~34 papers across 20 lectures, of which three were already listed (ReAct, Large Language Monkeys, The AI Scientist). Promoted: Weaver (arXiv:2506.18203, lecture 3), Let's Verify Step by Step (arXiv:2305.20050, lecture 3) and the Darwin Gödel Machine (arXiv:2505.22954, lecture 7). Two findings the survey produced that a summary would have missed. First, Weaver is not independent - Mirhoseini is a co-author, so the paper answering self-improvement.md's named open question cannot advance that note's independent source count, and the row says so; this was asserted as independent in conversation before the author list was checked, which is the exact failure self-improvement.md's playlist warning exists to prevent. Second, the syllabus mislabels its own lecture-7 paper - 2505.22954 is the Darwin Gödel Machine (Zhang, Hu, Lu, Lange, Clune), not "Automated design of agentic systems" (Hu et al., 2408.08435); the DGM is the better source anyway, being the artifact layer at scale and independent of both S13 and the lecturers. The survey's own result is recorded in the Not-promoted section: across all 20 lectures and all 34 papers there is no security and no MCP material whatsoever, which confirms from the syllabus what S14 n11 only implied from a slide, and is why the Greshake and memory-poisoning rows keep their priority. Nothing was ingested and no claim moved - this pass writes pointers only. |
| 2026-08-04 | 260804_agentpoison | Ingested AgentPoison (arXiv 2407.12784) as S16 - the brain's first paper source, its first measured attack, and its first source with no commercial position in what it claims. Run curator + mentor, fact-checker at the gate, architect on the routing calls. The paper flow had never been exercised: prd.md's R10 still recorded it as unrun, so this pass built it. curl the PDF into git-ignored raw/, then two complementary reads - the Read tool rendered pages 1-10 visually, since this kit's view capability is the vision model and a paper's figures are better read in place than extracted blind, while pdftotext -layout produced a grep-able text dump so the twelve appendix pages cost no tokens. pdftotext was installed for this (brew install poppler). Seven figures and tables were cropped from pdftoppm renders at 130 DPI and viewed to confirm framing; four were re-cut after viewing showed them truncated or catching the wrong float, and the p9 perplexity figure was replaced by Figure 10 on p17, which measures the same thing across three agents rather than two. 15 nodes, 13 corroborated, 1 single-leg, 2 divergences. Promoted claims 135-141 to agent-security.md, with 137 and 139 and 141 also feeding rag.md and 138 also feeding memory.md. Claim 63 was rewritten rather than duplicated: S7 gave it a propagation path needing a cooperating agent, and S16 shows an external attacker reaching the same outcome with one record and no cooperation, so the row moves from labelled commentary to measured threat. Three routing calls worth recording. First, agent-security.md stays emerging at four sources - S16 corroborates no claim of S3's, S7's or S12's because it studies a different subject from all three, and converting a commentary claim into a measured one is a real advance in evidence that is not the two-sources-on-the-same-material test. Second, the source was declined for agents.md under ADR-0012: it teaches how an agent is attacked, not how one is built, and filing it would have moved that note 11 -> 12 sources while adding nothing a reader would open it for. Third, the tier was corrected down: reading-list.md had queued this row as "T1/T3", and the ingested artifact reads "Preprint. Under review." with no journal reference on the arXiv listing, so the note is written to T3. The AgentPoison portion of the memory-poisoning reading-list row is retired, and the two 2026 studies stay high for their defenses rather than their attacks, since S16 closed the attack half and left claims 138-141 ruling out volume detection, embedder privacy, perplexity filtering and isolate-then-aggregate. Nothing external was gathered and the companion repo was not cloned, so every citation points inside one paper. |
| 2026-08-04 | 260804_indirect-prompt-injection | Ingested Greshake et al., "Not what you've signed up for" (arXiv 2302.12173), as S17 - the paper that named indirect prompt injection, and the source that moved agent-security.md to established (ADR-0019). Run curator + mentor, fact-checker at the gate, architect on the status call. Second paper ingest, on the flow S16 established hours earlier: curl the PDF to git-ignored raw/, Read tool over pages 1-10 visually, pdftotext -layout for the discussion and mitigation sections so twenty pages of appendix prompts cost no tokens. Six figures cropped from pdftoppm renders and viewed; four re-cut, which is now the expected rate for two-column ACM layouts rather than a surprise. 16 nodes, 13 corroborated, 3 single-leg, 3 divergences. Promoted claims 142-148. The status change is the substance of this pass and it rests on one node, not on the source count. This note has carried the bar "a second source that studies the same material as an existing one" through four sources and enforced it twice, including against S16 the same day. S17's n6 meets it: the agent itself is instructed to write an injection into long-term memory and a fresh session re-poisons itself on read, which is S16's conclusion by the opposite mechanism - there an external attacker writes poisoned records a triggered query retrieves. Independence was checked rather than assumed: no author, institution or country in common, seventeen months apart, neither a vendor. That pair is claim 145 and it is the first genuinely corroborated claim the topic has ever held. A second, weaker convergence is recorded and deliberately not relied on: S17's n12 (Bing Chat filtered its chat channel and not its retrieval channel) confirms a bound this brain had written as its own commentary against S12's claim 103, from architecture alone and before it held any source that had tested it - which is a source confirming the brain's inference, a different and lesser thing than two sources confirming each other. Routing calls. S17 is filed to agents.md where S16 was declined, and the asymmetry is deliberate: claim 147 is a statement about agent capability itself - the attacker supplies the goal and the agent's own planning supplies the method - which is a property of the loop that note describes, not a security control. The established status carries a warning that must not be edited away for brevity: it describes evidential coverage of what the note asserts, and the note asserts attacks. Everything gated is an attack, there is no gated defence, ten named defences are now documented as failing, and the entire 2023-2026 defensive literature is ungated - which is now the topic's highest-value research target. Also recorded: the source is entirely qualitative (d1) with no success rate for any of six threat classes, its strongest results run against a black-box product it concedes it cannot reproduce (d2), and its mitigations survey is three years old and is its weakest material. |
| 2026-08-05 | 260804_camel-prompt-injection-defense | Ingested CaMeL, "Defeating Prompt Injections by Design" (arXiv 2503.18813), as S18 - the brain's first gated defence, closing the gap S16 and S17 opened hours earlier. Run curator + mentor, fact-checker at the gate, architect on the routing. Folder keeps the 260804 slug because capture began before midnight; the pass landed on the 5th. Third paper ingest on the established flow: Read tool over the design pages (3-10), pdftotext -layout for evaluation, overheads and discussion. The appendix is 100 of the 125 pages and was deliberately not read - system prompts, per-suite tables and generated-code transcripts are replication material. Six figures cropped and viewed; note for the next long paper: pdftoppm zero-pads to three digits past 99 pages, which broke the first crop script. One crop landed on Figure 12 rather than the intended baseline chart and Figure 12 was kept instead, because a paper demonstrating its own bypass teaches more than another bar chart. 19 nodes, 15 corroborated, 2 single-leg, 3 divergences. Promoted claims 149-156; merged into agent-security.md, agents.md and context-engineering.md. The gate finding that governs the source: every efficacy number is measured on AgentDojo, whose first author is CaMeL's first author, with Tramer co-authoring both and the baseline defences implemented by CaMeL's authors (d1, d3). Nothing concealed, entirely normal practice, and under the independence rule still self-report rather than validation - so claim 153 is needs-check while the design claims 149-152 are corroborated, on the reasoning that an architecture is checkable against its own figures regardless of who ran the benchmark. This is why ingesting AgentDojo next will not resolve it: shared authors mean it cannot validate CaMeL, and that is recorded as an open question rather than discovered later. Two syntheses this brain makes that neither paper does. Claim 155 maps CaMeL's coverage onto S17's six threat classes and finds it covers information gathering, intrusion and malware - each needs a tool call - while fraud and manipulated content are explicit non-goals and availability is unaddressed, so an information-flow defence protects actions and not assertions, leaving half of S17's taxonomy untouched. And claim 149 notes CaMeL is claim 12 arriving from adversarial robustness rather than reliability - S2 found production ships small LLM steps inside deterministic code for reliability reasons; S18 derives the identical shape from security. Also recorded: S18 is claim 106's enforcement obligation finally specified - S12 said sharing converts a structural guarantee into an obligation nobody has named, and CaMeL names it as a Python policy file somebody owns. agent-security stays established at 6 sources; S18 corroborates nothing because nothing else here proposes a defence, and the status warning was rewritten rather than removed to say the topic now holds one structural defence covering the action half, unvalidated externally, whose authors say prompt injection is not solved. |
| 2026-08-05 | 260805_memory-poisoning-systematic | Ingested "From Untrusted Input to Trusted Memory" (arXiv 2606.04329) as S19 - the third independent leg of claim 145, and the first source anywhere here that enumerates how a poisoned memory write actually happens. Run curator + mentor, fact-checker at the gate, architect on the status calls. Prioritised ahead of AgentDojo deliberately: it is the only remaining candidate independent of S16, S17 and S18, and it lands on the note's one corroborated pair. At 14 pages the whole main paper fitted in one visual read. Five artifacts kept; two crops extended after viewing cut off their final rows. 15 nodes, 13 corroborated, 1 single-leg, 3 divergences. Promoted claims 157-163; merged into agent-security.md, memory.md and skills.md. The corroborating pair became a trio - S16, S17 and S19, three unrelated institutions across 2023, 2024 and 2026, no author overlap - and S19 adds the half the other two lacked: S16 and S17 showed that memory persists a compromise, S19 shows how the write happens. Four channels, and only one is an explicit command; the other three fire on a retention policy, a compaction threshold, or the agent deciding a finished task was a reusable skill, with the model's own judgement as write authority. You cannot filter an instruction nobody issued. Three findings worth recording beyond the claims. First, a scope correction that is this brain's and not the paper's (d3): the paper concludes "existing prompt injection defenses provide incomplete coverage" and the four things it tested are all detectors; no structural or information-flow defence was evaluated, so the supported conclusion is "detection-based defences fail" and claim 159 does not refute claim 149. Second, a convergence neither paper sees (claim 163): S19's central architectural proposal is write-path provenance tracking, which is S18's mechanism aimed at a surface S18 does not cover - CaMeL's capabilities live for one program's execution and nothing carries provenance across a session boundary into a store and back. That gap is unbuilt in both. Third, V-S5 answers a question this brain wrote down before it had a source: self-improvement.md records "what is the threat model for a system that writes its own training data?", and V-S5 is more specific than the question - the danger is not one bad write but that the improvement loop then works to make the bad write more effective, treating "executed without error" as validation. That is claim 114 with an adversary choosing the noise, and it is gated needs-check because the paper measures skill-procedure insertion and never the amplification. Also fixed during the pass: a stale block in agent-security.md still reading "Status stays emerging deliberately" after ADR-0019 advanced the note - marked superseded rather than deleted, because it records what the bar looked like while it was being held. Recorded limits: the benchmark hands the payload to the agent as a labelled block rather than routing it through a real tool call (d1, disclosed by the authors), one model throughout (d2), four of five authors Huawei Canada, and MPBench has no repository URL in the paper. |
| 2026-08-05 | 260805_agentdojo | Ingested AgentDojo (arXiv 2406.13352) as S20 - the topic's first eval harness, and the strongest venue in the security set (NeurIPS 2024 Datasets and Benchmarks Track, a peer-reviewed main-conference track rather than a workshop or preprint). Run curator + mentor, fact-checker at the gate. Five artifacts kept; one crop landed on Figure 7 rather than the intended attacker-knowledge ablation and Figure 7 was kept instead, because the spread of attack success across applications teaches more than the ablation. 17 nodes, 15 corroborated, 1 single-leg, 3 divergences. Promoted claims 164-168; merged into agent-security.md, evals.md and agents.md. The use restriction is the first thing recorded (d1): AgentDojo shares two authors with S18 - Debenedetti first-authors both, Tramer co-authors both - so nothing in it validates CaMeL, S18's 77% is measured on this benchmark, and the "next best defence is a tool filter" ranking S18 reports is the same team's benchmark measuring the same team's baseline. That open question was recorded when S18 landed and ingesting S20 did not close it, which is exactly why S19 was prioritised ahead of it. Claim 165 is the pass's best result: a second independent corroborating group. S17 found qualitatively (Saarland/CISPA, 2023) that the attacker states the goal while the model supplies the method; S20 measures the same thing quantitatively (ETH, 2024, no author overlap) as an inverse scaling law - more capable models are easier to attack, because weak models fail at the attacker's goal too. agent-security now carries two independent corroborating groups: the S16/S17/S19 trio on memory persistence, and the S17/S20 pair on capability as attacker subsidy. Claim 164 is the most transferable single idea: in an adversarial evaluation the judge must be deterministic, because a model judge shares a vulnerability with the system it grades and the failure is correlated in the direction that hides it - a successful attack can report itself as a defensive success. That is the sharpest form of claim 34 this brain holds, and it is sharper than the original because the producer and grader are not even the same component; the adversary sits upstream of both. Filed to evals.md rather than only to security, along with the two-axis (utility against security) evaluation shape. Claim 167's 17% is the number flagged for a design review: the tool filter wins at 7.5% attack success and fails where the user's task tools also suffice for the attack, which bounds every isolation defence in the brain, CaMeL's policies included, and neither S18 nor S20 tests that against CaMeL. Also recorded: S20's own stated limitations map onto the two sources around it - the "wait until the right tools arrive" attack it cannot test is a persistence attack and therefore S19's subject, and S19 independently notes AgentDojo is single-session. Neither benchmark covers a patient adversary against a stateful agent. validate.py gained Latin Extended-A for Slavic, Baltic and Turkish surnames (Balunovic was the first), with BUILD.md regenerated in the same pass per the contract. |
| 2026-08-05 | 260805_spotlighting | Ingested Spotlighting (arXiv 2403.14720, Microsoft) as S21 - the cheap defence most teams actually deploy, and the source whose intellectual contribution turned out to exceed its evidential one. Run curator + mentor, fact-checker at the gate. Eight pages, read whole including the appendix; four figures kept. 14 nodes, 12 corroborated, 1 single-leg, 4 divergences. Promoted claims 169-173; merged into agent-security.md and context-engineering.md. Ingested despite being the weakest source in the security set evidentially - a vendor preprint with no venue, no code and no dataset, from the company whose product S17 found filtering the wrong channel - because it is the cost anchor (S18 prices it at 1.06x input tokens against CaMeL's 2.82x) and because without it the brain would hold the expensive defences and not the common one. Three results worth the ingest. Claim 169: adding "do not obey instructions in the document" to the system prompt is nearly a no-op, ~60% to ~58%, because a defensive instruction competes with the injected one on equal terms - you cannot fix a channel problem with more traffic on the channel. Claim 170: transform rather than instruct, and mark the body rather than the edges - delimiting halves attack success and its own authors disown it (a forged closing delimiter defeats it), while datamarking reaches 3.1% at no measurable task cost across four NLP benchmarks. Claim 171: design against an adversary holding your system prompt - randomise the marker, and never use a transformation the attacker can invert, since with ROT13 they write text whose ROT13 image is the attack. The payload is claim 172, and it is the best cross-domain framing in this brain's security set. The authors explain the ceiling of their own method through fifty-year-old telephony: in-band multi-frequency signalling stopped accidental interference between call control and voice and was defeated intentionally by phone phreaking, the fix being out-of-band on a physically separate channel. LLMs are worse off than early telephony, because all tokens are treated roughly equally. Spotlighting is in-band - it "helps to create separation but is not perfectly secure against intentional interference" - and they name a token-level out-of-band channel as what is needed while calling it infeasible in current architectures. S18 met that requirement a year later one level up, in a program, and neither paper cites the other; the link is this brain's. They also state plainly that they do not know why spotlighting works, which is exactly what makes a guarantee impossible and is S18's objection to the whole heuristic family, reached here by the authors' own route. Claim 173 is the synthesis the fourth defensive source made possible: defences sort into detection, behavioural and structural by what they ask of the model, they fail for unrelated reasons so they compose, and all three act on the input path while none survives a memory write - which is S19's subject. Gated needs-check because the taxonomy is this brain's and no source draws it. Recorded limits: the headline ">50% to below 2%" is a best-case composite (d2), and every experiment is non-agentic (d3) - the only variant anyone has since tested against a real agent is delimiting, the one these authors recommend against, so datamarking and encoding have never been evaluated against a tool-calling agent in any source here. |
| 2026-08-05 | 260805_darwin-godel-machine | Ingested the Darwin Godel Machine (arXiv 2505.22954, ICLR 2026) as S22 - the best-evidenced source in this brain, and the second primary autonomous-research-loops.md had been waiting for since ADR-0017. Run curator + mentor, fact-checker at the gate, architect on the topic call. Last of the five-paper batch and the only non-security one. 72 pages, of which 9 are main paper; the 63 pages of appendix (ablation detail, prompts, per-modification diffs) were deliberately not read. Four figures kept, three re-cropped. 15 nodes, 12 corroborated, 3 single-leg, 4 divergences. Promoted claims 174-177; homed in autonomous-research-loops.md, cross-referenced from self-improvement.md and agent-security.md. ADR-0020 resolves two conditions at once. ADR-0017 recorded the note as a merge-back candidate if no second primary arrived; one has, and it is independent of S13 in every respect and far better evidenced, so the merge-back warning is removed. ADR-0018 asked for its artifact-versus-model boundary to be re-tested on the next source landing claims in both; S22 is that source and it scopes itself, since §3 concedes the work covers coding agents with frozen pretrained FMs while the abstract's framing suggests more (d3). The model never changes; the scaffolding does. The boundary held without adjustment, and a narrower re-test trigger is recorded: a single loop modifying both artifact and weights. Claim 174 is the first ablated architectural claim this brain holds - remove the archive and progress plateaus lowest, remove self-improvement and it plateaus in the middle, with SWE-bench going 20.0% to 50.0% over 80 iterations and the gains transferring to held-out models, benchmarks and languages. Claim 176 is the constructive payload: two freezes with stated reasons - a viability gate admitting only agents that compile and can still edit code, which is a liveness invariant deliberately separate from the performance metric, and a frozen meta-level, since anything that can rewrite its own selection criteria can rewrite them to prefer itself. That is claim 113 avoided by construction, where S13's protected metric still reached the decision through code the agent could rewrite. Claim 177 is the pass's most valuable result and it was not anticipated. S22's Safety Discussion states that if benchmarks "do not fully capture all desired agent properties, the self-improvement loop could amplify misalignment over successive generations" - which is S19's V-S5 said by the builders of such a loop, about their own system, non-adversarially. With S13's observed random-seed result that makes three independent sources on one mechanism: a self-improving loop optimises what it can measure and compounds what it cannot, of which S13's is the only observed instance and the other two are predictions. Claim 124 is the frame that explains all three - the verifier sets the rate of improvement and of silent degradation. Also recorded: every S22 safeguard is containment, not correctness (sandbox, time limit, scoped modifiable surface, traceable lineage), consistent with a paper whose premise is that a correctness check is impossible; and the authors name an unmodifiable supervisor as future work, which is this note's freeze pattern proposed for the safety layer and which nobody has built. The code is open-sourced and was not cloned - the cheapest un-taken second leg of the five sources ingested today. |
| 2026-08-05 | brain (kit) | Dream 0002 - the first reconciliation pass after a large single-session ingest (brain/dreams/0002-260805.md). Run architect + fact-checker, whole brain. Between dream 0001 and this one the brain went from 11 sources and 100 claims to 22 sources and 177 claims, with seven sources landing in one session, agent-security advancing to established, and two ADRs written - the largest compounding burst this repo has had. 8 findings applied, 3 proposed, 7 classes clean. The headline: belief drift is still zero and bookkeeping drift multiplied without diversifying. Nothing contradicts anything, no confidence needed moving, no topic needed splitting - and seven of the eight findings are the same two bugs dream 0001 already named. Three are stale source counts, including memory.md where the note said 4, its own list said 6 and INDEX.md said 5 - all three disagreeing. Three are absolute statements that a later source in the same session falsified and nobody revisited: agents.md's "nothing here yet on agent security" (written at two sources, survived twelve more, and pointing at agent-security.md as still having no threat source), agent-security.md's "nothing in this brain addresses fraud and manipulated content" (written during S18's compound, four hours before S21 landed a partial answer), and skills.md asking "nothing yet on skills as an attack surface" directly below a section added the same day answering it. One is rag.md's hard-coded brain-wide source count going stale for the second time - dream 0001 fixed it 8 to 11, it is now 22, and this pass replaced the number with a pointer so it cannot recur. The strongest recommendation is to build dream 0001's P1 check, which was proposed after three occurrences, never implemented, and has now fired three more times; six occurrences across two passes is a missing check rather than a pattern. Also recorded for the next pass: the identity-split trigger was tested against S18 and did not fire - CaMeL's "capabilities" are intra-process data-flow labels with no principal, token, flow or delegation, failing ADR-0015's test the same way S12 did - and /verify is now owed on seven sources written by the same session that gated them, which the contract forbids running in-session and which is the largest un-taken quality check in the repo. |
| 2026-08-05 | brain (kit) | Conjecture pass 2 - the first full run of the generative stage (brain/conjectures.md). Run synthesizer, whole brain, immediately after dream 0002. 5 generated (h5-h9), 4 discarded. Pass 1 (2026-08-03) flagged its own scope gap - it generated from claims.md plus whatever topic notes happened to be in session, never a fresh read of every note's Open questions, which is where the unclaimed material sits. This pass read all eleven, on a brain that had doubled since: claims 1-100 became 1-177, 11 sources became 22, almost all of it security. Four of the five combine a source from the last two days with something older, which is the pattern the contract predicts - new material generates against the existing stock rather than against itself. h5 and h9 are the two worth running first. h5 asserts that a self-improving agent's discovered scaffolding transfers its attack surface along with its capability, combining claim 174 (DGM's tooling transfers to held-out models, largest gain on the most capable one) with claim 165 (more capable models are easier to attack) - and both artifacts are open source, so running base and DGM-discovered agents through AgentDojo is a weekend. h9 asserts that drift in a compounding knowledge base accumulates with cross-references rather than sources, which would make this design's maintenance cost super-linear in coverage; it is the only entry testable with no external evidence at all, because the dream passes generate the data. h6 is the one that changes a design decision: provenance tracking is worth more as a recovery control than a preventive one, because prevention needs write-time maliciousness detection that claim 159 shows is undecidable for weak-signal payloads, while recovery needs only lineage, which is decidable by construction - and claim 163 records two independent groups proposing it for the preventive job. Four discards recorded with reasons, three of them restatements and one (provenance spanning sessions) an already-recorded open question rather than a proposition. No conjecture required a new source; all five were available from material already sitting in the brain unremarked, which is the argument for running this stage at all. Also fixed in-pass: the first write split the conjectures table with a stray blank line and omitted the Status column on three rows - caught by re-reading the rendered table rather than by validate.py, which cannot see a broken table. |
| 2026-08-05 | 260804_camel-prompt-injection-defense | Deep research pass R3 on S18 - the first external evidence in the security material, and it answers the question this brain recorded as open twice (context/01). Run fact-checker + synthesizer, targeting S18 n9/n10/n11 - the efficacy nodes gated needs-check because AgentDojo's first author is CaMeL's first author. Budget: 2 searches, 4 fetches of the permitted 8 and 12. Applied the owner's standing source floor for the first time: two blog hits from the opening search were dropped unread, and discovery ran citing-forward from the ingested papers. Verdicts: 1 supports, 3 refines, 0 contradicts. The finding: CaMeL has been independently evaluated, and it is bounded rather than refuted. AgentDyn (Li, Wen, Shi, Zhang, Vorobeychik, Xiao; arXiv 2602.03117 v3, T3, no author overlap with S18) builds 60 open-ended tasks and 560 injection cases, evaluates ten defences, and finds CaMeL at 0.00% utility and 0.00% attack success on open-ended dynamic tasks across every model - while its AgentDojo result replicates. The cause is structural: CaMeL writes a static program before execution, and an open-ended task's plan cannot be written in advance. That is perfect security by accomplishing nothing, the degenerate corner S20's two-axis design exists to expose, and it took a third party's benchmark to surface it. Claim 153 may now only be cited for plannable task suites. Three further results. Claim 167's 17% was the bound on a static benchmark - AgentDyn finds planning-dependent defences (Tool Filter, CaMeL, DRIFT) over-defend on dynamic tasks by blocking tools that look unnecessary at plan time, which is the limitation S20 named about itself and could not test. Claim 159 gains independent support from a second team on a different axis: where S19 showed detectors miss what looks legitimate, AgentDyn shows them block what is, both from "limited ability to distinguish helpful instructions from malicious injections". And S21's d3 is closed - Spotlighting had never been evaluated agentically by anyone, and on AgentDyn reaches 52.24% utility under attack at 27.61% ASR, so its low utility cost holds while its security benefit does not transfer from document tasks. Claim 173's taxonomy is partially supported: two 2026 preprints use "out-of-band defences" as the standard term for the structural class, which is S21's telecom analogy adopted as field vocabulary - but a shared word is not a shared taxonomy and neither draws the three-way split, so it stays needs-check. The coverage gap the pass exposed is larger than its findings. This brain holds three defences; AgentDyn evaluates ten and a second preprint names five in the structural family alone. Missing: Progent, DRIFT, FIDES, RTBAS, FORGE, Meta SecAlign, PromptGuard2, ProtectAI, PIGuard - and the three held are the three ranked planning-dependent, over-defensive or weakly effective, while the best of the ten (Meta SecAlign-70B, 53.35% utility / 8.98% ASR) is one it has never heard of. No defence in that comparison is acceptable on both axes. Next three sources named by the evidence rather than guessed: AgentDyn, Meta SecAlign, Progent. One independence fact recorded for later: Chaowei Xiao co-authors both AgentDyn and S16, so R3 is not a free second witness for anything S16 asserts. |
| 2026-08-05 | reports | Wrote 260805_isolation-ceiling-of-an-mcp-config.md - the first report written to decide whether to build something rather than to teach. Run synthesizer. Context: the owner co-built MCP Shark (github.com/mcp-shark/mcp-shark, Wireshark-like forensic analysis for MCP - 41 rules, toxic-flow analysis, tool pinning, SQLite traffic store), and the whole agent/MCP-security ingest track exists to find the next idea adjacent to it, which had not been stated until now. The proposal: compute statically the fraction of an MCP config's tool surface on which no plan-time isolation defence can help, by classifying tools read/write and measuring how often a task's required tools already suffice for an attacker's goal. Grounded in claim 167 (the tool filter wins at 7.5% and fails on the 17% of cases where task tools suffice for the attack), claim 166 (attack success is a property of the application), and R3's finding that planning-dependent defences - Tool Filter, CaMeL, DRIFT - collapse when the plan cannot be written up front. Three things make this report worth keeping as a template. First, it names its own weakest link in §3: the load-bearing step is conjecture h7, not a claim, and if defence failure rate does not track read/write overlap the metric predicts nothing. Second, the prior-art check bounded the novelty downward and is recorded as such - one arXiv search returned ChainCaps, Dynamic Capability Scoping, Prompt Flow Integrity and AgentSecBench, so the defensible claim shrank from "unclaimed territory" to "no one computes this as a static property of a deployed configuration", and Isolation as a First-Class Principle is flagged as the survey most likely to have got there first, to be read before any code. Third, §8 sequences validation before building: test h7 against AgentDojo's four suites, which is four data points - not a result, and enough to kill the idea - and note that a refutation is also a real outcome this brain records rather than discards. Also recorded: the claim-155 scope limit means the metric covers the action half only, so a config could score 0% un-isolatable and still be exploitable by fraud or manipulated content, and the number must ship with that caveat. |
| 2026-08-05 | reports | Executed step 1 of 260805_isolation-ceiling-of-an-mcp-config.md's own plan: read the prior-art survey, and it clears the idea. Isolation as a First-Class Principle for LLM-Agent System Safety (Jing et al., arXiv 2607.12406 v1, 2026-07-14) was flagged in that report as the single most likely place for the isolation-ceiling metric to already appear. Two probes with deliberately different vocabulary - one on metrics and capability sufficiency, one on over-restriction, least privilege, authority budgets and utility trade-offs - both negative on the same three points: no quantitative metric of any kind (Table 1 organises papers by boundary with no evaluation measures), no read/write overlap or tool-sufficiency analysis, and no argument anywhere that isolation is upper-bounded by the system's own required capabilities, which is the report's exact proposition. It cites CaMeL as a privilege-separation example and mentions neither Progent, AgentDyn nor tool filtering by name, so a July 2026 survey is already behind R3's material. Two findings in it actively help and are now cited in §6.1. Its own open-challenges section says "the field still lacks stable abstractions for authority, trust, and privilege in full workflows" - a survey stating the gap is better support than asserting it - and it observes that "many benchmarks still test single boundaries, while real failures are cross-boundary", which is an independent argument for the vehicle, since MCP Shark's toxic-flow analysis is already cross-server. Its boundary taxonomy also gives the metric a place to sit: it measures the agent-tool boundary. Recorded honestly as a bounded clearance: two targeted probes by a page-summarising model, not a full human read, and absence of a metric in one survey is not absence in the field. The survey is deliberately not added to reading-list.md - surveys are T5 under the aggregator rule and that file excludes them by policy. Net effect: the report's remaining risk is no longer novelty, it is whether conjecture h7 is true, and the next action is the AgentDojo correlation test rather than more reading. |
| 2026-08-05 | reports | Tested conjecture h7 against AgentDojo's published data - it holds, and is promoted to claim 178. The first result this brain produced itself rather than read. Run synthesizer + fact-checker. Method, entirely over artifacts S20 publishes and with no model calls or re-runs: AST-parse each suite's ground_truth tool sequences, mark a (user task, injection task) pair un-isolatable when the injection's required tools are a subset of the user task's, and read the shipped run data for the no-defence and tool_filter pipelines. Reproducible in one file, reports/experiments/260805_h7_agentdojo_test.py. Result: under the tool filter, un-isolatable pairs are attacked 46.7% of the time against 0.2% for isolatable pairs - a 220x risk ratio, chi-square 227.2 on n=547 - and 35 of the 36 attacks that survive the filter are un-isolatable pairs. The control is what makes it a mechanism rather than a correlation: with no defence the same split predicts nothing (50.7% against 47.9%, chi-square 0.2), so un-isolatability is not a proxy for general attackability and becomes predictive only once the defence is applied. This makes S20's 17% a property of the task and tool surface rather than a constant of the defence (claim 167), which was exactly h7's assertion. The methodological finding is worth as much as the result and is recorded in both the conjecture register and the report: the first cut was the per-suite correlation the plan originally specified, four data points, and it came back inconclusive and mildly discouraging - Spearman +0.80 against tool-filter ASR, but equally correlated with baseline ASR, with the discriminating residual-ratio test running the wrong way at -0.40. Aggregating to four suites was hiding a 220x effect; the per-pair unit was both the correct test and available all along. Before using the run data at all, the reading of its security field was verified against three published figures - Slack baseline 92.4% against the paper's stated 92%, overall baseline 47.7% against Figure 6a, tool filter 6.8% against the reported 7.5%. Claim 178 is gated needs-check for an unusual reason: the data is third-party and peer-reviewed while the analysis is this brain's and unreviewed. Three limits carried: task tool sets came from the benchmark's ground_truth, which no real deployment has; 87% coverage, 82 of 629 pairs skipped where ground truth would not parse statically; one model and one attack type. The report's risk register was rewritten accordingly - the "h7 might be false" risk is retired and replaced by the narrower one that now governs the build: approximating task tool sets without ground truth, whose error is unmeasured and which AgentDojo can validate directly because it has both versions. |
| 2026-08-06 | reports | Wrote a handoff pack for another repo: reports/mcp-shark-isolation-ceiling/ - the first artifact here written to be consumed by a coding agent rather than read by a person. Run synthesizer. Two documents. background.md is self-contained context: the threat model, the three defence classes with their measured failure modes, claim 178's numbers including the null control, what MCP Shark already has, and five hard constraints - the most important being that the metric covers the action half only, so a config can score 0% un-isolatable and still be exploitable. spec.md is the build, and its Phase 1 is a blocking validation gate rather than a step, because the 220x finding used AgentDojo's published ground truth and a real config has none. Three design decisions worth recording, all of them corrections to the original request. First, "self-contained" and "linked" were in tension and the tension is resolved explicitly: self-containment is the contract and links are provenance, so the pack works with no network and rewards having one - a coding agent in another repo may have fetching blocked. Second, every link is pinned to commit 638ebe9 rather than main, because this repo's notes are living documents that get rewritten - dream 0002 rewrote sentences in agent-security.md the previous day - and a spec must cite the sentence it was written against. Shipping a spec with the exact staleness bug this brain holds claims about would have been embarrassing, and all 7 pinned paths were verified to resolve before commit. Third, the spec states that its author has never seen MCP Shark's source, and marks every assumption about its internals [verify] rather than asserting it. Also recorded in the spec: the two sentences that are and are not defensible if any of this is published - "we are not aware of prior work" is supported by one arXiv search plus a full read of the closest survey, and "nobody has done this" is not. Set-theory notation was rewritten as prose rather than widening ALLOWED_NON_ASCII a third time in two days - the diacritics additions were load-bearing for citing researchers' names, and a subset symbol is losslessly expressible in words for the intended reader. |
| 2026-08-06 | brain (kit) | ADR-0021: the writing stage does not downshift, and the line is specifiable vs perceptual. The register retrofit exhausted the monthly quota - twelve agents spawned, ten died mid-write - so the question was whether a cheaper model could do the writing while a strong one does the reasoning. Tested as an ablation (claim 33) rather than argued: same note, same prompt, same contract, same reference implementation, Sonnet instead of Opus. Sonnet added 29% more words and did not apply the register. Its sentence rhythm was indistinguishable from the untouched original (median 21 vs 20, 90th percentile 42 vs 41) where Opus pulled the long tail from 41 to 35; discourse markers 2 vs 13; three forbidden bold lead-in labels survived. It also dissolved two of four corroborated by frame labels into prose, which are the gate's two legs made greppable and exactly what /verify check 6 reads - reported as "matching the reference implementation's style", a confident rationalisation for an edit the brief forbade. It was not cheap in the expected way either: 112k tokens and 32 minutes against ~97k and ~11. The finding that should outlive the measurement: the line runs between specifiable and perceptual, not between thinking and writing. Sonnet executed every instruction a script could check and failed every one needing a reader, which is this brain's own generation-verification gap arriving from the other side - a cheap generator is safe exactly where the verifier is cheap. An exception class is recorded (a transform checkable completely by diff/grep/validate.py, such as the pending reference-style citation pass), and the practical mitigation for quota is batch size, not model choice. needs-check and n=1 - one note, one prompt, one run, noise floor unmeasured, which claim 115 warns about. |
| 2026-08-07 | mcp, agent-security, agents | S23 ingested: Google's MCP stateless-updates announcement - sources/260807_mcp-stateless-updates/. mcp.md's first primary source, and the first anywhere in this brain to pin a spec version (2026-07-28, superseding 2025-11-25). Claims 179-183. The mechanism: the initialize handshake and Mcp-Session-Id deleted, _meta carrying the negotiated fields on every request, routing metadata promoted to mirrored HTTP headers with a -32020 mismatch code. The finding is what the article does not say - statelessness here is relocation, not elimination, to the wire, to the client and to your application, and its own "No Redis Sessions Needed" headline is contradicted four sections later by its own Tasks example (d2). The sharpest node came from executing an artifact rather than reading one: the requestState blob in the source's own MRTR example base64-decodes to unsigned plaintext JSON, beside the elicitation "Are you sure you want to delete these 3 files?" (n8, d1, claim 181) - which inverts claim 29 and defeats claim 28. The visual leg was skipped because the article has no figures, and the six printed protocol payloads turned out to be a better second leg than a diagram, because a payload can be decoded and a diagram can only be looked at. Two standing open questions closed: the spec version, and how MCP authorization builds on OAuth (RFC 9207 + RFC 8707, claim 182 - the audience restriction mcp.md and agent-security.md both asked for). Status held at emerging anyway - three disjoint sources, no two confirming the same mechanic - recorded as ADR-0022 so the "count rose, corroboration did not" complaint stops being rediscovered. No deep research was requested, so the specification itself is unread, and the requestState finding is a documentation gap rather than a proven design flaw. |
| 2026-08-14 | 260814_hermes-agent-architecture-p1 | S24 ingested: the brain's first source on the operational runtime of an agent - sources/260814_hermes-agent-architecture-p1/. Run curator + mentor, fact-checker at the gate, architect on the topic call. Twenty-four sources in, this is the first to work identity, persistence, idempotency and delivery rather than the shape of the loop. All four figures viewed and all four kept; 24 nodes, 22 corroborated, 2 single-leg, 5 divergences. Promoted claims 184-196; merged into agents.md (new "runtime half" section), agent-security.md, context-engineering.md and evals.md. The gate earned its keep twice. d1: the article's closing checklist tells the reader to determine whether their single-writer guarantee is process-local or distributed, and its own table already answers that about its own subject - the active-run guard's durable form is "memory only" - which the prose never states, so a reader taking the prose alone finishes believing the opposite (claim 191). d3: the piece opens by constructing a deterministic SHA-256 task specifically so the architecture can be traced through it, and that trace is never shown running - no session ID, no routing key, no transcript row from any real run appears, and the closing section hands the experiment to the reader. And the finding that outranks both: its subject is the same running artifact S19 attacked (claim 196). S19 names HERMES as one of its two evaluated agents and attributes it in its own bibliography to Nous Research, citing the project's developer guide; S24 documents that same project's session architecture. First time this brain holds an independent measured attack and an independent architecture description of one artifact. The use restriction was written before the claim (d5): Part 1 is not about memory, says so, defers it, and therefore moves neither claim 160 nor 161 - it confirms S19 attacked a real released product rather than a mock, supplies the mechanism behind the low compaction threshold S19 could only observe from outside, and exposes the write path S19's benchmark short-cut. Declined a new topic (ADR-0023): the area is real and reusable but has one unmeasured T4 source, so the material parks under the four existing notes with the create-trigger recorded - a second independent source teaching delivery or runtime-state semantics, which Parts 2-5 of this same series would not satisfy. ⚠️ Nothing in the source is measured, and both corroboration legs are one author's prose against the same author's diagrams. Author independence from Nous Research was checked and runs favourably. The repository was not cloned and is now the cheapest un-taken second leg in this brain - added to the reading list alongside Part 2, which outranks Part 3 because S19's stated injection mechanism is a system-prompt write and prompt assembly is Part 2's subject. |
| 2026-08-14 | brain (kit) | The TL;DR diagram becomes part of the LEARNING.md frame - a mermaid diagram inside ## TL;DR, under the prose, hard wherever the argument has a shape (the same conditional the mental model already carries). Written after S24 got one and it worked: the notes now run 5,000-9,000 words, and a reader deciding whether to spend twenty minutes was being handed four sentences of prose, which is the slowest possible way to convey a shape. The three existing compressed forms all answer questions in words - what is this, what does it say, what may I cite - and none of them showed the reader the structure of the argument before committing to it. The rule that took the actual thinking is the anti-duplication one. A note in this shape now carries up to three diagrams, and they collapse into three views of one picture almost by default, which wastes two of them: the TL;DR diagram owns the shape of the argument, the roadmap owns the reading order, the mental model owns how the subject works. The reliable trick, recorded with the worked example from S24, is to draw the note's thesis rather than its subject - S24's mental model already owned the six-stage flow, so its TL;DR diagram hung six "X is not Y" collapses off that spine instead, each pinned to the stage where it bites. Also fixed: build_site.py's render_home lifted TL;DR and Key claims onto the landing page through md_to_html without mermaidify, so the first diagram placed there published as a visible language-mermaid code block with the bundle not even loading - the failure was shipped and caught the same day, and is recorded in the contract so the next agent does not rediscover it. Deliberately not enforced by validate.py: whether an argument has a shape, and whether three diagrams are three different diagrams, are both judgement, and encoding that would launder judgement as a green check (same line ADR-0004 draws). Reinforced at the four points the 2026-07-25 audit established: contract (AGENTS.md), persona at the point of action (curator.md, mentor.md), template scaffold (sources/_TEMPLATE/LEARNING.md), and - here - a deliberate, stated absence at the validator. No ADR: the LEARNING.md shape has always been amended inline in AGENTS.md rather than by ADR, and this follows that precedent. BUILD.md regenerated in the same pass. |
| 2026-08-15 | experiments (kit) | The brain ran something for the first time - new experiments/ layer (ADR-0024) plus its first entry, 260815_runtime-boundaries. Twenty-four sources in, this brain had never executed anything, and recorded the gap about itself four times in INDEX.md ("ran none of the code (no GPU)", "repo not cloned" twice, "the cheapest un-taken second leg"). S24 turned that into two open questions answerable by running something and by nothing else, since nobody has published on either. A ~230-line stdlib harness - no model, no network, no API key, which is claim 194 taken at its word - tested four claims. PREDICTIONS.md was committed in its own commit (f53b44b) before any code existed, so git log proves it was not written afterwards; that is claim 34 applied to the one agent that would otherwise mark its own homework, and it is mandatory rather than stylistic. Two answers to standing open questions. Case 02, the reason this existed: two OS processes, one state.db, WAL plus busy timeout - all 80 rows landed, zero SQLITE_BUSY, and every turn number was written twice with half the increments lost. Perfect database integrity, semantic integrity destroyed, no error raised anywhere. n9's distinction between a storage concurrency rule and a semantic one has teeth, and a reviewer who reads "SQLite serialises writers" and concludes the conversation is safe has made a real error. Case 01: parallel tool results diverged from side-effect order in 12% of 200 runs with no artificial delay (49% with sub-3ms jitter), so n14 describes something a system meets by accident. And one finding nobody had asked for, which is the most useful thing here: the delivery ledger closes the commit-to-send window (0/25 duplicates) and is powerless in the effect-to-commit window (25/25 duplicates, ledger enabled and working as designed), because the evidence it reasons over is written after the effect it protects - so at-least-once delivery and at-least-once side effects are two different guarantees and only one was bought. Case 04 refined claim 187: the silence is a property of tool-layer forgiveness (exist_ok=True), not of the architecture, which makes the mitigation cheaper than "confirm the workspace". Recorded against itself: every prediction hit, which is a criticism of the predictions rather than a triumph - the 5-40% band on case 01(a) was too wide to be falsifiable, and case 02's magnitude was a partial miss by understatement. No node's gate changed and no claim's confidence rose: per ADR-0024 an experiment tests a mechanism and never corroborates the source that suggested it, so none of this is evidence about Hermes. Two S24 open questions struck through with pointers; n19 and claim 190 gained the effect-to-commit window. |
| 2026-08-15 | 260815_cybersecurity-evals | S25 ingested: the brain's first survey source - sources/260815_cybersecurity-evals/. Run curator + mentor, fact-checker at the gate, architect on the ADR. Seven cybersecurity benchmarks read at once (Cybench, CVE-Bench, CyberGym, ExploitGym, ExploitBench, MHBench, SCONE-Bench), which is the first time this brain has seen one eval-design problem solved seven independent times. 27 nodes, 8 divergences, 11 figures viewed and 8 kept. Promoted claims 197-205, amended claim 132 with a fourth instance rather than stacking a duplicate, merged into evals.md (primary home, new synthesis section), agent-security.md and agents.md, and added nine glossary terms. The structural call this source forced is ADR-0025: a secondary source corroborates its own reading, not the world. Every previous source here was a primary; Yan ran none of these seven experiments. Running the gate mechanically would have produced a page of corroborated / OK on a source where not one number is first-hand, because the two legs available are his prose against a figure lifted from a paper he did not write - so agreement establishes that he read the paper correctly and nothing else. Two amendments, both stated in the gate note rather than left implicit: corroborated degrades to OK (faithful summary) (the verdict word stays, so nothing ripples into validate.py; the confidence cell carries the information), and where prose and figure disagree the figure wins and the prose is the recorded defect, since the legs have different provenance and different authority. That asymmetry applies only to lifted figures - fig1 and fig2 are Yan's own diagrams and were gated symmetrically. The gate earned its keep five times in one direction (d1-d5, the article understating its own figures) and three more in other ways. d2 is the headline: a chart annotated "Doubling time ~1.3 mo" with a 90% CI and R^2 = 0.828 sits inside an article whose prose reports two endpoints from it and never mentions the trend, the fit or the doubling time - the most consequential quantity in a 3,960-word article about measuring offensive capability exists only inside an embedded image (claim 204). d5: a footnote records that with default safety filters enabled all exploit attempts by a model scoring 120 in the table above it are blocked, summarised in prose as "some model refusals still occurred" - a total effect reported as a marginal one (claim 203). n17: an entire adaptive-coaching experimental arm occupies half a table's rows and appears nowhere in the text, and it lowered the best model's top-tier result. d1: an 11-minute difficulty ceiling stated as a blanket fact belongs to the unguided regime only, with the subtask-guided column reading 52 minutes and 2h03 (claim 202). Also d3, d4, d6 and d8 - the last being a zero-day evaluation over 2,849 contracts announced and never reported. Three claims are the durable payload. Claim 198 (define success as a checkable property of the environment rather than of the trajectory, so the grader never needs to be as capable as the agent) and claim 199 (score as an ordered ladder, not a bit) are both downstream of one property - exploitation has a free mechanical verifier, which is claim 124 seen from its good end. Claim 197 is the synthesis and is marked needs-check because the five dials are this brain's framing and not the article's: the article turns all five and never lines them up. Best-evidenced claim is not the largest one - claim 201's 3-of-40-to-37-of-40 reads as a finding rather than an announcement only because component-wise ablations sit under it, and it ships with its own bound from the same source, since no scaffolding crossed the sandbox-escape cliff. One conflict recorded and deliberately unresolved: two benchmarks put a model on the scoring path of an adversarial eval, which claim 164 argues is unsound - noted as weaker than AgentDojo's in-loop case, with the observation that 94% inter-auditor agreement measures agreement rather than correctness (d7). ⚠️ The whole note rests on six T3 preprints and one T2 vendor page, none fetched, and the two most quotable findings (claims 204 and 205) are the two gated weakest. The What has aged section is the first here written about a two-month-old source, on the source's own arithmetic - 1.5 doublings between publication and ingest. |
| 2026-08-15 | 260815_llm-knowledge-bases | S26 ingested: the first independent instantiation of a source this brain already holds - sources/260815_llm-knowledge-bases/. Run curator + mentor, fact-checker at the gate. Ben Holmes (Warp) built Karpathy's llm-wiki pattern and demonstrated it; that pattern is S8, ingested 2026-07-31. The gate's whole job here was resisting a promotion that would have felt like progress. S8's eighteen nodes are all single-leg/needs-check because it is a text-only gist with no second leg, and a talk that displays the gist in full, names it on stage and agrees with every word of it is the same leg wearing a different hat - same author, same document, same revision. No S8 node moved and no S8 confidence rose, recorded as claim 212 and as a standing warning at the top of rag.md. This is ADR-0012 arriving from the opposite direction: that rule stops a passing mention inflating a count, and this is its mirror image, a source entirely about the prior work that still adds no evidential weight to it - and harder to catch, because faithful enthusiasm reads like corroboration. What is genuinely independent is instantiability: somebody other than the author built it and ran it on a real corpus. 16 nodes, 2 divergences, 32 candidate frames triaged in 4 contact sheets, 9 viewed at full resolution, 8 kept. Promoted claims 206-212; merged into rag.md (primary home, new synthesis section), memory.md, skills.md and agents.md; seven glossary terms. The best finding is a contradiction the talk never notices (d1): enrichment writes titles, frontmatter and backlinks into the raw notes while the same talk displays S8's "raw sources are immutable" and encodes it as a hard constraint in its own scheduled job. It is a refinement rather than a defect - the strict rule cannot be implemented, because capture must be frictionless so notes arrive with no metadata and it has to land somewhere, and enforcing purity means a shadow file per note to protect a property git already supplies. What survives is "one declared writer per layer, with the exception written down" (claim 211), and its cost is stated with it: the audit story moves from "the agent could not have edited this" to "check the history". The four mechanisms the pattern turned out to need are the payload - idempotence stamp (claim 206, and the missing precondition under claim 59's decoupled curation, which is why S8 says "periodically" and S26 says "daily"), controlled tag registry (claim 207), unattended scheduling, and per-directory discovered schema (claim 209). Claim 208 is a three-source synthesis nobody states: what decides where the human sits in an agent loop is reversibility, not autonomy (S2 asks inside the run, S26 reviews the batch diff, S13 suppresses the check-in). A capture near-miss worth recording as a kit fact: all DASH formats returned HTTP 403 because no JavaScript runtime was installed, the only downloadable format was 360p progressive, and at 360p every dense screen in this talk is illegible - the gist, the SKILL.md, the scheduled prompt. A transcript-only degrade was one decision away, which would have gated a talk whose entire payload is on screen. Fixed with brew install deno and pip install yt-dlp-ejs - both are required, deno alone still fails the challenge - after which 1080p downloaded normally and every node was gated against it. ⚠️ Nothing in the source is measured (n16), it carries a T2 commercial interest on its most novel section (d2), three of its most interesting mechanisms are figure-only (n11, n12, n15), and the review step carrying the whole safety argument for unattended operation is one unexamined sentence (n13) - the same shape as S7's d4, the load-bearing step with no mechanism behind it. |
| 2026-08-15 | brain (kit) | Two coverage numbers become visible, and a capture failure that nearly cost a source its second leg gets written down. Both came out of the S26 ingest rather than from planning, and both are counts made observable rather than new machinery - the deliberate choice, because the stages they describe were never broken. The finding that prompted it: /verify shipped 2026-08-03 and over the next twelve days fourteen sources were ingested and one was verified. The corpus nearly doubled and the evaluator ran once. Nothing was wrong with the stage, and its never-automatic trigger is still correct - an agent holding a source's argument for an hour has no independent vantage point on it (claim 34). What was missing is that the number appeared nowhere, so noticing 1-in-26 required running ls sources/*/verify.md, which nobody does. That is this kit reproducing the exact defect it records against S7 (d4) and now S26 (n13): the load-bearing step with no mechanism behind it. validate.py now prints a Coverage block on every run - /verify coverage, and the last dream pass with a count of sources ingested since (currently 4, last pass 2026-08-05). Informational, --no-coverage suppresses it, and it can never fail the run, because whether 1-in-26 is acceptable or a dream pass overdue are judgements, and a threshold in the validator would launder judgement as a green check (ADR-0004). AGENTS.md records that 100% is the wrong target - verification costs a separate session and most sources will never earn one - and that the honest reading is comparative: a number that has not moved while ten sources landed means the stage has quietly stopped existing, and the sources worth spending it on are the ones whose claims are most reused, not the most recent. The capture fix is two degrade-table rows and it is not a preference. Every DASH format returned HTTP 403 because no JavaScript runtime was installed - YouTube's n-challenge needs one - leaving format 18, progressive 360p, as the only download. At 360p every dense screen in S26 was illegible, so the frames looked contentless and the source was one decision from a transcript-only degrade for a reason that was not true. This is the false-STATIC failure (ADR-0006) arriving from a different direction, with the same asymmetry: re-extracting costs one download, and a wrong degrade costs the source's entire second leg, which the degrade rule then forbids reinstating. The fix needs both halves - brew install deno and pip install yt-dlp-ejs - and deno alone still fails with n challenge solving failed, which is the part that would otherwise be rediscovered. Recorded in the Env row, two degrade rows, and requirements.txt. BUILD.md regenerated. Three S26-derived ideas were rejected rather than built, with reasons, because a T4 unmeasured demo is a source of hypotheses and not of requirements. A personal-thought capture layer is a scope change that collides with the corroboration gate, and foundations/ plus conjectures.md already hold uncited material. An idempotence stamp for the dream pass would contradict its explicit "do not sample, read everything" rule and solves a cost problem this brain does not have at 26 sources. A generated graph view is cargo-culting the demo - the actual gap signal /conjecture needs is claims-per-topic, which is one grep and is now in the coverage block's spirit rather than a new renderer. |
| 2026-08-15 | 260731_llm-wiki (research) | R4: deep research on S8's n10, the ~100-source ceiling - the target four separate files had named as the highest-value one in the topic. Run fact-checker + synthesizer. Verdict refines, output to context/01. Budget 4 searches, 3 fetches of 8/12 - stopped early per the contract once two independent sources agreed. The claim turned out to be the rare kind someone had actually measured, and measured twice. LOFT (T3, Google DeepMind, 19 authors) tested 32k/128k/1M-token corpora and found long-context retrieval matches RAG at 128k and degrades at 1M, with a stated mechanism that is positional rather than capacity - accuracy falls as the answer-bearing document moves toward the end of the corpus. A LlamaIndex benchmark (T2, and its commercial interest points toward RAG, which is the safe direction) ran 5/100/1,000 papers and found filesystem tools beat hybrid RAG on quality at 5 papers (correctness 8.4 vs 6.4, relevance 9.6 vs 8.0) while 52% slower, then cross over at ~100, where RAG gets faster and quality converges. The synthesis neither source states: at ~15k tokens per paper those two results land in the same neighbourhood, near 1.5M tokens - different teams, different methods, different units, one ceiling. So n10's unit is wrong and its magnitude is a coincidence of its author's corpus: what binds is the token volume the navigation ranges over, so the same ~100 is far too low for short notes and far too high for books. Promoted as claim 213, and the topic's first external evidence on retrieval scale or on embeddings at all - rag.md had chunking, embeddings, vector stores and hybrid search at zero sources. The sharpest result is a no-evidence sitting inside the refines: neither study tested the design n10 describes. LOFT puts the whole corpus in context and LlamaIndex greps raw documents, while an index.md of one-line summaries attends to ~2k tokens rather than ~1.5M - so both studies bound n10's neighbours, and its likely failure mode is not attention at all but whether a one-line summary discriminates well enough to pick the right page. That relocated the open question rather than closing it. The cross-domain hop the contract asks for paid off and produced claim 214: n10 describes the Yahoo Directory - a curated, human-vetted hierarchical catalog - which lost to crawler-based search with a precisely recorded cause, roughly 100 editors against a web doubling every few months. That is the same constraint S8's Memex argument names (n15), which makes it the natural experiment for n13. The finding: a curated catalog has two independent ceilings and the LLM removes exactly one. It dissolves curation labour; it does not touch attention over the catalog, which Yahoo's editors never hit because a human reading a directory page does not decay positionally across a million tokens. n10 reports one number as though there were only ever one ceiling. Independence recorded in both directions. LOFT and LlamaIndex are independent of S8 and of each other, which is what carries the verdict. Anthropic's removal of the vector database from Claude Code was recorded as supports on direction only and explicitly barred from raising confidence - vendor describing its own product, corpus is code rather than prose, the primary is a podcast that could not be fetched so every written source is T4/T5 secondary, and it is the harness this brain runs in, which is about as conflicted as a source gets for this reader. n10 stays needs-check and is now needs-check for a better reason: it was unmeasured assertion, and it is now a claim whose neighbours are measured, whose unit is wrong, and whose own mechanism nobody has tested. Fed back to nodes.md, SOURCE.md, LEARNING.md (pointing at context/ rather than absorbing it), rag.md, claims.md and five glossary terms. |
| 2026-08-15 | experiments (X2) | X2: ran the experiment R4 said did not exist - experiments/260815_summary-index-ceiling. R4 had found S8's n10 bounded by two independent studies neither of which tested n10's actual design, and relocated its failure mode to whether a one-line summary discriminates well enough to pick the right page - noting the experiment was cheap and absent. PREDICTIONS.md was committed in its own commit (2bcd9fe) before any code in the directory existed, per ADR-0024, and five predictions were registered with falsifiers. This brain's own corpus is a live instance of the design and carries mechanical ground truth, so the measurement was free: Arm A is the 26 INDEX.md source rows (a demonstration - 26 items cannot answer a question about 100), and Arm B is the 475 gated nodes across all nodes.md files, each with a bolded one-line crux, with 1,161 LEARNING.md sentences citing exactly one node as queries. Scoring is lexical TF-IDF against N-1 sampled distractors, no model, per ADR-0024's determinism rule - which bounds the information in the summary rather than simulating an agent, and is stated as a floor. Two findings, promoted as claims 215 and 216. There is no knee. Across more than seven doublings to N=475, Recall@1 falls by a near-constant 3.05 points per doubling for rich summaries and 4.05 for one-line ones, and no doubling anywhere dropped more than 1.4x the median against a pre-registered falsifier of 2x. So "ceiling" was the wrong word, inherited from n10 and carried into claim 213, which is amended: what happens near a hundred items is cost overtaking a quality curve that is still declining gently, making it a budget decision rather than a capability limit. And richness buys slope, not offset - the one-line curve is 28% steeper and the gap widens monotonically from 11.0 points at N=8 to 18.3 at N=475 - a direct correction to n10's "one-line summary" specification and a retrospective justification for this brain's long annotated INDEX.md rows, which were chosen for readability rather than retrieval. Three of five predictions were wrong and the scoring says so. P2 missed badly (a 39.5-point richness gap against a predicted 10-25). P5 missed narrowly on the high side (~56.5% at N=100 against a 25-55% band that had been called "deliberately wide" in advance and was still wrong). And P4 is recorded as a miss despite surviving its own falsifier - the prediction said the rich-vs-one-line gap stays roughly constant, the falsifier was a 2x widening, and the observed 1.6x monotone widening passed it while being exactly the systematic effect worth catching. The transferable lesson is about the instrument, not the result: a falsifier calibrated to "obviously broken" cannot detect "systematically wrong", which is the interesting case - and it is the mirror of the previous experiment's failure, where every band was so wide that everything hit. Ceiling respected throughout (ADR-0024): no source's confidence moved, n10 stays single-leg/needs-check, and both new claims are marked as mechanism evidence only. Limits carried: n=1 corpus, one writing pipeline, a lexical scorer that is not an agent, and a predicted-in-advance authorship confound that inflates the absolute levels while probably leaving the promoted shape alone - untested, and now rag.md's cheapest open question. |
| 2026-08-15 | brain (kit) | A sixth persona - presenter - and a ## Presentation narrative section appended to LEARNING.md (ADR-0026). Drafted by chamin, reviewed and extended here. The gap it fills is real and had gone unnamed: every compressed form in the kit answers a reader's question, and none is a speaking artifact. More structurally, every layer here is written for the operator or for an agent - nodes.md for the gate, LEARNING.md for a reader, topic notes for recall, reports/ for a question - so the brain optimises entirely for its own correctness and not at all for transfer. This is the first artifact written for people who will never open the repo. The finding that shaped the design: the persona inverts a rule the contract enforces everywhere else. A presentation must lead with the takeaway and then earn it, and AGENTS.md forbids exactly that after the TL;DR, calling the punchline opening "the anti-pattern to watch for". Both are right - a ramp orders for someone who does not yet know, a presentation orders for someone who may leave after five minutes - and that is precisely why the section is appended rather than folded into the walkthrough, stated in the contract so a future agent does not read the two rules as a contradiction and pick one. Four improvements to the draft, each closing a failure this kit has already had in another form. First and largest: the null close is mandatory and first-class. The most common honest conclusion in this brain is that nobody measured the thing, and a persona told to end on what should be believed, funded or changed will manufacture actionability under pressure, in front of the audience most likely to act on it - the genre's default and this idea's biggest risk. "The decision is to not act yet, and here is precisely what would change that" is the stronger leadership close, because it converts "we do not know" into a scoped experiment with a trigger; the four verdicts are adopt, pilot, watch, reject. Second: movements reuse the curated frames by default. A visual per movement times five to seven movements would take a note from three diagrams to ten, and this kit already records that keeping three distinct is hard and that a diagram invented to satisfy a table row is decoration - so a new synthesized diagram is justified only where a movement's consequence has no existing visual, which also keeps the frame-prune rule's arithmetic intact. A frame earning its place twice was well chosen. Third: a narrow register carve-out - movement bullets are exempt from the ban on bold lead-in labels, colon-led lists and semicolon chains, because that is what bullets are, and the presenter explanations underneath them are not exempt, because they are prose carrying an argument. Fourth: a 700-1,200 word budget, since a presentation that is not compressed is not a presentation. Scoped to new ingests (from 260815) with validate.py reporting coverage rather than failing - twenty-six notes predate this and making it retroactive would create twenty-six units of debt on day one, which is what happened with the register retrofit and is still unpaid on S2. The counter joins the same informational block as /verify coverage, for the same reason: whether an old note earns a presentation is judgement, and a threshold would launder judgement as a green check (ADR-0004). It already reads 0/2, since S25 and S26 landed today. Two costs recorded rather than discovered. It is the first mandatory LEARNING.md element with no checkable property at all - the validator can confirm a heading exists, and whether a story lands is taste - which is a deliberate exception to the form-versus-judgement line. And /verify's six checks were not designed for it; checks 1 to 4 extend naturally since the section carries node IDs and evidence labels, but the /verify contract does not say so and this was left to be revisited once a few exist rather than guessed at now. No reference implementation yet, and the risk is named in the contract: AGENTS.md records that the LEARNING.md shape was specified wrongly twice before a worked example forced a rewrite. The next ingest is the test, and if the shape is wrong the contract is what changes. Persona registry, routing table, required-shape frame, section spec, sources/_TEMPLATE/LEARNING.md skeleton and BUILD.md all updated in the same pass. |
| 2026-08-15 | brain (kit) | Diagrams for a visual reader: movement diagrams, gap-fill diagrams, a banned narration pattern, and a strengthened TL;DR diagram. Requested by chamin, who named the actual requirement in four words - I'm a visual learner - which reframed the change from a diagram count into a comprehension target. The contract bug is the part worth recording, because it was mine and it was live. AGENTS.md's hard diagram rule demanded "Orientation - how to read it: direction of flow", and the presenter persona added hours earlier bans exactly that ("never say left to right, top to bottom"). Both were in force simultaneously, and the older rule is what produced the filler. Measured before fixing: 31 instances of arrow-narration across 12 LEARNING.md files, so this is systemic rather than occasional. Orientation is now one clause, and only where the notation is not self-evident - a legend where colour carries meaning, nothing else - and every diagram walkthrough in the kit now adopts presenter, which is a much wider remit than the persona had when it was written for the presentation section alone. The request was per-section and the implementation is per-movement, and the reasoning is recorded because the request was reasonable. On S26, per-section would have added ten diagrams of which eight duplicated a curated frame already teaching that step, for roughly +1,500 words - the walkthrough is contractually visual-led, so 8 of its 10 sections already had a visual. And a per-section diagram draws each step in isolation, where the connective tissue between steps is the part that is hard to hold. So: one diagram per movement, placed at the head of the movement inside the walkthrough rather than stacked in the roadmap - four abstractions before the reader has any content is worse than none, and at the head of a movement it works as an advance organiser. Plus a gap-fill diagram wherever a numbered section has no visual at all, which is the rule that matters most on a transcript-only source, where the visual leg was skipped and the entire walkthrough is unillustrated - the case the frame-led rule never covered. The distinction that justifies the whole change: a frame is evidence and a diagram is a model. A frame says here is the artifact; a diagram says here is how the parts relate. The kit was guaranteeing only the first. A diagram set in The 1-minute version was considered and rejected - that section's job is the argument in sixty seconds of scanning, and a set of diagrams makes it slower to scan than the prose it compresses, while its thesis-shape is already drawn by the TL;DR diagram. That diagram absorbed the requirement instead and now carries a sharper bar: a reader who looks at it and reads nothing else should be able to state the note's argument, not merely its topic. With five diagram types now in play the anti-collapse problem is worse, so the division of labour is an explicit table - thesis, structure, one movement's mechanism, one section's content, the subject itself - each with a must not column. ### Takeaway message renamed ### Key takeaway message. S26 rebuilt as the reference implementation: four movement diagrams, one gap-fill in the only section that still lacked a visual, its TL;DR walkthrough stripped of the banned opening, 3 mermaid diagrams to 8, full visual coverage, no numbered section left without something to look at. Cost recorded honestly: it is now 9,367 words, above the 5,000-9,000 the contract describes as typical, and roughly 150 words is the true all-in price of a diagram once its mandatory walkthrough is counted. The 31 arrow-narration instances in the other notes are not fixed - each needs a judgement replacement rather than a substitution, so they are a retrofit backlog and not a sweep. |
| 2026-08-15 | brain (kit) | The presentation tone rewritten from slide notes into a talk track, after chamin showed a reference artifact and said the existing one did not sound like someone presenting. He was right, and the diagnosis is one thing above all others: bullets. A bulleted slide reads as notes the presenter has not yet turned into sentences; flowing paragraphs read as somebody speaking. The persona now bans bullets outright in the presentation, which is the single largest tonal lever and turned out to simplify the contract rather than complicate it - the register carve-out added earlier the same day is withdrawn, because it existed only to exempt bullets and there are none left, and the Register rules' actual demands (complete sentences, paragraphs that hand off) are exactly what a spoken track needs anyway. Six further tone rules, each lifted from the reference and named so they are reproducible: slide titles are claims rather than labels ("The problem was contract fragmentation, not connectivity", not "The problem"), so a reader scanning only the titles receives the argument; one bolded declarative sentence opens each slide and then continues in ordinary prose in the same paragraph, which is a claim and not the lead-in label the register forbids; the audience is named out loud where the register shifts ("the leadership significance is...", "what engineers should take from this is..."), which is what actually lets one narrative serve a mixed room instead of neither half of it; pose and answer ("the question is therefore not whether X, it is whether Y"), because an audience that feels the wrong question being discarded understands the answer differently from one handed it; first person plural for what was done and third for what was found, keeping a decision grammatically distinct from a claim; and citations riding in brackets at the end of a sentence rather than interrupting the cadence. Diagram explanations now open by naming what kind of diagram it is and what it is not - "this is an ownership diagram, not a component diagram" - which does the orientation work that the banned arrow-narration was doing badly. Renamed Movement to Slide throughout the presentation, which fixes a collision this kit created hours earlier: movement already names the roadmap's groups of walkthrough sections, and using it for two things in one file was mine to fix. S26 rewritten end to end in the new register - six slides, an audience-facing framing note, zero bullets. The budget was widened from 700-1,200 to 900-1,500 and the reasoning matters more than the number: prose-only is intrinsically less dense than bullets, AGENTS.md already accepts that the Register rules cost +25 to +30% words for prose carrying an argument, and 1,200 x 1.3 is 1,560 against a first draft that landed at 1,557 before trimming to 1,501. Widening for a change of format is legitimate where widening because one draft ran long is not - the distinction is recorded, because the same file refused to widen for exactly that reason earlier today. |
| 2026-08-15 | brain (kit) | Propagation pass: the presentation tone rules reached only the persona and half of AGENTS.md, and chamin asked whether they had actually landed everywhere. They had not, and the audit found four stale files. sources/_TEMPLATE/LEARNING.md was the worst - its presentation block still said ### Movement 1, still carried the 700-1,200 budget, and still told the next agent that movement bullets are exempt from the Register rules, which is the precise instruction the tone rewrite deleted. A template is read by every future ingest and nothing else, so a stale template does not degrade gracefully - it silently reinstates the rule you just removed. Rewritten in full with all six tone rules stated inline rather than deferred to the persona, because an agent writing from the template will not open the persona first. brain/decisions/0026-presenter-persona.md stated a register carve-out that no longer exists. Handled by amendment rather than rewrite, since an ADR is a record and editing its Decision section to match today's contract destroys the thing an ADR is for - the note records what changed, why, and that the rest stands. It also corrects two of its own Consequences: the word budget, and the line saying no reference implementation exists, which is now false. That correction is the ADR's own prediction arriving early - it warned the LEARNING.md shape had been specified wrongly twice before a worked example forced a rewrite, and this shape lasted hours. The most useful fix is in the global diagram rule, which every diagram in the kit obeys and not only presentation ones. The banned arrow-narration needed a replacement rather than a prohibition, and the reference artifact supplied it: name what kind of diagram it is, and what it is not - "this is an ownership diagram, not a component diagram". One clause, and it does the orientation work properly, because it tells the reader what question the picture answers and pre-empts the wrong reading, which describing arrow direction never can. The rule now carries the measured count too (31 instances across 12 notes) so a future agent knows it was systemic rather than occasional. Also reconciled who writes a diagram walkthrough - mentor decides what the reader needs, presenter decides how it is said - which had been left ambiguous when presenter's remit widened, and updated personas/README.md, where presenter's row still described it as owning only the presentation section when it now owns every diagram walkthrough in the kit. BUILD.md regenerated. The lesson worth keeping: a contract change is not done when the rule is written, it is done when every file that teaches the rule agrees - and the template is the one that matters most, because it is the only one the next ingest is guaranteed to read. |
| 2026-08-15 | brain (kit) | AGENTS.md refactored: the four stage specs leave the root contract and a reserved-terms registry enters it (ADR-0027). Prompted by chamin asking whether a 1,400-line contract should be refactored. Measured before deciding: 1,408 lines, 17,164 words, ~23,000 tokens, every one of them in every session's context window - against four stage specs needed once a fortnight between them. The case rests on failures rather than on tidiness, and four of them happened in a single session that day. The word movement was given two meanings in one document, the roadmap's groups of walkthrough sections and the units of a presentation, and nobody noticed until a reader said the presentation read wrong. Two contradictory rules ran live for hours - the diagram rule demanding "direction of flow" against a new persona banning exactly that. sources/_TEMPLATE/LEARNING.md went stale and was caught only because the human asked. A register carve-out was written and voided within hours. A fifth surfaced while implementing this: personas/presenter.md had never been added to BUILD.md's embed list, so a clone built from the bundle would carry a contract citing a persona that does not exist. None of these is a contradiction git could catch - they are a specification that has outgrown one reader's working memory, including the agent's, which is the failure the file already recorded about itself in the multi-writer section. And the brain had measured the answer without applying it: X2 found discriminability decaying log-linearly with the volume attended over, and claim 209 - promoted from S26 the same week - says to scope behaviour with a schema file placed in the thing being managed, discovered when needed, overriding the generic instruction. The kit adopted that as a finding while running the opposite design. Result: 1,408 to 1,089 lines, ~23k to ~17k tokens, a 25% cut, with the specs in harness-neutral stages/ rather than .claude/commands/ because Copilot, Codex and Cursor read the repo and not Claude Code's command folder. The test that made this safe rather than merely tidy is whether extracted content has a guaranteed loader, because a rule in a file nobody loads is worse than a rule in a long file - each stage has one, since the command wrapper fires and reads its spec. That same test is why the 383-line LEARNING.md shape stayed, despite being the largest single block: it is needed on the most common operation, and its only guaranteed loader would be the template that had gone stale hours earlier. The registry is the part that attacks the cause rather than the instance - twelve reserved words each meaning exactly one thing, with a Not column, which is claim 207 applied to the contract itself. validate.py gains exactly one check: every stages/*.md must be linked from AGENTS.md, because an orphaned-but-correct spec is the single genuinely new failure a split introduces. The terms registry deliberately gets no check - whether a word is being used in two senses is a reading judgement, and encoding it would launder judgement as a green check. Recorded as reversible and provisional: this is scaffolding, claim 31 applies, and the revisit condition is whether the collision rate actually falls over the next few contract changes. If it does not, this was tidying rather than a fix, and the honest move is to say so. |
| 2026-08-16 | brain (kit) | A Provisional rules register in AGENTS.md, because the contract was not carrying its own state. chamin asked whether he would have to tell each new session by hand which rules were new and untested - and if so, whether that belonged in the contract. He was right on both counts, and the second question is the better one. The previous evening ended with a suggested paste-in note for the next session listing three untested rules; a human relaying contract state by hand is a job the document should be doing, and a rule written yesterday and a rule that has survived twenty sources look identical in this file when they should not. Five entries, each with an instance count, a graduation condition and a pointer: the ## Presentation narrative shape (1 instance, and it was rewritten within hours of being specified), movement and gap-fill diagrams (1 instance, and on a slide-heavy source where 8 of 10 sections already had a frame - the untested path is a transcript-only source, where every section triggers gap-fill), the 900-1,500 word budget (one measurement, widened from 700-1,200 on a single draft), the stages/ extraction (zero stage runs since the move), and the reserved-terms registry, which is preventative and graduates only when the next few contract changes produce no new collisions. The rule attached to the table is the point of it: a provisional rule has no seniority over your judgement, and if one feels wrong while you are using it, that is the signal it was put there for - fix the contract, not the artifact. Graduating a rule means deleting its row rather than marking it done, and validate.py now reports the count in the coverage block, so a list nobody prunes shows up as a number that stops falling - the same reason /verify coverage is printed rather than enforced. Also corrected a note that had gone false within a day: the Presentation section still said no reference implementation yet, when S26 had one and writing it had already exposed a word-budget failure and a naming collision. That sentence is now the honest version - one instance is not a settled shape - which is precisely why the rule is in the new register. |
| 2026-08-16 | 260816_scaling-github-for-agents | S27 - GitHub's own MCP server at operator scale, and the source that finally moved mcp.md to established. Sam Morrow (leads GitHub's MCP server), AI Engineer Europe, ~April 2026. Probe reported RICH (16 distinct), 35 transcript-anchored frames triaged in 4 contact sheets, pruned to 17 cited; gated 23 nodes (15 corroborated, 8 single-leg) plus 2 divergences. Two dense slides were unreadable at native resolution and cropping changed a number - the LangChain domain-confusion figure reads "60%+" on a contact sheet and 50%+ at full resolution, which is now a rule worth remembering: zoom before quoting a figure off a slide. The talk's own finding is a governance one wearing a context-window costume. Open contribution filled the server to 101 tools and agents got measurably worse at using GitHub; the team built three correct opt-in fixes - grouped toolsets, dynamic discovery, an unreleased semantic tool-search prototype - and everyone used the default settings (claim 217). Only the changes needing no user action landed. The payload is that authorization turned out to be the filter they had spent months trying to build: a token's scopes are a free, already-correct, zero-configuration filter on the tool surface, and step-up auth converts a permission failure into an interactive grant so the call continues (claim 221). Two corrections to this brain's own framing. Claim 218: response payloads dominate tool definitions - one list_pull_requests(100) call cost 657,272 tokens against a 64.6k catalog, and it exceeds even S10's 541k-token 1,180-tool manifest, so claim 85's 36x tool-search win is a win on the smaller number. Claim 219 joins the two catalogs into a ~450-650 tokens per tool estimation rule. mcp.md advances emerging -> established (ADR-0028), and not on the count, which ADR-0022 already forbade. S27 corroborates claim 180 (statelessness is relocation) from an independent implementation at a different company - stateless per request, no session affinity, Redis still in the architecture diagram - in a system built before the spec change S23 documents. The lesson about which source to hunt is the durable part: this note spent three sources trying to corroborate a specification by reading more about the specification, and what moved it was somebody building the thing and reporting what stayed in their diagram. Claims 217-226 added, claim 180 amended; 9 glossary terms; rows into mcp, context-engineering, agents, agent-security, evals. ⚠️ T2 vendor talk throughout - no external evaluation, no baseline against any other server, the most interesting claim (scope filtering) is the least quantified, and the eval section shows a method with no score quoted anywhere. The strongest caveat is the author's own: he expects thousands of tools to become normal and to "probably reverse many of the fewer tools decisions", so the mechanics are durable and the central recommendation carries a published expiry date. |
| 2026-08-16 | brain (kit) | The 1-minute version gets a diagram, reversing a decision taken the day before (ADR-0029). chamin asked whether we had discussed putting a mental diagram there. We had not in that session - but the contract had, on 2026-08-15, and had recorded a rejection inline: that section's job is the argument in sixty seconds of scanning, a set of diagrams makes it slower to scan, and its thesis-shape is already drawn by the TL;DR diagram. The concern was put to him explicitly and he reaffirmed, so the rule changed rather than the note quietly deviating. One source later that reasoning is half wrong, and the wrong half matters. Reason one priced a set of diagrams; one vertical diagram is the fastest element in a section that now runs ~900 words on S27, not the slowest. Reason two does not survive at all: the TL;DR diagram draws the note's thesis, where the argument lands, and nothing in the note drew the arc of how it got there - roadmap is reading order, movements are local mechanism, mental model is the subject. A reader stopping at The 1-minute version left with the claim and not the reasoning, and on a 12,000-word note that reader is the common case. The rule: one diagram between narrative and table, drawing the standard row sequence as a vertical flow, hard from 2026-08-16, scoped to new ingests on the ADR-0026 precedent. The anti-collapse table gains a fifth row and a mechanical test - components means you drew the mental model, the conclusion alone means you drew the TL;DR diagram, section numbers mean you drew the roadmap. Applied to S27, whose 1-minute diagram renders 586x1430 (0.67 at 390px). The honest expectation is recorded in the ADR: this is the rule most likely to produce a duplicate, and if the next two sources restate the TL;DR diagram it should be deleted rather than patched. Provisional rules now 6. AGENTS.md, sources/_TEMPLATE/LEARNING.md and BUILD.md updated in the same pass. |