Public sample audit: the agent instruction files in getsentry/sentry

What this is. A full Agent Memory Audit run on a public repository as a rehearsal of the paid product. Nobody commissioned it; Sentry was not contacted. It shows exactly what a customer receives: findings with a receipt (file and line) for each, quick wins, and a restructure plan.

Who wrote it. Astrid wrote this audit for Less Hands. Before publication a second, separate review pass against a nine-point checklist re-read every quoted receipt at its file and line; it made four corrections, all listed at the end of this page. For a paid audit, a person at Less Hands reads the report before it leaves.

Checked by a second, unrelated model. On 30 September 2026 every finding was sent cold, one at a time, to an OpenAI model (gpt-5.6-terra-pro via OpenRouter) with only the finding, its receipts and the cited files at the pinned commit, and the instruction to disprove it. Result: 1 of 12 refuted and withdrawn (F7, below), 10 narrowed or downgraded, 1 confirmed. The page now shows 11 findings; the withdrawn one is kept as a note so the count is honest.

Estate. https://github.com/getsentry/sentry at commit 0f24ade94f8d (read-only clone, 26 September 2026): 15 agent-guidance files out of 21,192 tracked; one symlink recorded, not followed. The repository has moved on since; every receipt below is as of that commit, and any of them may have been fixed by the time you read this.

Summary

The agent guidance for this repo is small and in good shape. There is one declared source of truth (AGENTS.md:3). Each area (src/, static/, tests/) has its own file, and the CLAUDE.md files are one-line @AGENTS.md imports rather than copies. Every command checked against package.json exists (typecheck, lint:js, fix, test-ci, dev, dev-ui). The tech stack claims in static/AGENTS.md match the dependencies (React 19, React Router 6, Rspack, Jest, Less). Several rules name the lint or runtime check that enforces them, such as S015, S020, MissingTTL and the mixed-PR CI check. The WRONG/RIGHT examples are concrete, and the security and customer-data rules are clear.

The three things that matter most:

1. Four instructions point at files that don't exist at this commit. The worst one tells an agent to register an integration in a registry.py that isn't there (F1). Three more point at moved or wrong paths (F4). One image-location rule contradicts tsconfig.json (F3).
2. Claude Code probably never loads the strictest rules in the repo. src/sentry/issues/derived/AGENTS.md ("Deterministic only") is the only area file with no CLAUDE.md shim beside it (F2).
3. Some guidance is reachable only through skills behind a symlink this audit didn't follow, and the shared permission allow-list doesn't cover several command forms the guidance tells agents to use (F5, F6; both low after the 30 Sep pass).

Verdict: a well-designed estate with no mechanism in the guidance itself to catch drift. A handful of stale paths have already crept in, and all of them can be fixed this week.

Findings

F1 · Convention drift · medium (was high; downgraded 30 Sep: the runtime consequence is inferred, not shown)

What: The integration how-to tells agents to register new integrations in a file that doesn't exist. Receipt: src/AGENTS.md:171: "Register it in src/sentry/integrations/registry.py and add a feature flag in temporary.py." The mechanical pass found no tracked file at that path. Its nearest matches are in unrelated domains (src/sentry/investigations/templates/registry.py, src/sentry/notifications/notification_action/registry.py, src/sentry/notifications/platform/registry.py). Why it matters for an agent: This is an action step, not background. An agent following it has no file to edit; the likely outcomes are a new registry.py that nothing imports, or an edit to one of the similarly named notification registries. This audit did not establish where integrations are actually registered at this commit, so the consequence is inferred from the missing path, not observed. Fix: In src/AGENTS.md:171, find where integrations are actually registered at this commit and name that file. If there's no single place, describe the real mechanism instead of naming a file.

F2 · Absence-read-as-fact exposure · low (was medium; narrowed 30 Sep)

What: The derived-pipeline rules have no CLAUDE.md pointer beside them, unlike every other area file. Whether Claude Code therefore misses them depends on the Claude Code version and settings in use, which this audit did not have; the gap in discovery is a candidate, not a shown failure. Receipt: Pointer files exist only at CLAUDE.md, src/CLAUDE.md, static/CLAUDE.md and tests/CLAUDE.md. Nothing sits beside src/sentry/issues/derived/AGENTS.md, whose line 15 reads "- Deterministic only: no I/O, randomness, clocks, or env." Neither the root map (AGENTS.md:98-103) nor src/AGENTS.md mentions this directory. Why it matters for an agent: These are the strictest rules in the estate. Breaking them (for example, reading a clock inside an aggregator) makes Group regeneration non-deterministic, and that won't show up in a single test run. Codex and Cursor read nested AGENTS.md files, so they're fine. Claude Code reads CLAUDE.md. Candidate only, because the Claude Code version and settings in use aren't known. Fix: Add src/sentry/issues/derived/CLAUDE.md containing @AGENTS.md. Add one line to src/AGENTS.md pointing at the directory.

F3 · Convention drift · medium

What: The image rule names a directory that the build's image alias doesn't point to. Receipt: static/AGENTS.md:97: "- Images: import via the sentry-images alias (webpack loader); keep them in static/app/images, never reference by static path." In tsconfig.json: "sentry-images/*": ["./static/images/*"],. The mechanical pass found no tracked file under static/app/images/ (the build's own rspack.config.ts was not read, so the alias is taken from tsconfig.json). The line says "webpack loader", but the build is Rspack (static/AGENTS.md:9, package.json "build": "... rspack --config ./rspack.config.ts"). Why it matters for an agent: An agent adding an image will create static/app/images/, and the sentry-images/... import won't resolve. It may then "fix" the problem with a relative path, which is exactly what the rule forbids. Fix: Change static/AGENTS.md:97 to static/images and drop "webpack". The sentry/icons → static/app/icons line above it is correct ("sentry/*": ["./static/app/*"]).

F4 · Convention drift · medium

What: Three more file references are stale. One of them sits beside a related module-symbol reference in the same file. Receipt: - src/AGENTS.md:101: "Check model's silo in src/sentry/models/outbox.py; …". No such tracked file. The mechanical pass suggests src/sentry/hybridcloud/models/outbox.py. - static/AGENTS.md:32: "- Routes defined in static/app/routes.tsx". Not tracked. static/app/router/routes.tsx exists. - tests/AGENTS.md:125: "- Factories: tests/sentry/testutils/factories.py". Not tracked. src/sentry/testutils/factories.py exists, and tests/AGENTS.md:74 names the module symbol sentry.testutils.factories.Factories (a symbol, not a path; the two lines are related, not contradictory).

Why it matters for an agent: Each one costs a failed read at best. At worst the agent creates a new file at the stated path. That risk is real for tests/, where the file's own rule says to mirror src/. Fix: Correct the three paths. For src/AGENTS.md:101, also check whether "check model's silo in outbox.py" is still the right instruction. Silo placement is normally read from the model's own decorator.

F5 · Absence-read-as-fact exposure · low (was medium; narrowed 30 Sep: the skills behind the symlink were not read, so how much rule content lives there is not shown)

What: Much of the actual rule content has moved into skills. The area files keep only a one-line pointer. Readers that don't load skills will treat the pointer as the whole rule. Receipt: AGENTS.md:105: "Workflow steering (commit, pre-commit, hybrid cloud, etc.) lives in skills (.agents/skills/)." That tree is reached through the symlink .claude/skills -> ../.agents/skills, which this audit recorded but didn't follow. The area files name 16 skills. An example is src/AGENTS.md:93: "Logging (logger.info/exception, LOG005/LOG011), tracing/spans (sentry.utils.tracing), metrics tag cardinality, and the options system (options.get()) → use the backend-conventions skill." AGENTS.md:36 calls setup-dev a "skill", but the estate has it as a command (.claude/commands/setup-dev.md). .claude/settings.json:83-89 allows a third, separate namespace (Skill(sentry-skills:…)). Why it matters for an agent: A Cursor or Codex agent, or any agent where skills fail to load, sees "logging → skill" and nothing more. It then writes logging code with no constraints. The intake lists all three agent families as in use. Fix: At each pointer, keep a one-line summary of the rule that must never be broken, followed by "details: skill X". Make AGENTS.md:105 state what to do when skills are unavailable (read .agents/skills/<name>/ directly). Resolve the setup-dev naming: either it's a skill or it's a command.

F6 · Convention drift · low (was medium; narrowed 30 Sep)

What: The team-shared permission allow-list doesn't cover several command forms AGENTS.md tells agents to use. Receipt: - AGENTS.md:11: "Python commands (pytest, mypy, prek, etc.) MUST run in the virtualenv". AGENTS.md:44 says to run .venv/bin/prek run -q. Against that, .claude/settings.json:53 allows "Bash(pytest:*)", :38 allows "Bash(mypy:*)" and :51 allows "Bash(prek run:*)", with no .venv/bin/ entries. - AGENTS.md:84 and :86 say pnpm run lint:js and pnpm run fix. Against that, .claude/settings.json:43 allows "Bash(pnpm lint:*)" and :40 allows "Bash(pnpm fix:*)". - Typecheck is covered both ways (:46, :49), so the gaps look accidental.

Why it matters for an agent: The documented form may prompt while a near-identical form is auto-approved, and agents drift toward whichever form doesn't prompt. The bare-Python entries are a candidate only: AGENTS.md:15 allows bare pytest in an activated shell and AGENTS.md:18 tells agents to request full permissions for Python commands, so the gap may be harmless there. The pnpm run mismatch stands. Fix: In .claude/settings.json, add Bash(.venv/bin/pytest:*), Bash(.venv/bin/prek run:*), Bash(pnpm run lint:*) and Bash(pnpm run fix:*). Then decide whether the bare Python entries should stay.

F7 · withdrawn 30 September 2026

The original F7 claimed copy drift between AGENTS.md, .claude/commands/setup-dev.md and .claude/commands/gh-pr.md (setup and PR-splitting guidance duplicated and diverging). The independent refutation pass showed the instructions are context-specific and consistent: AGENTS.md:25-36 recommends the frontend-only sync for the pytest case and plain devenv sync for a working dev server, which is what the command file prescribes (.claude/commands/setup-dev.md:47-53); the PR rules at AGENTS.md:125-129 and .claude/commands/gh-pr.md:16 say the same thing; and a Skill(sentry-skills:create-pr) permission entry is not evidence of a competing workflow. Withdrawn in full.

F8 · Naming ambiguity · low (candidate; narrowed 30 Sep)

What: The silo guidance uses "Region Silo" alongside cell_* decorators and a cell-architecture skill without saying how the two relate. Receipt: src/AGENTS.md:31: " - Region silo: RegionSiloEndpoint". src/AGENTS.md:100: "- Region Silo: Project data, events, issues". Then src/AGENTS.md:101 says "use @cell_silo_endpoint", and :102 names the cell-architecture skill. Why it matters for an agent: If the two names mean one thing, an agent can't tell which is current; if they mean different layers, the guide never says so. Candidate: symbol existence wasn't checked (see coverage), and nothing in the files says the terms are synonyms. Fix: Add one line at src/AGENTS.md:99-101 saying how "Region Silo" and the cell_* names relate, and which one new code should use.

F9 · Presence-read-as-relevance · low

What: An instruction that reads as tool-specific is written as if it applies to every agent. Receipt: AGENTS.md:18: "For AI agents: always use required_permissions: ['all'] for Python commands to avoid sandbox permission issues." This appears to be a parameter of one tool's command interface (candidate: this audit did not confirm which tool, nor whether every agent in use lacks it). Why it matters for an agent: Other agents will either invent a parameter or read the line as general permission to escalate sandbox permissions for Python commands. Fix: Prefix it with the tool it applies to ("Cursor only: …"), or move it into that tool's own config.

F10 · Trust pricing · low

What: No agent file carries an owner or last-reviewed marker. Readers also can't tell which rules are machine-enforced and which rely on judgement. Receipt: The mechanical pass lists all nine agent-guidance markdown files (five AGENTS.md, four CLAUDE.md) as having no owner or last-reviewed header (the word "owner" appears in prose, never as a header), including AGENTS.md, src/AGENTS.md, static/AGENTS.md and tests/AGENTS.md. Some rules state their enforcer: tests/AGENTS.md:51 ("Flake8 S015 flags …") and src/AGENTS.md:113 ("raise MissingTTL"). Others sit beside them with the same weight and no signal, such as src/AGENTS.md:108 ("Add indexes for queries on 1M+ row tables"). Why it matters for an agent: An agent can't weigh an instruction before acting on it. Enforced rules can be relied on. Unenforced ones need checking, and nothing marks which is which. Fix: Add a one-line header to each AGENTS.md (owning team, last reviewed). Tag rules with their enforcer where one exists, for example (enforced: S015) or (enforced: CI).

F11 · Unowned freshness · low

What: Several claims describe a moment in time, and nothing in these files names what keeps them current. Receipt: - static/AGENTS.md:38 bans "the deprecated useApiQuery/getApiQueryData/setApiQueryData". No tracking issue or end date is given. - static/AGENTS.md:11 lists "State Management: Reflux, React Query (TanStack Query)". package.json also ships "mobx": "6.13.7" and "mobx-react-lite": "4.1.0", which aren't mentioned (candidate: they may be approved only for narrow uses). - .claude/commands/setup-dev.md:24 pipes an installer from .../devenv/main/install-devenv.sh with no pinned version. - .github/CONTRIBUTING.md:11 points to https://docs.sentry.io/internal/, while the agent files point to develop.sentry.dev (src/AGENTS.md:21, src/AGENTS.md:115, AGENTS.md:117).

Why it matters for an agent: Each of these can go quietly wrong as the codebase moves, and nothing in the files tells an agent when to re-check (a tracking issue or policy may exist elsewhere; none is named here). Fix: Link the deprecation to its tracking issue. Add one line on where MobX is and isn't allowed. Make the two doc hosts agree. These are covered by the owner/review header in F10.

F12 · Silent-clobber surface · low

What: The root file directs agents to edit the shared guidance and does not itself name an owner or a review requirement for those edits. Receipt: AGENTS.md:3: "Always update the relevant AGENTS.md file when adding or modifying agent guidance." AGENTS.md:105: "Add or update guidance in the appropriate AGENTS.md or skill". Why it matters for an agent: Nothing in the guidance itself distinguishes an agent's edit to a shared rule from a reviewed one. Whether the repository's review controls (CODEOWNERS exists in the manifest but was not read) already cover this is not shown here. Fix: Put the AGENTS.md files and .agents/skills/ under a named reviewer (for example in CODEOWNERS). State in AGENTS.md:3 that agent edits to these files need human review.

Dropped as false positives: .artifacts/dev.log (gitignored runtime output, as AGENTS.md:40 says), logger.info/exception (not a path), sentry/icons (valid tsconfig.json alias), and src/sentry/foo/bar.py / tests/sentry/foo/test_bar.py (illustrative). The one near-copy signal (the "Command Execution Guide" pointer at the top of src/AGENTS.md and static/AGENTS.md) is a deliberate cross-reference to a section that exists (AGENTS.md:5), not drift.

Quick wins (this week; each under an hour)

1. Fix the five bad references: src/AGENTS.md:101 and :171, static/AGENTS.md:32 and :97, and tests/AGENTS.md:125.
2. Add src/sentry/issues/derived/CLAUDE.md containing @AGENTS.md.
3. Add the four missing allow-list entries to .claude/settings.json (F6).
4. Mark AGENTS.md:18 as specific to one tool.
5. Decide "region" vs "cell" in src/AGENTS.md:31 and :99-101.

Restructure plan

The estate is 15 files and about 35 KB, and its shape is already right. The plan is about keeping it true, not reorganising it.

• Phase 1: catch drift automatically (weeks 1–2). Add a small CI check over **/AGENTS.md and .claude/commands/*.md. It should:
• extract path-like backticked references and fail if they're missing,
• skip {placeholder} templates and an explicit allow-list (foo/bar, gitignored outputs, tsconfig.json aliases),
• fail if any directory holding an AGENTS.md has no CLAUDE.md shim.

This check would have caught F1, F3, F4 and F2. - Phase 2: price trust (weeks 2–4). Add owner and last-reviewed headers to each AGENTS.md. Put CODEOWNERS entries on AGENTS.md, .agents/skills/ and .claude/. Tag rules that have an enforcer. - Phase 3: one home per workflow (month 2). Decide which of .agents/skills/, .claude/commands/ and the sentry-skills plugin owns setup and PR creation, and make the others thin pointers, so the consistency found on 30 Sep is kept by structure rather than by care. At every place an area file hands off to a skill, keep a one-line summary of the rule that must never be broken, so agents without skills still see the core rule.

What this audit did not look at

• Skills: the contents of .agents/skills/ (behind the unfollowed symlink) and the sentry-skills plugin. Much of the real guidance lives there (F5), and none of it was assessed.
• Python lint config: the Makefile targets the guidance names (make reset-db, make build-api-docs, make test-selective) were checked and exist. The flake8 and prek configuration was not read, so the lint codes (S015, S020, LOG005, LOG011) are taken as stated, not verified.
• Other referenced files: .github/workflows/backend.yml (cited at AGENTS.md:55), .mcp.json, any Cursor rules, and CODEOWNERS.
• Symbols and links: whether code symbols exist (RegionSiloEndpoint, instrumented_task, MissingTTL, EAPClient), and whether external URLs work. None were fetched.
• Permission posture: this is outside the seven classes and wasn't scored. As an observation, .claude/settings.json:14 auto-allows Bash(find:*), which includes find -delete/-exec.
• Hostile content: I checked all 15 files for untrusted-content hostility and found none. The files contain instructions for agents working in the repo, and nothing addressed to or trying to steer this review.

Review gate record (published with the audit)

Reviewer: the resident gate (Astrid, fresh pass), 26 September 2026, against the nine-point checklist.

1. Receipts: PASS. The mechanical checker passed 12 of 17; the five it failed on were re-read by eye and found exact (the checker had tripped on bold markup, ellipses and quotes from build files, and was fixed afterwards). Every other quoted line was re-read at its file and line.
2. Repo evidence: PASS. static/app/router/routes.tsx, src/sentry/hybridcloud/models/outbox.py and src/sentry/testutils/factories.py are in the manifest; no src/sentry/integrations/registry.py; tsconfig.json maps sentry-images to static/images; package.json builds with rspack; mobx 6.13.7 is present.
3. False positives dropped: PASS. Fix: the draft misdescribed the near-copy signal; rewritten.
4. Over-claim: FIX. F9 had asserted which tool owns required_permissions; softened to a candidate. An unverified "given how busy this repo is" was removed from F12. Severities judged fair: one high, an action step pointing at a missing file.
5. Tone: PASS; the good is named first.
6. Injection residue: PASS; none found in the estate, none followed.
7. Walls: PASS; nothing about Less Hands systems, other customers, or any private matter.
8. Proportion: PASS; three short phases for a 15-file estate.
9. Coverage honesty: FIX. The draft said no Makefile was supplied; the pipeline had landed it and checked the make targets, but had not passed that to the judgment step (pipeline fixed). The coverage bullet was rewritten to the truth.

Verdict: PASS.

This page is a sample. To have the same audit run on your CLAUDE.md, AGENTS.md, skills, wiki or RAG index, see the offer. Questions: astrid@lesshands.com.au.

16 October 2026: scored against Sentry's own refresh

On 16 October Sentry merged a refresh of its backend agent docs, written by its own agent and checked against the code (getsentry/sentry#127884). It did not see this audit, and it left frontend docs out of scope. It did cover the backend files this sample read, so it gives an outside score for part of the sample. Here it is, both ways.

• Fixed by Sentry: F1 (no integrations/registry.py; providers register through SENTRY_DEFAULT_INTEGRATIONS), F4 (two paths in #127884, the routes path in #127030) and F8. I graded F8 too softly: I called it a naming ambiguity, but RegionSiloEndpoint did not exist.
• Still present at master 624fb343: F2, F3, F6, F9, F10, F11, F12. F3 is now confirmed against the build config: rsbuild.config.ts:505 maps sentry-images to static/images, not static/app/images.
• Kept: the pointer-to-skill layout F5 questioned. The PR says it "kept the existing layout".
• Missed by this audit, fixed by Sentry: five wrong facts that were checkable at the pinned commit. The prek hook id ruff (the real ids are ruff-check and ruff-format). Celery task guidance (tasks run on taskbroker). Organization management placed in the control silo (Organization is a cell-silo model). "Inherit from Integration" (a database model, not a base class). And an anti-pattern that, by Sentry's count, about 135 of 158 endpoint files break. Of the nine wrong facts the refresh fixed in files this audit read, I had named three fully and one partly.
• My own errors: F5 said setup-dev is a command and not a skill; .agents/skills/setup-dev/ exists at the pin. F10 and F12 said no owner is named; .github/CODEOWNERS:131 already assigns /AGENTS.md to a team. I did not read CODEOWNERS.

Every miss was a symbol or a meaning, which the coverage section above lists as not checked. Path checks did not catch them.