Nine sections of the harness inventory as a ring. Tick a layer to mark it covered; the dot fills, its petal grows, completeness updates. Click a name or dot for the explanation. Empty arcs are exposure.
Heads up: the layer list, the explanations, the risk-tier assignments and the
quality mapping are LLM-generated suggestions. They are a starting point and may not match what your
project actually needs, so treat them as a draft to adapt, not a checklist to obey.
The percentage counts adopted layers with equal weight: it measures the breadth of your harness, not
its strength. A tick says nothing about how well a layer is configured, and overlapping layers (linter,
SAST, type checker) do not add independent protection.
Prototype · self-assessment (one tick = one layer you run) · based on
llm-coding.github.io/Semantic-Anchors/harness-inventory.
Spot something wrong? Open an issue with your change request.
The wheel groups every layer under the 8 ISO/IEC 25010 quality characteristics, split into their sub-characteristics. Each layer names one primary sub-characteristic and, where it genuinely checks more, secondary ones (counted here too). A struck-through chip is a blind spot: no automated test or check in this inventory covers it, and no listed review closes it either. That is the whole point of the harness idea, error correction is layered and some layers can only be a human. Ingo Eichhorst explains the underlying principle in his Leanpub book Der Agent tickt anders. Cover these deliberately, or accept them as out of scope for your risk tier.
Not closed by any automated check here:
Standing on prior work: the idea combines threads that each exist on their own. Quamoco maps concrete measurements to ISO 25010 characteristics; SQALE maps rule violations to quality characteristics; OWASP SAMM shows maturity gaps as a radar (security only); OpenSSF Scorecard is an automated check inventory (supply chain only); and the Swiss cheese model is the underlying theory of layered error correction. This wheel wires them together for agentic coding: one broad layer inventory, full ISO 25010 coverage, explicit blind spots, a risk-tier dial.
What it checks: Catches syntax errors, type mismatches, and other language-rule violations that prevent the code from building.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, compiler errors name the file, line, and cause, so an agent can read them and fix directly.
Why this class: GREEN because the correctness standard is the language specification itself, external to the project, so you just turn it on.
Examples: gcc, rustc, javac, tsc
What it checks: Catches type errors and interface mismatches that a permissive compiler would let through.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each error points to a specific expression and expected type an agent can act on.
Why this class: GREEN because type soundness is defined by the type system, external to the project, so you enable it.
Examples: mypy, tsc, Pyright
What it checks: Catches inconsistent code formatting and whitespace style.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, formatters auto-rewrite the code or emit a diff the agent can apply.
Why this class: GREEN because the formatting standard is a fixed external convention you adopt as-is.
Examples: Prettier, Black, gofmt
What it checks: Catches unsorted or unused imports and unreachable or unused code.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, it flags exact symbols and lines and often fixes them automatically.
Why this class: GREEN because the rule (no unused, sorted order) is a fixed external convention, not project judgment.
Examples: isort, eslint-plugin-unused-imports, goimports
What it checks: Catches likely bugs, anti-patterns, and style violations beyond pure syntax.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each finding has a rule ID and location an agent can address or suppress.
Why this class: AMBER because it ships a default ruleset but teams tune which rules and severities fit the project.
Examples: ESLint, Ruff, golangci-lint
What it checks: Catches incorrect behavior in individual functions or units against expected results.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, a failing assertion reports expected versus actual so an agent can diagnose and fix.
Why this class: RED because the correct behavior is defined by the project's own tests, not any external standard.
Examples: pytest, Jest, JUnit
What it checks: Catches edge-case failures by generating many inputs and checking invariants or crashes.
How: property-based
When: in CI test stage
Closed-loop: Yes, it reports a shrunk counterexample input the agent can reproduce and fix.
Why this class: RED because the properties and invariants asserted are defined by the project itself.
Examples: Hypothesis, fast-check, AFL
What it checks: Catches weak or missing tests by mutating code and checking whether tests notice.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, surviving mutants show which code paths lack test coverage for the agent to strengthen.
Why this class: RED because it measures the project's own test suite against the project's own code.
Examples: Stryker, mutmut, PIT
What it checks: Catches defects in how modules or services interact when wired together.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, failures identify the failing interaction and expected result for the agent to trace.
Why this class: RED because correct cross-component behavior is defined by the project's own expectations.
Examples: pytest, Testcontainers, Supertest
What it checks: Catches mismatches between a consumer's expectations and a provider's actual API.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, a broken contract names the interaction and mismatched field an agent can reconcile.
Why this class: RED because the contract is authored by the project's own consumer and provider.
Examples: Pact, Spring Cloud Contract
What it checks: Catches gaps between implemented behavior and business-readable acceptance criteria.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, a failing scenario step maps to the Given/When/Then the agent can debug.
Why this class: RED because the acceptance criteria come from the project's own requirements.
Examples: Cucumber, Behave, SpecFlow
What it checks: Catches broken user flows across the full running application in a browser.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, failures include the step, selector, and often a trace or screenshot for the agent.
Why this class: RED because the expected user journeys are defined by the project itself.
Examples: Playwright, Cypress, Selenium
What it checks: Catches unintended changes to rendered output or serialized snapshots.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Partial, an agent sees a diff but often needs a human to judge whether the change is intended.
Why this class: RED because the accepted baseline is the project's own approved snapshot.
Examples: Jest snapshots, jest-image-snapshot, Playwright
What it checks: Catches performance regressions where code gets slower or uses more resources than expected.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, it reports measured versus threshold numbers an agent can investigate.
Why this class: RED because the acceptable performance budget is set by the project.
Examples: pytest-benchmark, JMH, k6
What it checks: Catches gross failures where core functionality is broken right after a build or deploy.
How: dynamic/runtime
When: pre-deploy
Closed-loop: Yes, a failed smoke check names the broken critical path for the agent to inspect.
Why this class: RED because the critical paths deemed must-work are chosen by the project.
Examples: pytest, curl, Playwright
What it checks: Catches committed credentials, API keys, and tokens leaking into the repository.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each hit names the file, line, and secret type the agent can remove and rotate.
Why this class: GREEN because what counts as a secret is a fixed external pattern set you enable.
Examples: gitleaks, TruffleHog, detect-secrets
What it checks: Catches known vulnerabilities in third-party dependencies.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, it reports the vulnerable package, CVE, and fixed version for the agent to bump.
Why this class: GREEN because vulnerability data comes from external advisory databases you consume.
Examples: Dependabot, Snyk, Trivy
What it checks: Catches vulnerable OS packages and misconfigurations inside container images.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it lists the affected layer, package, and CVE the agent can patch.
Why this class: GREEN because it checks against external CVE and hardening databases you enable.
Examples: Trivy, Grype, Clair
What it checks: Catches insecure or non-compliant infrastructure-as-code configurations.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each finding names the resource, rule, and remediation an agent can apply.
Why this class: GREEN because the checks encode external security best practices you turn on.
Examples: Checkov, tfsec, Terrascan
What it checks: Catches unverified provenance and undocumented components in the build supply chain.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it reports missing attestations or components the agent can generate or record.
Why this class: GREEN because SBOM and SLSA are external standards and frameworks you adopt.
Examples: Syft, in-toto, cosign
What it checks: Catches configurations that violate policy or compliance benchmarks.
How: static analysis
When: in CI test stage
Closed-loop: Yes, each violation cites the policy rule and offending resource for the agent.
Why this class: GREEN because the benchmarks (CIS, policy packs) are external standards you enforce.
Examples: Open Policy Agent, kube-bench, Conftest
What it checks: Catches dependencies with licenses incompatible with the project's policy.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it names the package and conflicting license so the agent can replace or flag it.
Why this class: GREEN because license terms and compatibility rules are external facts you check against.
Examples: FOSSA, license-checker, Scancode
What it checks: Catches security vulnerabilities in source code such as injection or unsafe calls.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each finding names the sink, rule, and location the agent can fix.
Why this class: AMBER because it ships standard vulnerability rules but teams add and tune project-specific rules.
Examples: Semgrep, CodeQL, Bandit
What it checks: Catches runtime security flaws by attacking the running application from the outside.
How: dynamic/runtime
When: pre-deploy
Closed-loop: Partial, findings describe the vulnerable endpoint but often need human triage to reproduce and fix.
Why this class: AMBER because it uses standard attack rules but scan scope and auth are project-configured.
Examples: OWASP ZAP, Burp Suite, Nikto
What it checks: Catches security vulnerabilities by instrumenting the app and observing real request flows.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Partial, it pinpoints vulnerable code paths but confirmation and fix often need human judgment.
Why this class: AMBER because detection rules are standard but instrumentation and exercised flows are project-specific.
Examples: Contrast, Seeker, HCL AppScan
What it checks: Catches security weaknesses that pattern-based scanners miss by reasoning over the code.
How: LLM review
When: pre-commit / CI
Closed-loop: Yes, the model returns explained findings with locations an agent can act on.
Why this class: AMBER because it applies general security knowledge but is steered by project-specific prompts and context.
Examples: Claude, GPT-based reviewers, custom LLM pipelines
What it checks: Catches design-level security risks by systematically enumerating threats to the system.
How: human review
When: design phase
Closed-loop: No, it is a structured human analysis producing threats rather than a machine signal an agent self-corrects from.
Why this class: RED because the assets, trust boundaries, and threats are specific to the project's own design.
Examples: Microsoft Threat Modeling Tool, OWASP Threat Dragon, IriusRisk
What it checks: Catches overly complex functions likely to be hard to test and maintain.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, it reports the function and its complexity score against a threshold for the agent.
Why this class: GREEN because the metric and thresholds are external, well-defined measures you enable.
Examples: radon, lizard, SonarQube
What it checks: Catches API definition problems such as missing responses or inconsistent naming.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each rule violation points to the spec path the agent can correct.
Why this class: AMBER because it ships default style rules but teams add project-specific API conventions.
Examples: Spectral, Redocly, Zally
What it checks: Catches defects through a formal, structured peer inspection of code or documents.
How: human review
When: pre-merge
Closed-loop: Partial, an agent can implement the written defect findings but cannot run the inspection itself.
Why this class: RED because the correctness being judged is defined by the project's own requirements and standards.
Examples: structured inspection process, review checklists
What it checks: Catches logic errors, design issues, and maintainability problems through peer reading.
How: human review
When: pre-merge
Closed-loop: Partial, an agent can act on the review comments but the review judgment comes from humans.
Why this class: RED because reviewers judge against the project's own conventions and intent.
Examples: GitHub PR review, Gerrit, GitLab MR
What it checks: Catches maintainability, correctness and readability issues by reasoning over a diff or module the way a human reviewer would.
How: LLM review
When: pre-commit / CI
Closed-loop: Yes, the model returns explained findings with file and line an agent can act on.
Why this class: AMBER because it applies general code-quality knowledge but is steered by the project's own conventions and context.
Examples: Claude, GPT-based reviewers, CodeRabbit
What it checks: Catches violations of architectural rules such as forbidden dependencies between layers.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it names the offending dependency and rule so the agent can restructure it.
Why this class: RED because the allowed architecture rules are defined by the project itself.
Examples: ArchUnit, dependency-cruiser, Deptrac
What it checks: Catches code that drifts from decisions recorded in architecture decision records.
How: static analysis
When: in CI test stage
Closed-loop: Yes, a violation cites the ADR and the offending code the agent can align.
Why this class: RED because the decisions being enforced are the project's own ADRs.
Examples: custom linters, ArchUnit, conftest
What it checks: Catches requirements that lack corresponding code or code that implements no requirement, via stable spec IDs.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it reports which requirement IDs are unlinked so the agent can add the missing reference or code.
Why this class: RED because the requirement IDs and their mapping are defined by the project's own spec.
Examples: custom Q-ID linter, traceability matrix scripts
What it checks: Catches architectural risks and tradeoffs by evaluating the design against quality-attribute scenarios.
How: human review
When: design phase
Closed-loop: No, it is a facilitated human evaluation producing risks rather than a signal an agent self-corrects from.
Why this class: RED because the quality goals and scenarios weighed are specific to the project.
Examples: ATAM workshop, quality-attribute scenarios
What it checks: Catches drift between the database schema and the application's ORM model definitions.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it reports the differing tables or columns the agent can migrate or update.
Why this class: RED because the intended schema is defined by the project's own model.
Examples: Alembic autogenerate, Prisma migrate diff, Liquibase
What it checks: Catches design and maintainability issues by having a model reason over the code or design.
How: LLM review
When: pre-merge
Closed-loop: Yes, the model returns explained findings an agent can read and revise.
Why this class: RED because it judges against the project's own design intent supplied in context.
Examples: Claude, GPT-based reviewers, custom LLM pipelines
What it checks: Catches payloads or documents that violate a declared JSON Schema or OpenAPI definition.
How: static analysis
When: in CI test stage
Closed-loop: Yes, validation errors name the failing field and rule the agent can fix.
Why this class: AMBER because the validators are standard but the schemas they enforce are project-defined.
Examples: ajv, openapi-spec-validator, Spectral
What it checks: Catches broken or unsafe database migrations before they run against real data.
How: dynamic/runtime
When: pre-deploy
Closed-loop: Yes, a failed dry-run reports the SQL error the agent can correct.
Why this class: AMBER because the migration engine is standard but the migrations checked are project-specific.
Examples: Flyway, Liquibase, Alembic
What it checks: Catches personal or sensitive data appearing where it should not.
How: static analysis
When: in CI test stage
Closed-loop: Yes, it flags the detected PII and location the agent can redact or move.
Why this class: AMBER because PII detectors ship default recognizers but teams tune what counts as sensitive.
Examples: Amazon Macie, Presidio, Nightfall
What it checks: Catches invalid or missing configuration values before the application uses them.
How: static analysis
When: pre-deploy
Closed-loop: Yes, it reports the invalid key and expected constraint the agent can fix.
Why this class: RED because the valid configuration shape is defined by the project itself.
Examples: Zod, Pydantic, envalid
What it checks: Catches data-quality violations where incoming or stored data breaks expected properties.
How: dynamic/runtime
When: in production
Closed-loop: Yes, a failed expectation names the column and rule the agent can investigate.
Why this class: RED because the expectations describing valid data are authored by the project.
Examples: Great Expectations, Soda, dbt tests
What it checks: Catches automatable accessibility violations such as missing labels or bad ARIA usage.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, each finding cites the element and WCAG rule the agent can remediate.
Why this class: GREEN because the rules encode external WCAG standards you enable.
Examples: axe-core, pa11y, Lighthouse
What it checks: Catches text and UI color combinations with insufficient contrast ratios.
How: static analysis
When: in CI test stage
Closed-loop: Partial, it reports failing ratios but choosing compliant colors may need design judgment.
Why this class: GREEN because the contrast thresholds are fixed external WCAG values.
Examples: axe-core, Lighthouse, Stark
What it checks: Catches rendering or behavior defects that appear only in certain browsers.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, failures identify the browser and broken behavior for the agent to reproduce.
Why this class: GREEN because browser conformance is an external target you test against.
Examples: Playwright, BrowserStack, Sauce Labs
What it checks: Catches inconsistent or off-style wording in user-facing interface text.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each finding names the phrase and rule the agent can rewrite.
Why this class: GREEN because it enforces an external style guide you adopt.
Examples: Vale, textlint, write-good
What it checks: Catches accessibility problems that only surface when a real assistive technology is used.
How: human review
When: pre-release
Closed-loop: No, it requires a human using a screen reader and cannot be read back as a machine signal.
Why this class: AMBER because it targets external accessibility standards but assessment is human and app-specific.
Examples: NVDA, VoiceOver, JAWS
What it checks: Catches internationalization issues like hardcoded strings or missing translation keys.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, it flags the untranslated string or missing key the agent can externalize.
Why this class: AMBER because the rules are standard but the locales and key sets are project-specific.
Examples: i18next-parser, eslint-plugin-i18next, Lingui
What it checks: Catches unintended visual changes by comparing rendered screenshots to a baseline.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Partial, an agent sees the pixel diff but usually needs a human to approve or reject it.
Why this class: RED because the accepted visual baseline is the project's own approved snapshot.
Examples: Percy, Chromatic, Applitools
What it checks: Catches latency and failure points by tracing requests across service boundaries.
How: dynamic/runtime
When: in production
Closed-loop: Yes, a trace pinpoints the slow or failing span an agent can investigate.
Why this class: GREEN because tracing follows external standards like OpenTelemetry you instrument with.
Examples: Jaeger, Zipkin, OpenTelemetry
What it checks: Catches bad releases by exposing them to a small slice of traffic before full rollout.
How: dynamic/runtime
When: pre-deploy
Closed-loop: Yes, metric regressions during the canary give a signal an agent can act on or roll back.
Why this class: AMBER because the delivery mechanism is standard but the success metrics and gates are project-set.
Examples: Argo Rollouts, Flagger, Spinnaker
What it checks: Catches unusual runtime behavior that deviates from learned or expected baselines.
How: dynamic/runtime
When: in production
Closed-loop: Partial, alerts flag anomalies but often need human interpretation to know the root cause.
Why this class: AMBER because the detectors are standard but baselines and sensitivity are tuned per project.
Examples: Prometheus, Datadog, Anodot
What it checks: Catches violations of expected conditions during execution, failing fast when state is wrong.
How: dynamic/runtime
When: in production
Closed-loop: Yes, a triggered assertion names the violated condition and location the agent can fix.
Why this class: RED because the invariants asserted are defined by the project's own logic.
Examples: assert statements, invariant(), contract libraries
What it checks: Catches unhealthy or not-yet-ready service instances so traffic is not routed to them.
How: dynamic/runtime
When: in production
Closed-loop: Yes, a failing probe indicates which check failed for the agent to diagnose.
Why this class: RED because what constitutes healthy and ready is defined by the project's own endpoints.
Examples: Kubernetes probes, Spring Boot Actuator, custom endpoints
What it checks: Catches releases or states that breach service-level objectives and error budgets.
How: dynamic/runtime
When: in production
Closed-loop: Yes, an SLO breach reports the objective and measured value an agent can respond to.
Why this class: RED because the SLO targets and error budgets are set by the project.
Examples: Prometheus, Sloth, Nobl9
What it checks: Catches resilience weaknesses by deliberately injecting failures into the running system.
How: dynamic/runtime
When: in production
Closed-loop: Yes, a failed experiment reveals the weakness and blast radius the agent can harden.
Why this class: RED because the resilience hypotheses and steady-state definitions are the project's own.
Examples: Chaos Monkey, Gremlin, LitmusChaos
What it checks: Catches bad behavior early by toggling functionality off without a redeploy.
How: dynamic/runtime
When: in production
Closed-loop: Yes, flag-driven metrics let an agent decide to disable or adjust a feature.
Why this class: RED because which features are gated and their rules are defined by the project.
Examples: LaunchDarkly, Unleash, Flagsmith
What it checks: Catches bugs by exploring program paths symbolically to find inputs that trigger failures.
How: symbolic
When: in CI test stage
Closed-loop: Partial, it produces failing inputs an agent can use, but setup and interpretation are involved.
Why this class: GREEN because it checks against generic properties like no crashes and safe memory, external to the project.
Examples: KLEE, angr, SymCC
What it checks: Catches whole classes of errors at compile time by encoding invariants in rich types.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, a type error identifies the unmet obligation the agent can satisfy.
Why this class: GREEN because the type system's soundness rules are external to the project.
Examples: Idris, F*, Agda
What it checks: Catches design or implementation defects by mathematically proving properties hold.
How: symbolic
When: design phase
Closed-loop: Yes, a failed proof or counterexample shows the violated property the agent can address.
Why this class: RED because the properties being proven are specified by the project itself.
Examples: TLA+, Dafny, Coq
What it checks: Catches concurrency and state-machine defects by exhaustively exploring reachable states.
How: symbolic
When: design phase
Closed-loop: Yes, a violated property yields a counterexample trace the agent can reason from.
Why this class: RED because the model and temporal properties checked are defined by the project.
Examples: Spin, NuSMV, TLC
What it checks: Catches malformed or inconsistent markup in documentation source files.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each rule violation names the file and line the agent can fix.
Why this class: GREEN because the markup rules are a fixed external convention you adopt.
Examples: markdownlint, Vale, asciidoctor
What it checks: Catches broken or dead links in documentation and content.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, it reports the failing URL and status code the agent can update or remove.
Why this class: GREEN because a link either resolves or not, an external fact independent of the project.
Examples: lychee, markdown-link-check, htmltest
What it checks: Catches documentation code examples that no longer run or produce the stated output.
How: dynamic/runtime
When: in CI test stage
Closed-loop: Yes, a failing example reports expected versus actual output the agent can fix.
Why this class: GREEN because the example is checked against the actual runtime behavior, external to the docs.
Examples: doctest, pytest --doctest, mdBook test
What it checks: Catches spelling mistakes and typos in code comments and documentation.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, it flags the misspelled word and location the agent can correct.
Why this class: GREEN because correct spelling is defined by an external dictionary you enable.
Examples: cspell, codespell, typos
What it checks: Catches broken diagram source that fails to render into an image.
How: static analysis
When: in CI test stage
Closed-loop: Yes, a render error reports the syntax problem the agent can fix.
Why this class: GREEN because valid diagram syntax is defined by the external tool, not the project.
Examples: PlantUML, Mermaid CLI, Graphviz
What it checks: Catches style, tone, and inclusive-language issues in written prose.
How: static analysis
When: pre-commit / CI
Closed-loop: Yes, each finding names the phrase and rule the agent can rewrite.
Why this class: AMBER because it ships default style rules but teams add their own vocabulary and style guide.
Examples: Vale, alex, textlint
What it checks: Catches documentation and code that reference stale or mismatched stable requirement IDs.
How: static analysis
When: in CI test stage
Closed-loop: Yes, the audit reports which IDs are out of sync so the agent can realign doc and code references.
Why this class: RED because the ID scheme and the doc-to-code mapping are defined by the project itself.
Examples: custom Q-ID audit linter, traceability scripts