# Nerlo — Threat Model

Living document. Last reviewed: 2026-06-04.

> **For external readers**: this is the same threat model we use
> internally. We publish it because we ask MCP server authors to
> publish theirs (see Req 22 — per-server audit reports). It would be
> hypocritical to demand transparency from publishers while keeping
> our own attack surface opaque. If you're evaluating whether to trust
> Nerlo badges, section (1) is what you want to audit. If you're a
> security researcher, the [scanner provenance manifest][provenance]
> records, for every tool in our scanner image, the exact mechanism
> that verifies it — a SHA-256 of the downloaded artifact for the
> binary installs (bearer, trivy, osv-scanner, gosec), a signed
> Go transparency-log `h1:` module hash for govulncheck and capslock,
> and hash-pinned dependency installs for the three package-manager
> tools (`npm ci` against a committed lockfile with per-dep SHA-512
> integrity; `pip install --require-hashes`). Read the
> `verification_method` on each entry — they are not all the same, and
> the manifest says which is which so you can verify it independently.
>
> *Corrected 2026-08-11: this read "exact SHA-256 hashes for every
> binary in our scanner image". Four of the nine entries do carry a
> binary SHA-256; the other five verify something else — a module zip
> against a transparency log, or a dependency set against a lockfile.
> The posture is not weaker for it, but "every binary" was the
> load-bearing word and it was not true, on the document the
> self-attestation endpoint publishes.*
>
> [provenance]: ../docker/scanner-base/scanner-manifest.json

## Why this document exists

Nerlo is a security transparency aggregator: it runs a suite of
scanners against MCP servers and publishes each scanner's
verdict separately — most of them third-party tools we pin, the rest
detectors we wrote ourselves, each labelled with its provenance on
[/scanners](https://nerlo.ai/scanners). (No count is stated here on
purpose: `web/src/lib/scanners.ts` is the roster, the web suite fails on
a hardcoded scanner count, and this file is outside that guard's reach.)
Consumers choose which scanners they trust. If
Nerlo itself is compromised, every downstream consumer reading our
badges is poisoned. Three classes of attacker we worry about most:

1. **Upstream supply-chain attackers** — they compromise a scanner
   (AgentShield, Bearer, Trivy, OSV-Scanner, …) or its dependencies.
   We ship the compromised binary in our scanner image. Every scan
   runs malware, or worse, silently hides findings.
   (`Bearer` stays on this list deliberately after its 2026-08-27
   deregistration: it is no longer EXECUTED, but the binary is still
   installed by the scanner image, so it remains shipped attack surface.
   Removing it from the image is open work.)
2. **Local dev exfiltrators** — they compromise an IDE extension,
   pre-commit hook, or build tool on a maintainer's machine.
   Maintainer commits a backdoor or leaks credentials.
3. **Registry consumer manipulators** — they get a malicious MCP server
   into the registry with a Verified badge (typosquatting, race
   conditions, scanner blind spots, AI classifier manipulation).

This document focuses on (1) and (2) because those are the threats
*we* uniquely have to defend against — (3) is the threat *the entire
spec* exists to defend against, and it's covered in `requirements.md`
and `design.md`.

## (1) Upstream supply-chain attacks

### Attack surface

| Surface | What we currently pull | Failure if compromised |
|---|---|---|
| **Base OS image** | `ubuntu:24.04` (pinned by SHA-256 digest) | Compromised registry → all scans run on backdoored kernel/userland |
| **Linux packages** | apt: curl, ca-certificates, git, python3, python3-pip, firejail, nodejs | Compromised mirror → backdoored binaries in scanner image |
| **NodeSource setup script** | Node.js 20.x setup (SHA-256 verified) | Compromised script → adds malicious apt repo → backdoored Node.js |
| **AgentShield** | `npm ci` via committed lockfile (ecc-agentshield@1.5.0) | Compromised npm registry → supply-chain payload |
| **Bearer binary** | GitHub release tarball (SHA-256 verified) | Compromised release pipeline → malicious binary |
| **Trivy .deb** | AquaSecurity GitHub release (SHA-256 verified) | Compromised release → malicious binary |
| **OSV-Scanner binary** | Google GitHub release (SHA-256 verified) | Compromised release → malicious binary, or silently-hidden CVEs |
| **Cisco Skill Scanner** | PyPI `cisco-ai-skill-scanner` 2.0.11 (hash-pinned requirements.txt) | Compromised PyPI package → malicious AST analyzer |
| **AgentAuditKit** | PyPI `agent-audit-kit` 0.3.74 (hash-pinned requirements.txt) | Compromised PyPI package → malicious analyzer |
| **capslock & govulncheck** | Go module transparency log `h1:` module hashes | Compromised Go module proxy → poisoned capability analyzer |
| **gosec** | GitHub release tarball (SHA-256 verified; activation gated) | Compromised release → backdoored Go SAST binary |
| **Trivy vulnerability DB** | `ghcr.io/aquasecurity/trivy-db` pinned by manifest digest | Poisoned CVE data → wrong verdicts |
| **OSV vulnerability DBs** (npm/PyPI/Go) | `osv-vulnerabilities.storage.googleapis.com` pinned by GCS object generation + SHA-256 | Poisoned CVE data → wrong verdicts |
| **Deferred scanners** (Augustus fuzzer, Garak, ModelScan, Pompelmi) | Future phases (Phase 7/9) | Evaluated per scanner prior to integration |

### Defenses — Layer 1 (implemented 2026-06-04)

- ✅ **Pin base image by SHA-256 digest**: `FROM ubuntu:24.04@sha256:786a8b558f...`. Even if Canonical's registry is compromised, we pull the bytes we verified.
- ✅ **Pin npm installs via committed lockfile**: `docker/scanner-base/agentshield/{package.json, package-lock.json}`. Lockfile has SHA-512 integrity per dep. `npm ci` aborts if upstream tarball doesn't match.
- ✅ **Pin pip installs via hash-pinned requirements.txt**: `cisco-skill-scanner` (91-dep lockfile) and `agent-audit-kit` (3-dep lockfile) compile with `--generate-hashes`. Pip verifies SHA-256 before extraction.
- ✅ **Pin Go module installs via transparency log**: `capslock` and `govulncheck` verified against Go checksum database (`sum.golang.org`) `h1:` hashes.
- ✅ **SHA-256 verify curl-installed scripts and binaries** before executing: Bearer install script checked against `13ed2d511adb...`; Trivy .deb against `d5f52c441416...`; OSV-Scanner binary against `bc98e15319ed...`; gosec tarball against `d7882e505b1f...`. If any upstream byte changes, build fails.
- ✅ **Pre-cache Bearer rules at build time** (`/opt/bearer-rules-cache`) → restore in container at scan time. Reduces runtime network calls.
- ✅ **Pre-cache Trivy vuln DB at build time** (`/opt/trivy-cache`, ~1.2GB). Phase 2 scans run with `--network=none`.
- ✅ **Pre-cache OSV vuln DBs at build time** (`/opt/osv-cache`, ~250MB across npm/PyPI/Go ecosystems). Phase 2 scans run with `--network=none`.
- ✅ **Pin both vulnerability FEEDS, not just the binaries that read them** _(task 34.14, 2026-08-06)_. `docker/scanner-base/feed-pins.json` fixes the trivy DB by OCI manifest digest and each OSV `all.zip` by GCS object generation + SHA-256, verified in-build by `docker/scanner-base/fetch-osv-db.py`.
- ✅ **Document scanner provenance in `docs/scanner-sources.md`** + machine-readable `docker/scanner-base/scanner-manifest.json` — per-scanner: source channel, pinned version, install command, license, SHA-256, verification date.
- ✅ **Nerlo-original detectors are repository code**: `nerlo-behavioral`, `nerlo-install-instruction`, and `nerlo-multi-source` run as internal Semgrep/AST detectors, governed by PR review and CI SAST rather than external binary execution. _(This line said "branch protection" until 2026-08-14 and that was false: `main` carries no protection rule and no ruleset — `gh api .../branches/main/protection` → 404, `.../rulesets` → `[]`. `tests/test_merge_block_claims_are_true.py` caught it. CI SAST does run on every PR; nothing mechanically blocks a merge, so the claim is now scoped to what exists.)_

### Defenses — Layer 2 (next turn / pre-launch)

- ⏳ **SBOM generation**: `syft` against the built image → committed `docker/scanner-base/sbom.json`. Every release of the image carries a manifest of contents.
- ✅ **Image signing** (R.23, shipped 2026-08-06): `cosign` keyless via the GitHub Actions OIDC identity, in `.github/workflows/deploy.yml`. Every image the deploy pipeline pushes — `scanner-base`, `api`, `worker`, `web` — is signed by digest immediately after push, and the signature is verified in the same step. The signer is the workflow itself, `https://github.com/nerlo-ai/nerlo/.github/workflows/deploy.yml@refs/heads/main`, issued by `https://token.actions.githubusercontent.com` — the same class of identity `src/nerlo/reports/signing.py` uses for audit-report signing (Req 22.4). Anyone can check an image with one command:

  ```
  cosign verify --certificate-identity 'https://github.com/nerlo-ai/nerlo/.github/workflows/deploy.yml@refs/heads/main' --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' <region>-docker.pkg.dev/<project>/nerlo/<image>@sha256:<digest>
  ```

  **That `github.com/nerlo-ai/nerlo/...` string is a certificate identity, not a page to visit.** The repository is private (shared with XPRIZE judges, not published), so opening it in a browser returns 404. Nothing is lost: `cosign verify` matches the string against the SAN in the signing certificate and checks the Rekor transparency-log entry — it never fetches the repository, so the verification above works for anyone, with no access to our source. This document itself is served from our own API rather than from that repository, which is why it renders here.

  The run summary of every deploy prints that line with the digest already filled in, and `.github/workflows/image-signature-verify.yml` (`workflow_dispatch`) runs it against what is in Artifact Registry right now. **Signing is advisory**: a signing failure raises a `::warning::` and does not fail the deploy, because the deploy pipeline is the only path to production here and coupling it to Fulcio/Rekor availability would let a Sigstore outage block the fix for an unrelated production outage. **Discloses metadata:** keyless signing records to Rekor, a public append-only log, so the repository name, workflow path and commit SHAs become permanently public. No code, no secrets, no image contents.
- ⏳ **Runtime signature enforcement** (R.23 follow-up, deliberately a SEPARATE PR — JesseTop's "signing + enforcement, sequenced" decision of 2026-08-06): a Binary Authorization policy so Cloud Run refuses an unsigned digest on `nerlo-api` and `nerlo-web`. Sequenced because a policy that requires signatures, applied before signatures exist, blocks **every** deploy — and signing is additive, so it cannot. Gate before opening it: a green `image-signature-verify` run for every image the policy will cover. Not covered by it either way: the worker VM is plain GCE running `docker compose`, so verifying `worker` at pull time is a separate problem.
- ⏳ **Trivy nightly scan of our OWN scanner-base image** in CI. Recursive transparency.
- ⏳ **Dependabot for every pinned upstream** (Dockerfile FROM digest, npm lockfile, pip requirements, scanner versions). Auto-PR for human review.

### Defenses — Layer 3 (architectural — already in place from aggregator stance)

- ✅ **Multi-scanner cross-check** — if AgentShield is compromised and starts hiding findings, Cisco + Trivy + OSV-Scanner + the Nerlo-original behavioral ruleset still flag them. (`Bearer` was named here until 2026-08-27; it was deregistered by task 117 over Elastic-2.0 and no longer runs, so it can no longer corroborate anything. Removing it narrowed this defence by one scanner — stated rather than quietly re-listed.) The per-scanner badges reflect each scanner's verdict; consumers see disagreements directly rather than us papering over them. A single-scanner compromise can't silently corrupt the registry.
- ✅ **Cross-vuln-DB redundancy for Phase 2** — Trivy queries Aqua's curated DB; OSV-Scanner queries OSV.dev + GitHub Advisory. Different sources, different coverage. If one DB is poisoned, the other catches the lie. Smoke verified 2026-06-04: both detect the same 19 CVEs in our `vulnerable-deps` fixture independently.
- ⏳ **Divergence detection** — when one scanner's findings on the same fixture set diverge sharply from peers over time, alert.

## (2) Local dev environment compromise

### Attack surface

| Surface | Risk |
|---|---|
| **IDE extensions** (Kiro / VS Code via OpenVSX) | Compromised extension can read entire filesystem, intercept clipboard, run arbitrary code, exfiltrate to attacker server |
| **Pre-commit hooks** | Third-party hooks run at every commit with shell access |
| **Shell history / `.bash_history`** | Often contains accidentally-typed secrets |
| **`.env` files / `~/.aws`, `~/.gcloud`** | Long-lived credentials accessible to any extension or hook |
| **`gh` / `gcloud` / `git` CLIs** | Auth tokens stored in user-readable files |
| **`npm install` / `pip install` during dev** | Postinstall scripts can execute arbitrary code |
| **AI agents (Claude Code, Cursor, Kiro itself)** | Wide tool access; can read/write filesystem, run shell commands |

### Defenses — Layer 1 (implemented 2026-06-04)

- ✅ **Pre-commit `gitleaks`** — detects accidentally committed secrets (AWS keys, GitHub PATs, Stripe keys, generic high-entropy strings) before commit lands. In `.pre-commit-config.yaml`.
- ✅ **`.env` already gitignored** — verified at session start.
- ✅ **`.claude/settings.local.json` already gitignored** — per-user permissions don't leak.
- ✅ **AgentShield self-scan Kiro hook** at `.kiro/hooks/agentshield-self-scan.kiro.hook` — fires on edits to `.claude/settings*.json`, `.claude/agents/*.md`, `.claude/hooks/*.kiro.hook`, `.claude/skills/**/*.md`. Runs `npx ecc-agentshield@1.4.0` (pinned to scanner-image version) against our own config. Findings printed to stderr; non-blocking. We are our own first consumer. First run: 95/100, 4 findings (1 high, 2 medium, 1 low) — same gaps as a fresh Claude Code install with no deny list or PreToolUse hooks.
- ✅ **AgentShield self-scan pre-commit hook** — same script wired as a local pre-commit hook, scoped to changes under `.claude/`. Belt-and-suspenders: catches the change at commit time even if the Kiro hook didn't fire.
- ✅ **`security-reviewer` Kiro subagent** at `.kiro/agents/security-reviewer.md` — read-only audit pass of `git diff --staged` for: hardcoded secrets, supply-chain risks (unpinned new deps, `curl | sh` without SHA verify, new MCP server URLs in `.kiro/settings/mcp.json`), outbound network in code (new `requests`/`httpx`/`fetch` imports), per-language code smells (`pickle.loads`, `eval`, `subprocess shell=True`), new Kiro hooks/agents claiming wide tool grants.
- ✅ **VS Code / Kiro extension recommendations** at `.vscode/extensions.json` — minimum audited set (ruff, basedpyright, terraform, docker, yaml, postgres, gitlens). `unwantedRecommendations` flags `ms-python.pylance` (Microsoft-licensed; not installable on Kiro's OpenVSX anyway, and `detachhead.basedpyright` is the project standard). New entries require security-reviewer agent approval.

### Defenses — Layer 2 (next turn)

- ⏳ **`commit.gpgsign true`** — sign commits so impersonation is detectable. Per-developer config; document in README when we write one.

### Defenses — Layer 3 (operational)

- ⏳ **Move secrets out of `.env` to Doppler / 1Password / GCP Secret Manager**.
- ⏳ **Short-lived OAuth tokens for `gh` / `gcloud`** via WIF (already true for CI; bring to dev).
- ⏳ **Run dev in a container/VM** with restricted egress. Heavy-handed but neutralizes most extension exfil attacks.

## (3) Registry-level attacks (mentioned for completeness)

The whole spec — scanning pipeline, badge override removal (aggregator stance), Discovery_Agent classification, badge appeal workflow — is the defense. See requirements.md.

## (4) Executing third-party vendor images (task 42.2, gates `nerlo-scanner` NS-J)

Written 2026-07-27 as the prerequisite 42.3 needs before any NS-J code lands.

### The trust boundary this crosses

Everything in sections (1)–(3) is about code WE chose to run. This is different: NS-J
proposes running a **publisher's container image** to drive a live MCP handshake. The
current phase-4 rule is "network for dependency resolution". This is "execute
untrusted vendor code", which is not an extension of that grant — it is a new
capability with a different blast radius, and the reason it needs a section here
before it needs an implementation.

Note what task 40 does and does not do, because the distinction is the whole point:

| | Task 40 (shipped) | NS-J (proposed) |
|---|---|---|
| Image is **pulled** | yes, by digest | yes |
| Image is **read** | yes — `trivy image` on an exported layout tar | yes |
| Image is **executed** | **NO** | **YES** |
| Network during analysis | none (`--network=none`, verified) | required for the handshake |

Task 40 deliberately stops at reading. Pulling and unpacking a tar is not running a
binary, so 40's blast radius is "a malicious manifest exploits Trivy's parser" —
bounded, offline, in a read-only container. NS-J's is "hostile code runs with
whatever the daemon grants it".

### Attack surface

| Surface | Vector | Failure if hostile |
|---|---|---|
| **`runtimeArguments`** | Registry-supplied. Publisher-controlled DATA. | Replaying it naively hands an attacker our `docker run` flags. `-v /:/host`, `--privileged`, `--network=host`, `--pid=host` are each **one malicious registry entry away** from a full host compromise. |
| **`environmentVariables`** | Registry-supplied, publisher-controlled | `LD_PRELOAD`, `PATH`, `NODE_OPTIONS`, `PYTHONPATH` alter what executes before any of our code sees it. |
| **The image entrypoint** | Vendor-controlled | Arbitrary code, at container start, before a single MCP byte is exchanged. |
| **The MCP handshake itself** | Vendor-controlled responses | Malformed/oversized frames against our client; a slow-loris peer holding a scan slot open. |
| **The Docker socket** | Ours, if the scan container can reach it | **MEASURED 2026-07-27 (42.1): a nested container is a SIBLING.** The socket reaches the HOST daemon — identical daemon ID observed from inside the container — so anything created through it is a peer of the scanner, and the scan container's `network=none` / `cap_drop=ALL` / mem / pids limits constrain it **not at all**. It gets whatever flags the caller passes, which under NS-J means flags derived from publisher-controlled `runtimeArguments`. |
| **Egress during handshake** | Required by design | C2, exfiltration of anything the container can read, or using us as an attack relay. |

### What an escape reaches

Assuming a breakout from the vendor container on `nerlo-worker`:

- **The Docker socket** → create privileged containers, mount the host filesystem. This is the escalation that matters, and 42.1 has now answered whether it is constrained: **it is not**. Today the socket is NOT mounted into the scan container and the image has no docker CLI, so nesting is impossible — both would have to be added deliberately, and neither belongs in the shared scanner image.
- **The worker VM's service account** — `roles/cloudsql.client`, `secretmanager.secretAccessor`, `aiplatform.user`, `artifactregistry.reader`. Via the metadata server that is the **production database and every secret**.
- **`/opt/nerlo/worker-vm/.env`** — the Cloud SQL password. Mode 0600 root-only as of task 45.2, which helps only if the escape is unprivileged.
- **Other artifacts' clones** in the shared scan workspace, i.e. cross-tenant read.
- **Redis** — the Celery broker. Enqueue arbitrary scan jobs, or poison results.

That list is the argument for keeping this off the worker VM entirely rather than
hardening it in place.

### Required mitigations, if NS-J proceeds

Ordered by how load-bearing each is. The first two are not negotiable.

1. **NEVER replay `runtimeArguments` as flags.** Allowlist environment-variable NAMES only; drop every volume, network, user, privileged, pid, ipc and capability flag; never route the invocation through a shell. The registry entry is an untrusted string, and this is the difference between a sandbox and a remote-root API. Task 40.0 already stores only four vetted `packages[]` fields for exactly this reason — `runtimeArguments` is deliberately NOT persisted, and that decision should hold.
2. **Never on the shared worker VM.** A dedicated, disposable, egress-filtered sandbox — gVisor/Firecracker or a per-scan micro-VM — with no Docker socket, no metadata-server access (`--add-host metadata.google.internal:127.0.0.1` is not sufficient; block it at the network layer) and its own service account holding nothing.
3. **Digest-pinned, size-capped, verified** — already built in task 40.1; reuse it rather than re-implementing a pull.
4. **Egress allowlist**, not merely "network on". The handshake needs the container's own stdio, not the internet; if a transport genuinely needs egress, enumerate the destinations.
5. **Hard wall-clock and resource caps**, and treat a timeout as `incomplete` — never `not_applicable`. A vendor that hangs must not score as "nothing to assess" (the Req 36.5 coverage-honesty rule).
6. **Publish nothing derived from a hostile response without provenance.** A handshake result is the vendor's claim about itself.

### Recommendation for 42.3

**Do not build NS-J on the worker VM.** Mitigations 1 and 2 together mean it needs a
separate isolated execution environment, which is a materially larger piece of work
than the handshake harness — and task 42.1's finding (child vs sibling containers
under the shared daemon) determines whether even that is safe as designed.

The honest comparison: task 40 already closed the concrete coverage gap NS-J was
motivated by. `hashicorp/terraform-mcp-server` yielded **41 CVEs (1 critical, 15
high)** from static image analysis with **zero code execution**. Live-handshake
behavioural evidence is genuinely additional, but it is not the same order of value
as that, and it carries the only remote-code-execution risk in the entire system.

**42.1 has now reported, and it strengthens this.** A nested container is a sibling
under the shared daemon, so the isolation the scan phases rely on does not extend to
it. Hardening in place is therefore not available, which makes mitigation 2 — a
separate isolated execution environment — a requirement rather than a preference.

Static-first stands.

## (5) Authentication & API Key Security Architecture

Nerlo issues personal and operator API keys for authenticated CLI operations (`nerlo submit`, `nerlo rescan`, `nerlo check`) and programmatic registry access. The credential architecture is designed so that even a total database breach cannot compromise user keys.

### Security Guarantees & Implementation

1. **High-Entropy Cryptographic Generation**:
   Keys are minted using 32 cryptographically secure random bytes (`secrets.token_urlsafe(32)`, yielding ~256 bits of entropy) with an identifiable prefix (`nerlo_...`). Brute-force attacks against 256-bit space are computationally impossible.

2. **Zero Plaintext Storage (One-Way SHA-256 Hashing)**:
   Plaintext tokens are **never written to disk or the database**. Upon generation, the server returns the plaintext token once to the client and immediately stores only `key_hash = hashlib.sha256(token.encode('utf-8')).hexdigest()` in the `api_keys` table. If the database is dumped, an attacker obtains only irreversible cryptographic hashes, not usable credentials.

3. **Prefix-Only Masking on Read**:
   All listing endpoints (`GET /api/v1/users/me/api-keys`) and dashboard views expose only the masked prefix (e.g. `nerlo_8f3a9e...`) for human identification. The full key is never reconstructible from the API.

4. **Multi-Tenant Session & Row-Level Isolation**:
   Key creation, listing, and revocation are strictly gated by Google Cloud Identity Platform (Firebase SDK) session tokens. All database queries enforce tenant isolation (`WHERE operator_id = :user_id`). A compromised session on Account A cannot read, generate, or revoke keys belonging to Account B.

5. **Zero-Secret Observability**:
   Structured logs (`api.auth.*`, `api.user_api_key_*`) emit only opaque UUIDs (`key_id`, `user_id`) and status codes. Plaintext tokens and hashes are scrubbed by design and never enter log sinks or Cloud Logging.

6. **Instant Revocation**:
   Revoking a key via `DELETE /api/v1/users/me/api-keys/{id}` deletes its hash in a single transaction, immediately terminating authorization across all in-flight CLI sessions and CI/CD pipelines.

## Maintenance

- When adding a new scanner: add a row to the "Attack surface" table in section (1), pin via Layer 1 defenses, update scanner-sources.md, regenerate SBOM.
- When adding a new dev dependency / IDE extension: assess against section (2) attack surface.
- Before ANY change that executes third-party code (not merely reads it): re-read section (4). Pulling, unpacking and statically analysing an artifact is a different act from running it, and only the second one needs that section's mitigations.
- Quarterly: review all pinned digests + hashes against fresh upstream; bump after verification.