audit fleet activity

Audit a fleet’s recent activity

When you need to know what a fleet has been doing — for a quarterly review, an incident response, or before broadening its fleets_allowed scope — this runbook walks you through reconstructing the fleet’s recent PR trail.

What you can audit from the petrova metadata block alone

Every PETROVA-emitted PR has a YAML metadata block in the body:

petrova:
  verb: propose_fix
  target_repo: kahn-hq
  idempotency_key: 4f8c3e21d97a...
  schema_version: 1
  schema_fingerprint: a8b3c4d5e6f7
  actor: fleet:kahn-implementer
  triggered_by:
    kind: audit_fail
    ref: ci-run-12345
  applied_at: 2026-04-29T14:32:18Z

This is enough to reconstruct: what verb fired, with what inputs (via the idempotency key), under what trigger, against what schema version.

Step 1 — pull recent PRs by author

gh search prs \
  --owner kahn-hq \
  --repo kahn-hq/kahn \
  --label petrova \
  --state all \
  --created ">2026-04-01" \
  --json number,title,author,createdAt,state,url

Or by branch prefix (every PETROVA PR’s branch starts with petrova/):

gh pr list --repo kahn-hq/kahn --state all --search 'head:petrova/' --limit 100

Step 2 — extract the metadata block from each PR

gh pr view <number> --repo kahn-hq/kahn --json body | \
  jq -r '.body' | \
  awk '/<!-- petrova:metadata -->/,/<!-- \/petrova:metadata -->/'

For batch extraction:

for pr in $(gh pr list --repo kahn-hq/kahn --state all --search 'head:petrova/' --limit 100 --json number -q '.[].number'); do
  echo "=== PR #$pr ==="
  gh pr view $pr --repo kahn-hq/kahn --json body | \
    jq -r '.body' | \
    awk '/^petrova:/,/^```$/' | \
    head -20
done > /tmp/audit-trail.txt

Step 3 — group by `actor`

grep -oP 'actor: \S+' /tmp/audit-trail.txt | sort | uniq -c | sort -rn

Output:

  42 actor: fleet:kahn-implementer
   8 actor: fleet:kahn-diagnostics
   3 actor: human:alex@devarno.com
   1 actor: fleet:kahn-reviewer

Step 4 — reconstruct the trigger chain

For a specific fleet’s PRs, look at triggered_by:

grep -A 2 'actor: fleet:kahn-implementer' /tmp/audit-trail.txt | \
  grep -oP 'kind: \K\S+' | sort | uniq -c

Output:

  35 audit_fail
   5 friction_item
   2 schedule

Now you know: this fleet primarily acts on audit failures, with some friction-item follow-ups and a couple of scheduled runs.

Step 5 — link to the originating findings

Each triggered_by.ref should resolve to a real artefact. For kind: finding, it’ll be a path like docs/findings/20260429-1100-rls-drift.md. Verify they exist:

grep -oP 'ref: \S+\.md' /tmp/audit-trail.txt | sort -u | while read path; do
  if [[ -f "$path" ]]; then
    echo "OK   $path"
  else
    echo "MISS $path"
  fi
done

MISS lines mean the fleet referenced a finding that doesn’t exist or has been moved. That’s a flag — either the fleet is fabricating triggers (drift) or you’ve reorganised findings since (legitimate, but should be documented).

Step 6 — check schema fingerprint distribution

If the schema for a verb has bumped recently, fleets running on old fingerprints may be using outdated contracts:

grep -oP 'schema_fingerprint: \S+' /tmp/audit-trail.txt | sort | uniq -c | sort -rn

If you see multiple fingerprints for the same verb across recent PRs, it means the fleet was restarted between schema versions. Usually fine; worth a glance to confirm the fleet’s runtime is synced.

What to look for

Pattern	What it means
All PRs from one fleet have identical `idempotency_key`	Retry loop firing — the fleet is hitting `skipped_idempotent` repeatedly. Probably benign but inspect.
`triggered_by.ref` consistently doesn’t resolve	Fleet making up triggers. Suspend its `fleets_allowed` membership.
`schema_fingerprint` is months old	Fleet’s runtime hasn’t been redeployed since a schema bump. Consider whether the old contract is still valid.
Fleet emitting verbs outside its declared scope	Doc the fleet’s intended scope; surface the surprise via finding doc.
High failure rate (closed without merge)	Fleet’s fixes aren’t landing. Investigate proposal quality.

Revoking fleet access

If a fleet’s behaviour has drifted and you want to halt its writes immediately:

Edit registry.yaml — remove the fleet ID from the relevant repo’s fleets_allowed.
Open via petrova request_review petrova-hq (yes, the registry is itself edited via verb).
Apply with --apply. The fleet’s next write will fail with FLEETS_ALLOWED.

For instant revocation without waiting for PR review (incident mode), edit the registry directly and push (only the operator should do this; document the reason in a same-day decision doc).