Incident Response Plan

Purpose and Scope

This plan defines the detection, triage, containment, eradication, recovery, and post-mortem procedures for security incidents affecting the dotfiles distribution.

In Scope

Out of Scope

---

Severity Classification Matrix

Severity levels and response times align with the Vulnerability Response SLA defined in COMPLIANCE.md.

---

Incident Response Phases

Phase 1: Detection

Identify the incident through automated or manual signals.

Automated detection sources:

Manual detection signals:

• Unexpected chezmoi diff output on a clean system
• Unrecognized entries in git log --show-signature
• Binary hash mismatch for tools in ~/.local/bin
• Shell startup latency spike (possible injected sourcing)

Phase 2: Triage

Classify the incident severity and assign ownership.

# Gather initial evidence
dot audit | tail -50
git log --oneline --show-signature -20
chezmoi diff
chezmoi verify

# Check for secrets in recent history
gitleaks detect --source . --log-opts="-20"

# Validate binary integrity
mise ls --current
nix flake metadata

Triage decision tree:

1. Are credentials or secrets exposed publicly? -> Critical
2. Is a signed commit chain broken? -> High
3. Is the CI pipeline producing unexpected artifacts? -> High
4. Is configuration drift limited to non-sensitive files? -> Medium
5. Is the finding advisory with no active risk? -> Low

Phase 3: Containment

Stop the incident from spreading. Actions depend on severity.

Immediate containment (Critical/High):

# Revoke exposed credentials
dot secrets rotate --all-exposed

# Revoke SSH certificates
dot ssh-cert revoke

# Lock configuration files to prevent further modification
lock-configs

# Disable compromised CI workflow
gh workflow disable <workflow-name>

# Force-protect the main branch
gh api repos/{owner}/{repo}/branches/main/protection \
  --method PUT \
  --field required_status_checks='{"strict":true,"contexts":["ci"]}' \
  --field enforce_admins=true

Short-term containment (Medium/Low):

# Pin the affected dependency to a known-good version
# In .chezmoidata.toml or flake.nix, revert to last verified version

# Re-apply known-good configuration
chezmoi apply --force

# Clear potentially tainted caches
rm -rf ~/.cache/shell/

Phase 4: Eradication

Remove the root cause of the incident.

Phase 5: Recovery

Restore normal operations and verify integrity.

# Re-apply dotfiles from clean source
chezmoi init --apply

# Verify deployed state matches source
chezmoi verify

# Run full test suite
./tests/framework/test_runner.sh

# Run compliance checks
pre-commit run --all-files

# Validate system health
dot health
dot doctor

# Confirm audit log captures recovery
dot audit | tail -20

Phase 6: Post-Mortem

Conduct a structured review within 5 business days of resolution. Use the Post-Incident Review Template below.

---

Runbooks

Runbook 1: Secrets Leaked to Git History

Detection: Gitleaks pre-commit hook, TruffleHog CI scan, or manual discovery.

# Step 1: Identify exposed secrets
gitleaks detect --source . --verbose --report-path /tmp/gitleaks-report.json

# Step 2: Determine exposure window
git log --all --oneline --diff-filter=A -- '**/.*env*' '**/*key*' '**/*token*'

# Step 3: Rotate all exposed credentials immediately
# API keys: regenerate in provider dashboard
# SSH keys: generate new keypair and update authorized_keys
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new
# Tokens: revoke and reissue via provider

# Step 4: Remove secrets from git history
git filter-repo --invert-paths --path <file-containing-secret>

# Step 5: Force-push cleaned history (requires branch protection override)
git push origin --force --all

# Step 6: Notify GitHub to purge cached views
# Open a support ticket at https://support.github.com for cache invalidation

# Step 7: Update Gitleaks baseline
gitleaks detect --source . --baseline-path config/gitleaks-baseline.json

# Step 8: Verify clean state
gitleaks detect --source . --verbose

Post-action: Add the leaked pattern to config/gitleaks.toml allowlist if it was a false positive, or add a new rule if the pattern was not previously covered.

Runbook 2: Supply Chain Compromise

Detection: CVE advisory, unexpected binary behavior, hash mismatch, or nightly.yml alert.

# Step 1: Identify the compromised package
mise ls --current
brew list --versions
nix flake metadata

# Step 2: Check upstream advisories
gh api /advisories --jq '.[] | select(.package.name == "")'

# Step 3: Pin to last known-good version
# For Nix: revert flake.lock to known-good commit
git checkout <known-good-commit> -- flake.lock
nix flake lock --update-input nixpkgs

# For Homebrew: pin the formula
brew pin <formula>

# For mise: set explicit version
mise use <tool>@<safe-version>

# Step 4: Verify integrity of installed binary
sha256sum "$(which <tool>)"
# Compare against published checksums from upstream release

# Step 5: Re-apply configuration with clean dependencies
chezmoi apply --force

# Step 6: Run full test suite to detect behavioral changes
./tests/framework/test_runner.sh

Runbook 3: Configuration Drift

Detection: chezmoi diff shows unexpected changes, chezmoi verify fails, or dot health reports drift.

# Step 1: Identify drift scope
chezmoi diff
chezmoi managed --include=files | wc -l

# Step 2: Capture current deployed state for forensic comparison
chezmoi dump --format=json > /tmp/chezmoi-state-$(date +%s).json

# Step 3: Check audit log for unauthorized operations
dot audit | grep -E "(apply|edit|add)" | tail -20

# Step 4: Determine root cause
# Option A: Local manual edit (benign)
# Option B: External tool modified config (investigate)
# Option C: Compromised apply (escalate to Critical)

# Step 5: Restore to source-of-truth state
chezmoi apply --force

# Step 6: Verify restoration
chezmoi verify
chezmoi diff  # Should produce no output

# Step 7: Lock critical configs to prevent recurrence
lock-configs

Runbook 4: Tool Compromise (mise/Nix Managed)

Detection: Unexpected binary behavior, hash mismatch, or CVE disclosure for a managed tool.

# Step 1: Quarantine the affected tool
chmod 000 "$(which <tool>)"

# Step 2: Record forensic evidence
sha256sum "$(which <tool>)" > /tmp/quarantine-evidence.txt
ls -la "$(which <tool>)" >> /tmp/quarantine-evidence.txt
file "$(which <tool>)" >> /tmp/quarantine-evidence.txt

# Step 3: Check if the tool executed during shell startup
grep "<tool>" ~/.cache/shell/*
grep "<tool>" ~/.local/share/dotfiles.log

# Step 4: Reinstall from verified source
mise install <tool>@<verified-version> --force
# Or for Nix:
nix profile remove <package>
nix profile install nixpkgs#<package>

# Step 5: Verify replacement binary
sha256sum "$(which <tool>)"

# Step 6: Clear cached eval output that may reference the compromised tool
rm -rf ~/.cache/shell/

# Step 7: Restart shell and verify clean startup
exec "$SHELL" -l
dot health

Runbook 5: CI Pipeline Compromise

Detection: Unexpected workflow behavior, modified workflow files, compromised action dependency.

# Step 1: Disable the affected workflow
gh workflow disable <workflow-name>

# Step 2: Audit recent workflow changes
git log --oneline --all -- '.github/workflows/'
git diff HEAD~10 -- '.github/workflows/'

# Step 3: Check action pinning integrity
grep -r "uses:" .github/workflows/ | grep -v "@"  # Find unpinned actions

# Step 4: Verify no unauthorized secrets access
gh api repos/{owner}/{repo}/actions/runs \
  --jq '.workflow_runs[] | {id, name, conclusion, head_sha}' | head -20

# Step 5: Review workflow permissions
grep -r "permissions:" .github/workflows/

# Step 6: Restore workflows from known-good state
git checkout <known-good-commit> -- .github/workflows/

# Step 7: Re-pin all actions to verified SHA
# Replace tag references with full commit SHA
# Example: actions/checkout@v4 -> actions/checkout@<full-sha>

# Step 8: Re-enable workflow and verify
gh workflow enable <workflow-name>
gh workflow run <workflow-name>
gh run list --workflow=<workflow-name> --limit=1

---

Communication Protocol

Notification Matrix

Escalation Path

1. Incident detected
   └── Automated: CI/hook blocks and logs
   └── Manual: Reporter opens private advisory

2. Triage (within Initial Response SLA)
   └── Classify severity
   └── Assign owner

3. Escalation triggers
   └── No response within SLA -> escalate to next severity level
   └── Scope expansion -> reclassify severity upward
   └── Active exploitation confirmed -> immediate Critical classification

External Notification

---

Evidence Preservation

Preserve all forensic evidence before performing eradication or recovery actions.

Evidence Collection Checklist

Structured Log Format

Incident evidence is recorded in JSONL format for automated processing:

# Append structured incident event to log
log_incident() {
  local severity="$1" type="$2" description="$3"
  printf '{"timestamp":"%s","severity":"%s","type":"%s","description":"%s","user":"%s","hostname":"%s"}\n' \
    "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
    "$severity" \
    "$type" \
    "$description" \
    "$(whoami)" \
    "$(hostname)" \
    >> ~/.local/share/incident-log.jsonl
}

# Usage
log_incident "critical" "secrets-exposure" "API key found in commit abc1234"

Retention Policy

---

Recovery Procedures

Standard Recovery

# Roll back to a known-good dotfiles state
dot rollback

# Restore specific configuration files
dot restore <target>

# Full re-apply from source of truth
chezmoi init --apply --force

# Verify system health
dot health
dot doctor

Full System Recovery

For incidents requiring complete re-provisioning:

# Step 1: Export current secrets (if not compromised)
dot secrets export > /tmp/secrets-backup.age

# Step 2: Clear all deployed dotfiles
chezmoi purge

# Step 3: Clear all caches
rm -rf ~/.cache/shell/
rm -rf ~/.cache/chezmoi/

# Step 4: Re-initialize from clean clone
git clone <repo-url> ~/.dotfiles
cd ~/.dotfiles
git verify-commit HEAD  # Verify signature chain

# Step 5: Re-apply
chezmoi init --apply

# Step 6: Re-import secrets (if exported)
dot secrets import /tmp/secrets-backup.age

# Step 7: Verify
chezmoi verify
dot health
./tests/framework/test_runner.sh

Recovery Verification Checklist

---

Post-Incident Review Template

Conduct a review within 5 business days of resolution. Copy the template below into a new file under docs/security/incidents/.

# Post-Incident Review: [INCIDENT-YYYY-NNN]

## Metadata

| Field | Value |
|-------|-------|
| **Date detected** | YYYY-MM-DD HH:MM UTC |
| **Date resolved** | YYYY-MM-DD HH:MM UTC |
| **Severity** | Critical / High / Medium / Low |
| **Incident type** | Secrets exposure / Supply chain / Drift / Tool compromise / CI compromise |
| **Responder** | @handle |

## Timeline

| Time (UTC) | Event |
|------------|-------|
| HH:MM | Incident detected by [source] |
| HH:MM | Triage completed, classified as [severity] |
| HH:MM | Containment action taken: [description] |
| HH:MM | Root cause identified: [description] |
| HH:MM | Eradication completed |
| HH:MM | Recovery verified |

## Root Cause

[Describe the root cause. Include the specific technical failure, misconfiguration, or external event.]

## Impact

| Dimension | Assessment |
|-----------|------------|
| Data exposed | [What data, if any, was exposed] |
| Systems affected | [Which machines, configs, or pipelines] |
| Duration of exposure | [Time between introduction and remediation] |

## Response Evaluation

| Metric | Target | Actual |
|--------|--------|--------|
| Detection time | Automated / < 1 hour | [actual] |
| Initial response | Per severity SLA | [actual] |
| Resolution | Per severity SLA | [actual] |

## Lessons Learned

### What went well

- [Item]

### What needs improvement

- [Item]

## Action Items

| Action | Owner | Due Date | Status |
|--------|-------|----------|--------|
| [Preventive measure] | @handle | YYYY-MM-DD | Open |
| [Detection improvement] | @handle | YYYY-MM-DD | Open |
| [Process update] | @handle | YYYY-MM-DD | Open |

---

References

• NIST SP 800-61 Rev. 2 — Computer Security Incident Handling Guide
• COMPLIANCE.md — Vulnerability Response SLA
• THREAT_MODEL.md — Attack Surfaces and Mitigations
• SECURITY.md — Security Controls and Hardening
• SLSA Framework
• Gitleaks