SCANNER_PIPELINE.md

Activation Scanner Pipeline

Last updated: 2026-06-03 Status: pipeline-first methodology

Position

The activation scanner should ship as a repeatable pipeline before the dataset is perfect.

The first public claim should be about the method and the update loop:

We scan tool, MCP, skill, and plugin descriptions before install by combining static checks with a frozen local sensor model, activation probes, and optional SAE explanations. The probe can be retrained as new attack families and pentest cases are added.

This is different from claiming that the current dataset is complete or that the current probe is final.

Priority framing: the March 29, 2026 activation-probe paper in this repo is the starting point. Later internal-signal papers such as RouteGuard strengthen the category argument; they do not change the product wedge, which is to ship a local scanner path rather than only a paper result.

Launch posture:

• Show the method publicly now.
• Keep the benchmark harness easy to rerun.
• Mark the current dashboard scanner as static/text/rule based.
• Mark the activation scanner as the research-to-product preview until hook integration and calibrated release gates exist.
• Improve the dataset and probe through normal scanner updates, not a hidden research freeze.

Core Method

The base language model is frozen. Its weights do not change during scanner training.

The training process is:

1. Collect clean and poisoned tool descriptions.
2. Run each description through a frozen local sensor model.
3. Extract hidden-state activations from selected layers.
4. Train a small supervised probe on those activation vectors.
5. Evaluate the probe across source/style/family splits.
6. Package the scanner with the frozen base model id, layer policy, probe weights, thresholds, and benchmark report.

In notation:

text -> frozen sensor model -> activation vector X
label -> clean or poison y
train probe: X -> y

The first probe model should stay simple, usually logistic regression, because a linear probe gives a clean read on whether the activation signal is real. More complex models such as LightGBM, XGBoost, or a small MLP can be added after the dataset curriculum is stable.

Scanner Runtime

At runtime, the scanner does not retrain anything. It only scores new inputs.

new tool description
  -> static risk checks
  -> frozen sensor model forward pass
  -> selected activation features
  -> trained probe score
  -> optional SAE feature explanation
  -> decision policy
  -> allow / warn / block / quarantine

The scanner must inspect metadata and files before running untrusted code. It should not execute install scripts just to decide whether a tool is safe.

The product hook path now has two runtime shapes:

• one-shot gate: normalize a tool/MCP/skill/hook payload, scan it, and return a JSON decision plus an exit code for install-time blocking;
• warm JSONL process: keep the Python scanner alive, feed one payload per line, and reuse cached tokenizer/model/SAE objects across repeated scans.

Secret values in MCP env/config payloads are redacted before scanner text is constructed. Key names remain visible because they carry risk context without leaking the secret itself.

The current runtime policy is corroborated-block-v3: warn can come from a high risk score, but block needs a nearby static block bundle or action-oriented exfiltration, hidden-action, or host-modification evidence. This keeps the review signal broad while preventing a threshold-only high score or generic secret-management language from becoming an automatic hard block.

Why Data Still Matters

This pipeline still needs labeled examples. A better feature source does not remove the need for good labels.

The current failure mode is that many public poisoning examples teach overt instruction chaining:

must call X
before using Y
ignore previous instructions

Realistic MCP and tool poisoning can be quieter:

read private keys
forward cookies or credentials
upload configuration contents
log generated secrets
make live handshakes with stored credentials

The scanner should therefore learn attack families, not only prompt-injection phrases.

Data Curriculum

Each dataset row should eventually carry:

• label: clean or poisoned
• family: risk family or benign lookalike family
• source: original dataset, MCP repo, skill repo, pentest case, synthetic case
• pair_id: optional clean/poison matched pair
• notes: short reason for the label

Initial families:

Family	Meaning
instruction_chaining	Forces hidden or unsafe tool-call sequencing.
secret_file_access	Reads private keys, tokens, credentials, cookies, or env files.
credential_forwarding	Sends or validates credentials outside the expected local scope.
network_exfiltration	Uploads, transmits, tracks, or forwards sensitive data.
hidden_persistence_logging	Stores secrets, adds hidden logs, or keeps background state.
live_system_access	Performs live handshakes, registry pulls, database checks, or network probes beyond passive parsing.
system_inventory	Reads host identity or system inventory data.
tool_shadowing	Tries to intercept, route, or override tools belonging to another server.

The benchmark should include clean twins wherever possible. For example:

Clean: Reads SSH config and returns host aliases, hostnames, and ports.
Poison: Reads SSH config and also reads private key files to verify identities.

Clean twins are important because a scanner that blocks every mention of SSH, database, file, cookie, or URL is not useful.

Current curated dataset files:

• research/datasets/family_curated_v0.json - 38 matched clean/poison pairs across 8 risk families. Rows include curated/synthetic pairs, real-carrier-adapted pairs, regression fixtures, 8 accepted public Skill-Inject SKILL.md pairs whose unsafe action is source-confirmed, and bridge rows that teach host fingerprinting across multiple risk families.
• research/datasets/DATA_CURRICULUM.md - manual labeling rubric for risk families, curriculum levels, benign lookalikes, and promotion rules.
• research/build_curriculum_review_queue.py - builds a non-training review queue from external rows before any promotion into curated data.
• research/promote_skillinject_curriculum.py - idempotently promotes the accepted Skill-Inject clean/poison SKILL.md pairs into curated data and writes research/datasets/curriculum_review_decisions_v0.json.
• research/schemas/activation_curated_dataset.schema.json - row contract for family, source type, pair id, split group, text, and notes.
• research/validate_curated_dataset.py - validates ids, labels, pair balance, family coverage, split-group consistency, and schema shape.

Update Loop

New cases should be added like pentest findings.

1. Find a new suspicious tool, MCP server, skill, plugin, or attack pattern.
2. Reduce it to a clear description-level case.
3. Label the risk family and write the reason.
4. Add a clean twin or benign lookalike if one does not already exist.
5. Add the case to the dataset with source and family metadata.
6. Run duplicate/leakage checks.
7. Retrain the probe.
8. Compare against the previous scanner version on fixed benchmark gates.
9. Release a new scanner version only if it improves or preserves the gates.

This makes the model update process incremental. We do not need a perfect dataset on day one, but every new case must make the benchmark more realistic instead of just making the training set larger.

Benchmark Gates

A scanner version is not ready just because it scores well on the same dataset style it trained on.

Required gates:

Gate	Required check
Same-split regression	Confirms the runner and probe have not broken.
Cross-style generalization	Train on one source/style and test on another.
Family holdout	Hold out one attack family and test whether the scanner generalizes.
Benign lookalikes	Measure false positives on safe tools that mention risky objects.
Runtime	Measure wall time, memory, and model footprint on a normal laptop.
Explanation	Record static reasons and SAE feature hints where available.

Same-split results are useful as regression checks. They are not enough for a product claim.

Versioned Artifacts

Each scanner release should record:

• dataset version
• base sensor model id
• layer or layer-selection policy
• SAE release and SAE id, if used
• probe algorithm
• probe weights
• threshold policy
• benchmark report
• known failure modes
• release date

The important object is not just one trained probe. It is the full recipe that can be rerun and improved.

Current artifact path:

research/train_probe_artifact.py
  -> metadata.json
  -> probe_weights.npz
  -> research/activation_scanner_core.py
  -> research/activation_scanner_cli.py
  -> JSON risk object

The runtime core loads the artifact and scores new text without retraining. It also supports batch scanning so a group of tool descriptions can share one model/SAE feature-extraction pass. The CLI wrapper adds a product-preview entrypoint with artifact doctor, single scan, batch scan, human summaries, JSON stdout for hooks, and --fail-on warn|block exit codes.

Curated v0 can be used directly as a training source. The current product path uses the pooled Qwen artifact and v3 policy-aware calibration queue:

research/.venv-audit/bin/python -m research.validate_curated_dataset --pretty
research/.venv-audit/bin/python -m research.train_probe_artifact --model qwen2.5-0.5b --feature-kind raw --train-source pooled-curated-core --layers 13,14,15 --layer-mode concat --artifact-id qwen-pooled-curated-core-l13-15-v2 --output-dir research/_results/activation_scanner_artifacts --overwrite --warn-threshold 0.30 --block-threshold 0.85 --pretty
research/.venv-audit/bin/python -m research.calibrate_scanner_thresholds --artifact research/_results/activation_scanner_artifacts/qwen-pooled-curated-core-l13-15-v2 --pretty
research/.venv-audit/bin/python -m research.build_calibration_error_queue --calibration research/_results/activation_scanner_calibration/20260603T102832Z-qwen-pooled-l13-15-threshold-calibration.json --output research/datasets/calibration_error_review_queue_qwen_pooled_policy_v3_warn030_2026-06-03.json --pretty
research/.venv-audit/bin/python -m research.build_calibration_review_decisions --queue research/datasets/calibration_error_review_queue_qwen_pooled_policy_v3_warn030_2026-06-03.json --output research/datasets/calibration_error_review_decisions_qwen_pooled_policy_v3_warn030_2026-06-03.json --source-calibration research/_results/activation_scanner_calibration/20260603T102832Z-qwen-pooled-l13-15-threshold-calibration.json --decision-policy corroborated-block-v3 --pretty
research/.venv-audit/bin/python -m research.materialize_calibration_review_outputs --pretty
research/.venv-audit/bin/python -m research.build_policy_regression_cases --pretty
research/.venv-audit/bin/python -m research.activation_scanner_cli_regression --pretty
research/.venv-audit/bin/python -m research.activation_scanner_regression --artifact research/_results/activation_scanner_artifacts/qwen-pooled-curated-core-l13-15-v2 --cases research/fixtures/activation_scanner_policy_regression_cases.json --no-build --pretty

Runtime contract files:

• research/schemas/activation_scanner_risk.schema.json
• research/fixtures/activation_scanner_regression_cases.json
• research/activation_scanner_regression.py
• research/activation_scanner_cli.py
• research/activation_scanner_cli_regression.py
• research/calibrate_scanner_thresholds.py
• research/build_calibration_error_queue.py
• research/build_calibration_review_decisions.py
• research/materialize_calibration_review_outputs.py
• research/build_policy_regression_cases.py
• research/fixtures/activation_scanner_policy_regression_cases.json
• research/PRODUCT_REPRODUCIBILITY_LEDGER_2026-06-03.md

Current Default Direction

Current evidence supports a first MVP lane, but not a final universal detector.

The working direction is:

• use static checks first for obvious risk;
• use Qwen2.5-0.5B raw fixed layers 13,14,15 / qwen-pooled-curated-core-l13-15-v2 as the current external-transfer activation candidate for warn/review;
• keep qwen-curated-family-best3-v1 as the curated-family proof artifact and hook-smoke reference;
• keep Pythia-70M raw/SAE as the cheap canary and explanation comparison lane;
• keep Gemma 1B/4B as optional heavier/deep-scan lanes;
• use corroborated-block-v3 for preview hard-block decisions, and do not ship threshold-only hard block from the current Qwen pooled artifact;
• improve broad clean warnings, line-level Skill-Inject/BIPIA recall, and warm-process runtime before calling any artifact the final default scanner.

The MVP should be honest:

This is an activation-based scanner pipeline with updateable probes and a growing attack-family benchmark, not a one-time frozen detector that claims to know every future MCP attack.