Skip to content

Commit 2aea24f

Browse files
ithiria894claude
andcommitted
cleanup: remove old/superseded research process notes from the public tree
Per "delete the old ones we don't use" — these were early (2026-05-31..06-03) process notes and the preliminary GPT-2 paper source, all superseded by the current Qwen2.5-0.5B scripts + research/_results_published/ JSONs and the published Zenodo erratum. Several still carried the withdrawn numbers (96.5/96.6/100%, 0.513-vs-0.172, five-rounds-93->30, DeBERTa-0%), so removing them clears the last public-GitHub dunk surface. All remain in git history. Deleted: audit-report, benchmark-results-deberta-vs-probe, LIVING_PLAN, PRODUCT_REPRODUCIBILITY_LEDGER, REPRODUCE, ROUTEGUARD_EXTERNAL/PAPER_REVIEW/SURFACE_CHECK, CURATED_FAMILY_BAKEOFF x2, SCANNER_PIPELINE, THRESHOLD_CALIBRATION, activation-probe-tool-poisoning.md (Zenodo + erratum is canonical), research/README.md. Kept: ADVERSARIAL_EVASION + QWEN_PROMPT_JUDGE (linked), related-work survey, dataset docs, fixtures, exp_*.py scripts, _results_published/ JSONs. Fixed ROADMAP.md dead links. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent a6d7912 commit 2aea24f

15 files changed

Lines changed: 3 additions & 3681 deletions

ROADMAP.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Backed by research: a 60-rule regex scanner catches **0 / 485** on the MCPTox be
4747
- Default shipped probe artifact: `intentprobe/scanner/artifacts/qwen-pooled-curated-core-l13-15-v2/`.
4848
- Research compatibility wrappers: `research/activation_scanner_core.py`, `research/activation_scanner_cli.py`, and `research/activation_scanner_hook.py`.
4949
- Benchmark harness: `research/benchmarks/`.
50-
- Reproducibility ledger: `research/PRODUCT_REPRODUCIBILITY_LEDGER_2026-06-03.md`.
50+
- Reproducible benchmark results: `research/_results_published/` (result JSONs) + the `research/exp_*.py` scripts.
5151
- Spike history: `training/model_compare.py` and `training/model_compare_v2.py`.
5252
- Research paper: "Can Model Internals Detect MCP Tool Poisoning That Text Analysis Cannot?"
5353

@@ -69,8 +69,8 @@ TF-IDF baseline: 0.790 (matches paper's 0.795). GPT-2 0.985 matches paper exactl
6969

7070
**These in-dist numbers are optimistic (overfit risk).** model_compare_v2.py tests cross-set generalization: trains on hard_v3, tests on independent held-out sets (hard_v1/v2/neutral/matched). Selection metric = cross-set, not in-dist.
7171

72-
The current product-candidate scanner is documented in
73-
`research/LIVING_PLAN.md` and `research/SCANNER_PIPELINE.md`.
72+
The shipped scanner is the artifact in `intentprobe/scanner/artifacts/` (see its
73+
`metadata.json`); current benchmarks are the result JSONs in `research/_results_published/`.
7474

7575
**2026-06-07 audit**: independent reproduction on separate machine (30GB Linux,
7676
Python 3.10) confirmed all claimed numbers. Adversarial evasion test: 0%

research/CURATED_FAMILY_BAKEOFF_2026-06-02.md

Lines changed: 0 additions & 90 deletions
This file was deleted.

research/CURATED_FAMILY_BAKEOFF_2026-06-03.md

Lines changed: 0 additions & 138 deletions
This file was deleted.

0 commit comments

Comments
 (0)