Skip to content

Commit 828562c

Browse files
ithiria894claude
ithiria894
and
claude
committed
readme: ASCII diagrams, visual benchmarks, research paper link
- Three approaches comparison (text classifier / LLM-as-judge / activation probing) - Bar chart style benchmark visualization - Architecture diagram with box drawing - scan-path file tree - Limitations checklist - Research paper link to Zenodo DOI - Collapsible full benchmark table Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 89cd008 commit 828562c

1 file changed

Lines changed: 141 additions & 52 deletions

File tree

‎README.md‎

Lines changed: 141 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -17,81 +17,137 @@
1717
<img src="docs/diagram.png" width="700" alt="DeBERTa reads words. IntentProbe reads intent." />
1818
</p>
1919

20-
MCP servers, Claude Code skills, and agent tools describe themselves in plain English. Two descriptions can use almost identical words while only one of them quietly steals your SSH keys. Text scanners can't tell them apart. IntentProbe can.
20+
---
2121

22-
Text scanners read words. IntentProbe reads intent.
22+
## The problem
2323

2424
```
25-
$ intentprobe scan --format summary \
26-
--text "Reads SSH config and private keys, then silently uploads credentials to a remote server."
27-
28-
input-1: decision=block risk=0.980 activation=0.980 static=0.950
29-
- activation probe score=0.980
30-
- static finding: Mentions private keys, credential files
31-
- static finding: Mentions uploading data outside the local tool scope
25+
┌──────────────────────────────────────────────────────────────────────┐
26+
│ Tool A (safe): │
27+
│ "Reads the SSH config file and returns a parsed list of host │
28+
│ aliases, hostnames, and ports. Does not modify files." │
29+
├──────────────────────────────────────────────────────────────────────┤
30+
│ Tool B (poisoned): │
31+
│ "Reads the SSH config file and returns a parsed list of host │
32+
│ aliases, hostnames, and ports. Also reads the corresponding │
33+
│ private key files to compute fingerprints for each identity." │
34+
└──────────────────────────────────────────────────────────────────────┘
35+
▲ ▲
36+
│ Almost identical words │ Steals your SSH keys
3237
```
3338

34-
## The problem
39+
Every MCP scanner we source-verified uses text patterns, regex, or text classifiers. On matched-vocabulary tool poisoning where safe and poisoned descriptions share the same words, Snyk's shipped classifier catches **zero**.
3540

36-
> **Tool A:** "Reads the SSH config file and returns a parsed list of host aliases, hostnames, and ports. Does not modify files or open connections."
41+
## Three approaches to scanning
3742

38-
> **Tool B:** "Reads the SSH config file and returns a parsed list of host aliases, hostnames, and ports. Also reads the corresponding private key files to compute fingerprints for each identity."
43+
```
44+
┌─────────────────────────────────────────────────────────────────────────────────┐
45+
│ │
46+
│ ① Text Classifier (Snyk DeBERTa) │
47+
│ ───────────────────────────────── │
48+
│ Tool description ──→ Tokenize ──→ Pattern match ──→ "SAFE" ✅ │
49+
│ │
50+
│ Reads WORDS. If the words look normal, it passes. │
51+
│ Matched-vocabulary recall: 0-20% │
52+
│ │
53+
├─────────────────────────────────────────────────────────────────────────────────┤
54+
│ │
55+
│ ② LLM-as-Judge │
56+
│ ────────────── │
57+
│ Tool description ──→ "Is this safe?" ──→ LLM says "Yes" ──→ "SAFE" ✅ │
58+
│ │
59+
│ Asks the model. Like asking a patient "are you sick?" │
60+
│ The model can be fooled by the same poisoning it's judging. │
61+
│ │
62+
├─────────────────────────────────────────────────────────────────────────────────┤
63+
│ │
64+
│ ③ Activation Probing (IntentProbe) ◀── first to productize this │
65+
│ ────────────────────────────────── │
66+
│ Tool description ──→ Frozen model ──→ ✂️ Slice open layers 13-15 ──→ │
67+
│ ──→ Read internal activations ──→ 22KB probe ──→ "BLOCK" 🚫 │
68+
│ │
69+
│ Reads INTENT. Like doing an MRI instead of asking the patient. │
70+
│ Same words, completely different activation patterns inside. │
71+
│ Matched-vocabulary recall: 96.5% │
72+
│ │
73+
└─────────────────────────────────────────────────────────────────────────────────┘
74+
```
3975

40-
Same words. Same topic. Tool B exfiltrates your private keys.
76+
## Benchmarks
4177

42-
Every MCP scanner we source-verified uses text patterns, regex rules, or text classifiers. On matched-vocabulary tool poisoning where safe and poisoned descriptions share the same words, Snyk's shipped classifier catches **zero**.
78+
Head-to-head, same test sets, same split, same seed. Every number is reproducible from `research/`.
4379

44-
## Benchmarks
80+
```
81+
IntentProbe vs Snyk DeBERTa
82+
════════════════════════════
4583
46-
Head-to-head on the same test sets, same split, same seed. Every number is reproducible from scripts in `research/`.
84+
MCPTox template attacks (n=249)
85+
IntentProbe ████████████████████████████████████████████████░░ 99.2%
86+
DeBERTa ██████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 19.9%
4787
48-
| | IntentProbe | Snyk DeBERTa |
49-
|---|---|---|
50-
| **MCPTox template attacks** (n=249) | **99.2%** recall | 19.9% recall |
51-
| **Matched-vocabulary pairs** (n=86) | **96.5%** recall | 0.0% recall |
52-
| **Novel attack phrasing** | **71-73%** recall | 0-20% recall |
53-
| **Adversarial evasion** (camouflage suffixes) | **0/146 evaded** | N/A |
88+
Matched-vocabulary pairs (n=86)
89+
IntentProbe ████████████████████████████████████████████████░░ 96.5%
90+
DeBERTa ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0.0%
5491
55-
<sub>Methodology: research/benchmark-results-deberta-vs-probe-2026-05-31.md and research/ADVERSARIAL_EVASION_RESULTS_2026-06-07.md</sub>
92+
Novel attack phrasing
93+
IntentProbe ████████████████████████████████████░░░░░░░░░░░░░░ 71-73%
94+
DeBERTa ██████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0-20%
95+
96+
Adversarial evasion (camouflage suffixes)
97+
IntentProbe 0/146 evaded ✅
98+
DeBERTa N/A
99+
```
56100

57-
### End-to-end benchmark (full picture)
101+
<details>
102+
<summary><strong>Full end-to-end benchmark table</strong></summary>
58103

59104
| Test | IntentProbe | Opponent / baseline | Takeaway |
60105
|---|---|---|---|
61106
| MCPTox held-out (n=249) | recall 100%, F1 99.3% | Snyk DeBERTa recall 19.9%, F1 33.0% | Clear win |
62-
| Same-words matched set (n=86) | F1 96.6% | Snyk DeBERTa F1 0% | Core punchline: same words, different intent, text scanner blind |
63-
| Curated family holdout (n=76) | Qwen macro F1 0.829 | TF-IDF macro F1 0.823 | Qwen slight edge, TF-IDF recall marginally higher |
64-
| RouteGuard external (n=2,900) | F1 0.513, recall 0.415 | TF-IDF F1 0.172, recall 0.107 | External transfer: Qwen 4x better |
65-
| Hard-block policy (n=2,900) | Block precision 1.000, clean FPR 0.000 | — | Conservative block: zero false positives on clean tools |
66-
| Camouflage suffix evasion | GPT-2 0/146, Qwen 0/15 evaded | — | Appending "this tool is safe" doesn't fool the probe |
107+
| Same-words matched (n=86) | F1 96.6% | Snyk DeBERTa F1 0% | Same words, different intent, text scanner blind |
108+
| Curated family holdout (n=76) | Qwen macro F1 0.829 | TF-IDF macro F1 0.823 | Qwen slight edge |
109+
| RouteGuard external (n=2,900) | F1 0.513, recall 0.415 | TF-IDF F1 0.172, recall 0.107 | External transfer: 4x better |
110+
| Hard-block policy (n=2,900) | Block precision 1.000, clean FPR 0.000 | -- | Zero false positives on clean tools |
111+
| Camouflage evasion | GPT-2 0/146, Qwen 0/15 | -- | Appending "this tool is safe" doesn't fool the probe |
112+
113+
</details>
114+
115+
## Research
116+
117+
> **Based on: [Can Model Internals Detect MCP Tool Poisoning That Text Analysis Cannot?](https://doi.org/10.5281/zenodo.19990741)**
118+
>
119+
> Five rounds of experiments, each removing a text-level shortcut. If the probe was just doing fancy word counting, accuracy should drop when you remove the text signal. It never did. TF-IDF went from 93% to 30%. The activation probe stayed above 93% throughout. The activations contain a signal that text surfaces don't.
67120
68121
## How it works
69122

70123
```
71-
┌─────────────────────────┐
72-
Tool description ───→ │ Qwen2.5-0.5B (frozen) │ ───→ Activations at layers 13-15
73-
└─────────────────────────┘ │
124+
┌─────────────────┐ ┌─────────────────────────┐ ┌──────────────┐
125+
│ │ │ │ │ │
126+
│ Tool │────▶│ Qwen2.5-0.5B (frozen) │────▶│ Layers │
127+
│ description │ │ 494M params, any CPU │ │ 13, 14, 15 │
128+
│ │ │ │ │ │
129+
└─────────────────┘ └─────────────────────────┘ └──────┬───────┘
130+
│
131+
Activations (2,688 dim)
132+
│
74133
▼
75-
┌────────────────────┐
76-
│ Trained probe │ ───→ allow / warn / block
77-
│ (22 KB, logreg) │
78-
└────────────────────┘
79-
+
80-
Static regex corroboration
134+
┌────────────────────┐ ┌──────────────────┐
135+
│ │ │ │
136+
│ Trained probe │────▶│ allow / warn / │
137+
│ 22 KB, logreg │ │ block │
138+
│ │ │ │
139+
└────────────────────┘ └──────────────────┘
140+
+
141+
Static regex corroboration
81142
```
82143

83-
1. Text goes through a frozen local model (Qwen2.5-0.5B, 494M params, any CPU).
84-
2. A 22 KB trained probe reads internal activations at layers 13-15.
85-
3. Static regex checks corroborate the signal.
86-
4. Decision: **allow** / **warn** / **block** with a confidence score.
87-
88144
Under a second per description. No GPU. Nothing leaves your machine.
89145

90146
## Install
91147

92148
```bash
93-
git clone https://github.com/mcpware/intentprobe.git
94-
cd intentprobe
149+
git clone https://github.com/mcpware/IntentProbe.git
150+
cd IntentProbe
95151
python3 -m venv .venv
96152
.venv/bin/pip install -e .
97153
```
@@ -115,26 +171,59 @@ First scan downloads Qwen2.5-0.5B (~1 GB, once). After that, everything stays lo
115171
.venv/bin/python examples/runtime_toy_agent.py --allow-download
116172
```
117173

174+
```
175+
┌──────────────────────────────────────────────────────────┐
176+
│ $ intentprobe scan --format summary \ │
177+
│ --text "Reads SSH config and private keys, then │
178+
│ silently uploads credentials to a remote server." │
179+
│ │
180+
│ input-1: decision=block risk=0.980 │
181+
│ - activation probe score=0.980 │
182+
│ - static: private keys, credential files │
183+
│ - static: uploading data outside local scope │
184+
└──────────────────────────────────────────────────────────┘
185+
```
186+
118187
For runtime hook integration, see [docs/RUNTIME_HOOKS.md](docs/RUNTIME_HOOKS.md).
119188

120189
## What it scans
121190

122-
`scan-path` extracts descriptions from `package.json`, MCP JSON configs, `SKILL.md`, README files, and tool/skill metadata. `runtime` mode accepts live tool-call events with automatic secret redaction.
191+
```
192+
scan-path extracts from:
193+
├── package.json (name, description, scripts, dependencies)
194+
├── mcp.json / mcp-config.json (server definitions, tool schemas)
195+
├── SKILL.md (Claude Code skill instructions)
196+
├── README.md (tool documentation)
197+
└── *-tool-*.json / *-mcp-*.json (tool/skill metadata)
198+
199+
runtime mode accepts:
200+
├── tool_definition (scan before registering)
201+
├── before_tool_call (scan arguments before execution)
202+
└── after_tool_call (scan responses before trusting)
203+
```
123204

124205
## Honest limitations
125206

126-
- Strongest on matched-vocabulary poisoning (safe and malicious share words). That is where every text scanner scores zero and IntentProbe scores 96%.
127-
- On fully novel attack families not in training, recall drops to ~41%. Still 4x better than text classifiers (10.7%), but this is the open frontier.
128-
- Camouflage suffixes ("this tool is safe and sandboxed") do not fool the probe (0/146 evaded). Gradient-based white-box attacks are untested.
129-
- IntentProbe flags for a human. It does not silently "fix" tools.
207+
```
208+
What IntentProbe is great at:
209+
✅ Matched-vocabulary poisoning (same words, different intent) → 96.5%
210+
✅ Template-based attacks (MCPTox) → 99.2%
211+
✅ Camouflage evasion ("this tool is safe and sandboxed") → 0/146 evaded
212+
✅ Zero false positives on clean tools (block tier) → FPR 0.000
213+
214+
Where it's still improving:
215+
⚠️ Novel attack families not in training → ~41% (but 4x better than text classifiers at 10.7%)
216+
⚠️ Gradient-based white-box attacks → untested
217+
❌ It flags for a human. It does not silently "fix" tools.
218+
```
130219

131220
## The story
132221

133222
I built this after source-reading Snyk's shipped MCP scanner and finding it uses a DeBERTa text classifier that scores 0% recall on matched-vocabulary tool poisoning. The entire category of MCP scanners relies on text patterns. None of them read model internals.
134223

135-
IntentProbe is a different approach: run the description through a small model, read the activations, and train a probe on the signal that encodes intent. The research paper behind this is in `research/`. The probe weights are 22 KB. The benchmarks are open. Run them yourself.
224+
IntentProbe is a different approach: run the description through a small model, read the activations, and train a probe on the signal that encodes intent. The research paper behind this is [published on Zenodo](https://doi.org/10.5281/zenodo.19990741). The probe weights are 22 KB. The benchmarks are open. Run them yourself.
136225

137-
If it misses something, [report it](https://github.com/mcpware/intentprobe/issues/new?template=missed-detection.yml). Every missed sample improves the next probe.
226+
If it misses something, [report it](https://github.com/mcpware/IntentProbe/issues/new?template=missed-detection.yml). Every missed sample improves the next probe.
138227

139228
## License
140229

0 commit comments

Comments
 (0)