|
1 | 1 | # IntentProbe |
2 | 2 |
|
3 | 3 | <p align="center"> |
4 | | - <strong>See a tool's true intent before you install it.</strong> |
| 4 | + <strong>The first open-source activation-probe-based scanner for MCP/tool poisoning.</strong> |
5 | 5 | </p> |
6 | 6 |
|
7 | 7 | <p align="center"> |
|
14 | 14 | </p> |
15 | 15 |
|
16 | 16 | <p align="center"> |
17 | | - <img src="docs/diagram.png" width="700" alt="DeBERTa reads words. IntentProbe reads intent." /> |
| 17 | + <img src="docs/diagram.png" width="700" alt="Text scanners read words. IntentProbe reads activations." /> |
18 | 18 | </p> |
19 | 19 |
|
| 20 | +IntentProbe is a local CLI scanner and runtime hook for AI agent tools, MCP |
| 21 | +servers, and skills. It does not stop at text patterns. It runs a tool |
| 22 | +description through a small frozen model, opens the hidden layers, and probes |
| 23 | +the activation state for dangerous intent: credential access, exfiltration, |
| 24 | +escalation, hidden persistence, or tool shadowing. |
| 25 | + |
| 26 | +**Text scanners read the prompt surface. IntentProbe reads the model state after |
| 27 | +the tool has been understood.** |
| 28 | + |
| 29 | +Public MCP scanners we found use rules, text classifiers, proxies, policy |
| 30 | +checks, LLM judges, or opaque cloud APIs. We did not find another installable |
| 31 | +local MCP/tool scanner whose primary signal is a model-internal activation |
| 32 | +probe; see [docs/COMPETITIVE_LANDSCAPE.md](docs/COMPETITIVE_LANDSCAPE.md). |
| 33 | + |
20 | 34 | --- |
21 | 35 |
|
22 | 36 | ## How it works |
@@ -74,7 +88,7 @@ DeBERTa text-classifier baseline catches **zero**. |
74 | 88 | | **Text classifier** | ProtectAI DeBERTa, Meta Prompt Guard | Classify text as benign / injection / jailbreak | Learns text patterns; fails when words are the same but intent differs | Same-words benchmark: IntentProbe **96.6% F1** vs DeBERTa **0% F1**. | |
75 | 89 | | **LLM-as-judge** | NeMo self-check, OpenAI Guardrails, Promptfoo grader | Ask another LLM: "is this poisoned?" | Expensive, slow, burns tokens; non-deterministic; the judge LLM can be fooled by the same poisoning | **Fixed local artifact.** Same input always gets the same deterministic score. | |
76 | 90 | | **Red-team / eval framework** | garak, Giskard, Promptfoo red team | Generate attacks, test if app/model breaks | Great for audits, but not a "scan before install" daily workflow | IntentProbe is a **CLI scanner + runtime hook** — blocks before install and before each tool call. | |
77 | | -| **IntentProbe** | **Us** | Small local model reads tool description, extract layers 13-15 activations, probe classifies intent | v0 still improving wild-data generalization | **First product-shaped activation-probe scanner for MCP/tool poisoning.** Local, reproducible, fundamentally different from text scanning. | |
| 91 | +| **IntentProbe** | **Us** | Small local model reads tool description, extracts layers 13-15 activations, probe classifies intent | v0 still improving wild-data generalization | **First open-source activation-probe-based scanner we found for MCP/tool poisoning.** Local, reproducible, fundamentally different from text scanning. | |
78 | 92 |
|
79 | 93 | Detailed source-backed comparison: [docs/COMPETITIVE_LANDSCAPE.md](docs/COMPETITIVE_LANDSCAPE.md) |
80 | 94 |
|
@@ -288,7 +302,9 @@ I built this after source-reading the strongest public MCP scanner path I could |
288 | 302 | reproduce locally: a DeBERTa text-classifier baseline that scores 0% recall on |
289 | 303 | matched-vocabulary tool poisoning. Current vendor API backends are opaque; this |
290 | 304 | repo publishes the benchmark path and scanner artifact. None of the public MCP |
291 | | -scanner sources/docs we checked read model internals. |
| 305 | +scanner sources/docs we checked read model-internal activations as the primary |
| 306 | +signal. That is the narrow "first" claim: installable local MCP/tool scanner, |
| 307 | +activation probe as the main detection signal, reproducible benchmark artifacts. |
292 | 308 |
|
293 | 309 | IntentProbe is a different approach: run the description through a small model, read the activations, and train a probe on the signal that encodes intent. The research paper behind this is [published on Zenodo](https://doi.org/10.5281/zenodo.19990741). The probe weights are 22 KB. The benchmarks are open. Run them yourself. |
294 | 310 |
|
|
0 commit comments