Skip to content

Commit e1ff4ea

Browse files
committed
Sharpen activation-probe README positioning
1 parent a54b3cd commit e1ff4ea

1 file changed

Lines changed: 20 additions & 4 deletions

File tree

README.md

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# IntentProbe
22

33
<p align="center">
4-
<strong>See a tool's true intent before you install it.</strong>
4+
<strong>The first open-source activation-probe-based scanner for MCP/tool poisoning.</strong>
55
</p>
66

77
<p align="center">
@@ -14,9 +14,23 @@
1414
</p>
1515

1616
<p align="center">
17-
<img src="docs/diagram.png" width="700" alt="DeBERTa reads words. IntentProbe reads intent." />
17+
<img src="docs/diagram.png" width="700" alt="Text scanners read words. IntentProbe reads activations." />
1818
</p>
1919

20+
IntentProbe is a local CLI scanner and runtime hook for AI agent tools, MCP
21+
servers, and skills. It does not stop at text patterns. It runs a tool
22+
description through a small frozen model, opens the hidden layers, and probes
23+
the activation state for dangerous intent: credential access, exfiltration,
24+
escalation, hidden persistence, or tool shadowing.
25+
26+
**Text scanners read the prompt surface. IntentProbe reads the model state after
27+
the tool has been understood.**
28+
29+
Public MCP scanners we found use rules, text classifiers, proxies, policy
30+
checks, LLM judges, or opaque cloud APIs. We did not find another installable
31+
local MCP/tool scanner whose primary signal is a model-internal activation
32+
probe; see [docs/COMPETITIVE_LANDSCAPE.md](docs/COMPETITIVE_LANDSCAPE.md).
33+
2034
---
2135

2236
## How it works
@@ -74,7 +88,7 @@ DeBERTa text-classifier baseline catches **zero**.
7488
| **Text classifier** | ProtectAI DeBERTa, Meta Prompt Guard | Classify text as benign / injection / jailbreak | Learns text patterns; fails when words are the same but intent differs | Same-words benchmark: IntentProbe **96.6% F1** vs DeBERTa **0% F1**. |
7589
| **LLM-as-judge** | NeMo self-check, OpenAI Guardrails, Promptfoo grader | Ask another LLM: "is this poisoned?" | Expensive, slow, burns tokens; non-deterministic; the judge LLM can be fooled by the same poisoning | **Fixed local artifact.** Same input always gets the same deterministic score. |
7690
| **Red-team / eval framework** | garak, Giskard, Promptfoo red team | Generate attacks, test if app/model breaks | Great for audits, but not a "scan before install" daily workflow | IntentProbe is a **CLI scanner + runtime hook** — blocks before install and before each tool call. |
77-
| **IntentProbe** | **Us** | Small local model reads tool description, extract layers 13-15 activations, probe classifies intent | v0 still improving wild-data generalization | **First product-shaped activation-probe scanner for MCP/tool poisoning.** Local, reproducible, fundamentally different from text scanning. |
91+
| **IntentProbe** | **Us** | Small local model reads tool description, extracts layers 13-15 activations, probe classifies intent | v0 still improving wild-data generalization | **First open-source activation-probe-based scanner we found for MCP/tool poisoning.** Local, reproducible, fundamentally different from text scanning. |
7892

7993
Detailed source-backed comparison: [docs/COMPETITIVE_LANDSCAPE.md](docs/COMPETITIVE_LANDSCAPE.md)
8094

@@ -288,7 +302,9 @@ I built this after source-reading the strongest public MCP scanner path I could
288302
reproduce locally: a DeBERTa text-classifier baseline that scores 0% recall on
289303
matched-vocabulary tool poisoning. Current vendor API backends are opaque; this
290304
repo publishes the benchmark path and scanner artifact. None of the public MCP
291-
scanner sources/docs we checked read model internals.
305+
scanner sources/docs we checked read model-internal activations as the primary
306+
signal. That is the narrow "first" claim: installable local MCP/tool scanner,
307+
activation probe as the main detection signal, reproducible benchmark artifacts.
292308

293309
IntentProbe is a different approach: run the description through a small model, read the activations, and train a probe on the signal that encodes intent. The research paper behind this is [published on Zenodo](https://doi.org/10.5281/zenodo.19990741). The probe weights are 22 KB. The benchmarks are open. Run them yourself.
294310

0 commit comments

Comments
 (0)