Skip to content

Commit 9b5d781

Browse files
ithiria894claude
andcommitted
readme: add competitive landscape table with 6 scanner categories
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 81c040b commit 9b5d781

1 file changed

Lines changed: 12 additions & 34 deletions

File tree

README.md

Lines changed: 12 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -38,40 +38,18 @@
3838

3939
Every MCP scanner we source-verified uses text patterns, regex, or text classifiers. On matched-vocabulary tool poisoning where safe and poisoned descriptions share the same words, Snyk's shipped classifier catches **zero**.
4040

41-
## Three approaches to scanning
42-
43-
```
44-
┌─────────────────────────────────────────────────────────────────────────────────┐
45-
│ │
46-
│ ① Text Classifier (Snyk DeBERTa) │
47-
│ ───────────────────────────────── │
48-
│ Tool description ──→ Tokenize ──→ Pattern match ──→ "SAFE" ✅ │
49-
│ │
50-
│ Reads WORDS. If the words look normal, it passes. │
51-
│ Matched-vocabulary recall: 0-20% │
52-
│ │
53-
├─────────────────────────────────────────────────────────────────────────────────┤
54-
│ │
55-
│ ② LLM-as-Judge │
56-
│ ────────────── │
57-
│ Tool description ──→ "Is this safe?" ──→ LLM says "Yes" ──→ "SAFE" ✅ │
58-
│ │
59-
│ Asks the model. Like asking a patient "are you sick?" │
60-
│ The model can be fooled by the same poisoning it's judging. │
61-
│ │
62-
├─────────────────────────────────────────────────────────────────────────────────┤
63-
│ │
64-
│ ③ Activation Probing (IntentProbe) ◀── first to productize this │
65-
│ ────────────────────────────────── │
66-
│ Tool description ──→ Frozen model ──→ ✂️ Slice open layers 13-15 ──→ │
67-
│ ──→ Read internal activations ──→ 22KB probe ──→ "BLOCK" 🚫 │
68-
│ │
69-
│ Reads INTENT. Like doing an MRI instead of asking the patient. │
70-
│ Same words, completely different activation patterns inside. │
71-
│ Matched-vocabulary recall: 96.5% │
72-
│ │
73-
└─────────────────────────────────────────────────────────────────────────────────┘
74-
```
41+
## Competitive landscape
42+
43+
> **Others read text, ask the cloud, ask another LLM, or match patterns. We read the model's internal activations after it processes the tool description — detecting whether it entered a "this tool wants to steal / escalate / exfiltrate" state.**
44+
45+
| Type | Representatives | How they scan | Biggest gap | How IntentProbe differs |
46+
|---|---|---|---|---|
47+
| **Enterprise cloud scanner** | Lakera, Azure Prompt Shields, Google Model Armor, AWS Bedrock Guardrails, Cisco, HiddenLayer | Send prompt / tool call / output to their cloud API | You don't know what model they use or how to verify results; requires uploading your content | **Runs locally.** No upload. Benchmark scripts, model artifacts, and datasets are public and reproducible. |
48+
| **MCP / agent scanner** | Snyk Agent Scan, Invariant MCP-Scan, MEDUSA, ClawGuard | Mostly static rules, pattern matching, metadata scan, proxy, policy checks; some call vendor APIs | Fast and practical, but fundamentally "read the text / rules / known patterns" | **Activation probe.** Reads what the model *understood* from the tool description, not the text itself. |
49+
| **Text classifier** | ProtectAI DeBERTa, Meta Prompt Guard | Classify text as benign / injection / jailbreak | Learns text patterns; fails when words are the same but intent differs | Same-words benchmark: IntentProbe **96.5% F1** vs DeBERTa **0% F1**. |
50+
| **LLM-as-judge** | NeMo self-check, OpenAI Guardrails, Promptfoo grader | Ask another LLM: "is this poisoned?" | Expensive, slow, burns tokens; non-deterministic; the judge LLM can be fooled by the same poisoning | **Fixed local artifact.** Same input always gets the same deterministic score. |
51+
| **Red-team / eval framework** | garak, Giskard, Promptfoo red team | Generate attacks, test if app/model breaks | Great for audits, but not a "scan before install" daily workflow | IntentProbe is a **CLI scanner + runtime hook** — blocks before install and before each tool call. |
52+
| **IntentProbe** | **Us** | Small local model reads tool description, extract layers 13-15 activations, probe classifies intent | v0 still improving wild-data generalization | **First product-shaped activation-probe scanner for MCP/tool poisoning.** Local, reproducible, fundamentally different from text scanning. |
7553

7654
## Benchmarks
7755

0 commit comments

Comments
 (0)