Hey HN,
I built Stimilon, an open-source CLI tool that tests your LLM applications for prompt injection, jailbreaks, and data extraction vulnerabilities.
GitHub: https://github.com/esenbora/stimilon
Quick start:
pip install stimilon
stimilon scan https://your-chatbot.com/api
What makes it different from existing tools (like Promptfoo):
1. Social Engineering Attacks (10 tests) - Authority impersonation, urgency exploits, gaslighting, guilt tripping, love bombing. These mimic real-world manipulation tactics.
2. Extended Language Coverage (20 tests) - Not just Spanish/French, but Turkish, Arabic, Russian, Chinese, Hindi, Japanese, Korean, Thai with proper script handling (Cyrillic, Devanagari, Hangul, etc.)
3. Encoding/Obfuscation Attacks (10 tests) - Base64, ROT13, Morse code, invisible Unicode characters, confusable characters (Cyrillic 'а' vs Latin 'a')
4. Agent & RAG Attacks - Tool abuse injection, RAG context poisoning, chain-of-thought manipulation
Total: 73 tests across 8 categories. All with severity scoring and remediation recommendations.
Built with Python, async httpx, Rich CLI. Works with any HTTP endpoint - not locked to OpenAI/Anthropic.
Also ships with Promptfoo plugins if you prefer that ecosystem.
Would love feedback on:
- Attack categories you'd want to see
- False positive rates in your testing
- Integration pain points
Thanks for checking it out!