apunkt/flakestorm
1
0
Fork 0
mirror of https://github.com/flakestorm/flakestorm.git synced 2026-04-25 00:36:54 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update README.md to reflect the addition of 24 mutation types, enhancing clarity on core and advanced prompt-level attacks as well as system/network-level attacks. Introduce a new support section with issue templates for better user engagement.

Browse source
This commit is contained in:
Entropix 2026-01-15 13:41:01 +08:00
parent d4ccb2d2c8
commit 611dd82229
1 changed files with 12 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

19
README.md
View file

@ -102,9 +102,10 @@ Flakestorm is built for production-grade agents handling real traffic. While it
Flakestorm follows a simple but powerful workflow:
1. **You provide "Golden Prompts"** — example inputs that should always work correctly
2. **Flakestorm generates mutations** — using a local LLM, it creates adversarial variations across 22+ mutation types:
- **Prompt-level**: Paraphrases, typos, tone shifts, prompt injections, encoding attacks, context manipulation, length extremes, multi-turn attacks, advanced jailbreaks, semantic similarity attacks, format poisoning, language mixing, token manipulation, temporal attacks
- **System/Network-level**: HTTP header injection, payload size attacks, content-type confusion, query parameter poisoning, request method attacks, protocol-level attacks, resource exhaustion, concurrent patterns, timeout manipulation
2. **Flakestorm generates mutations** — using a local LLM, it creates adversarial variations across 24 mutation types:
- **Core prompt-level (8)**: Paraphrase, noise, tone shift, prompt injection, encoding attacks, context manipulation, length extremes, custom
- **Advanced prompt-level (7)**: Multi-turn attacks, advanced jailbreaks, semantic similarity attacks, format poisoning, language mixing, token manipulation, temporal attacks
- **System/Network-level (9)**: HTTP header injection, payload size attacks, content-type confusion, query parameter poisoning, request method attacks, protocol-level attacks, resource exhaustion, concurrent patterns, timeout manipulation
3. **Your agent processes each mutation** — Flakestorm sends them to your agent endpoint
4. **Invariants are checked** — responses are validated against rules you define (latency, content, safety)
5. **Robustness Score is calculated** — weighted by mutation difficulty and importance
@ -116,9 +117,10 @@ The result: You know exactly how your agent will behave under stress before user
## Features
- ✅ **22+ Core Mutation Types**: Comprehensive robustness testing covering:
- **Prompt-level attacks**: Paraphrase, noise, tone shift, prompt injection, encoding, context manipulation, length extremes, multi-turn attacks, advanced jailbreaks, semantic similarity, format poisoning, language mixing, token manipulation, temporal attacks
- **System/Network-level attacks**: HTTP header injection, payload size attacks, content-type confusion, query parameter poisoning, request method attacks, protocol-level attacks, resource exhaustion, concurrent patterns, timeout manipulation
- ✅ **24 Mutation Types**: Comprehensive robustness testing covering:
- **Core prompt-level attacks (8)**: Paraphrase, noise, tone shift, prompt injection, encoding attacks, context manipulation, length extremes, custom
- **Advanced prompt-level attacks (7)**: Multi-turn attacks, advanced jailbreaks, semantic similarity attacks, format poisoning, language mixing, token manipulation, temporal attacks
- **System/Network-level attacks (9)**: HTTP header injection, payload size attacks, content-type confusion, query parameter poisoning, request method attacks, protocol-level attacks, resource exhaustion, concurrent patterns, timeout manipulation
- ✅ **Invariant Assertions**: Deterministic checks, semantic similarity, basic safety
- ✅ **Beautiful Reports**: Interactive HTML reports with pass/fail matrices
- ✅ **Open Source Core**: Full chaos engine available locally for experimentation and CI
@ -126,7 +128,7 @@ The result: You know exactly how your agent will behave under stress before user
## Open Source vs Cloud
**Open Source (Always Free):**
- Core chaos engine with all 22+ mutation types (no artificial feature gating)
- Core chaos engine with all 24 mutation types (no artificial feature gating)
- Local execution for fast experimentation
- CI-friendly usage without external dependencies
- Full transparency and extensibility
@ -199,6 +201,9 @@ See what's coming next! Check out our [Roadmap](ROADMAP.md) for upcoming feature
- [🔧 Fix Installation Issues](FIX_INSTALL.md) - Resolve `ModuleNotFoundError: No module named 'flakestorm.reports'`
- [🔨 Fix Build Issues](BUILD_FIX.md) - Resolve `pip install .` vs `pip install -e .` problems
### Support
- [🐛 Issue Templates](https://github.com/flakestorm/flakestorm/tree/main/.github/ISSUE_TEMPLATE) - Use our issue templates to report bugs, request features, or ask questions
### Reference
- [📋 API Specification](docs/API_SPECIFICATION.md) - API reference
- [🧪 Testing Guide](docs/TESTING_GUIDE.md) - How to run and write tests