Refactor documentation and remove CI/CD integration references

- Updated README.md to clarify local testing instructions and added error handling for low scores.
- Removed CI/CD configuration details from CONFIGURATION_GUIDE.md and other documentation files.
- Cleaned up MODULES.md by deleting references to the now-removed github_actions.py.
- Streamlined TEST_SCENARIOS.md and USAGE_GUIDE.md by eliminating CI/CD related sections.
- Adjusted CLI command help text in main.py for clarity on minimum score checks.
This commit is contained in:
Entropix 2025-12-30 16:03:42 +08:00
parent 8d752e9746
commit 0c986e268a
7 changed files with 13 additions and 282 deletions

View file

@ -16,9 +16,8 @@ This comprehensive guide walks you through using flakestorm to test your AI agen
6. [Running Tests](#running-tests)
7. [Understanding Results](#understanding-results)
8. [Integration Patterns](#integration-patterns)
9. [CI/CD Integration](#cicd-integration)
10. [Advanced Usage](#advanced-usage)
11. [Troubleshooting](#troubleshooting)
9. [Advanced Usage](#advanced-usage)
10. [Troubleshooting](#troubleshooting)
---
@ -450,19 +449,6 @@ flakestorm run --quiet
flakestorm run --verbose
```
### CI/CD Mode
```bash
# Fail if score < 0.8
flakestorm run --ci --min-score 0.8
# Exit codes:
# 0 = Score meets threshold
# 1 = Score below threshold
# 2 = Configuration error
# 3 = Runtime error
```
### Individual Commands
```bash
@ -630,103 +616,6 @@ agent = AgentExecutor(...)
---
## CI/CD Integration
### GitHub Actions
Create `.github/workflows/flakestorm.yml`:
```yaml
name: Agent Reliability Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
reliability-test:
runs-on: ubuntu-latest
services:
ollama:
image: ollama/ollama
ports:
- 11434:11434
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install flakestorm
pip install -r requirements.txt
- name: Pull Ollama model
run: |
curl -X POST http://localhost:11434/api/pull \
-d '{"name": "qwen2.5-coder:7b"}'
- name: Start agent
run: |
python -m my_agent &
sleep 5 # Wait for startup
- name: Run flakestorm tests
run: |
flakestorm run --ci --min-score 0.8 --output json
- name: Upload report
uses: actions/upload-artifact@v4
if: always()
with:
name: flakestorm-report
path: reports/
```
### GitLab CI
```yaml
flakestorm-test:
image: python:3.11
services:
- name: ollama/ollama
alias: ollama
variables:
OLLAMA_HOST: "http://ollama:11434"
script:
- pip install flakestorm
- flakestorm run --ci --min-score 0.8
artifacts:
paths:
- reports/
when: always
```
### Pre-commit Hook
Add to `.pre-commit-config.yaml`:
```yaml
repos:
- repo: local
hooks:
- id: flakestorm
name: flakestorm Agent Tests
entry: flakestorm run --ci --min-score 0.8
language: system
pass_filenames: false
always_run: true
```
---
## Advanced Usage
### Custom Mutation Templates