Add pre-flight validation, flexible response handling, and improved error detection - Add pre-flight check to validate agent with first golden prompt before mutations - Improve response extraction to handle various agent response formats automatically - Add support for non-JSON responses (plain text, HTML) - Enhance error detection for HTTP 200 responses with error fields - Add comprehensive auto-detection for common response field names - Improve JSON parsing error handling with graceful fallbacks - Add example YAML config for GenerateSearchQueries agent - Update documentation with build and installation fixes

This commit is contained in:
Entropix 2026-01-02 15:21:20 +08:00
parent c52a28377f
commit 661445c7b8
8 changed files with 647 additions and 21 deletions

View file

@ -134,17 +134,28 @@ __all__ = ["load_config", "FlakeStormConfig", "FlakeStormRunner", "__version__"]
```bash
# Check pyproject.toml is valid
python -m pip install .
# NOTE: Use editable mode for development, regular install for testing wheel builds
pip install -e . # Editable mode (recommended for development)
# OR test the wheel build process:
python -m pip install build
python -m build --wheel
python -m pip install dist/*.whl
# Verify the package works
flakestorm --version
```
**Important:** If you get `ModuleNotFoundError: No module named 'flakestorm.reports'` when using `pip install .` (non-editable), it means the wheel build didn't include all subpackages. Use `pip install -e .` for development, or ensure `pyproject.toml` has the correct `packages` configuration.
### Step 2: Build the Package
```bash
# Install build tools (if not already installed)
pip install build
# Clean previous builds
rm -rf dist/ build/ *.egg-info
rm -rf dist/ build/ *.egg-info src/*.egg-info
# Build source distribution and wheel
python -m build
@ -153,22 +164,33 @@ python -m build
# dist/
# flakestorm-0.1.0.tar.gz (source)
# flakestorm-0.1.0-py3-none-any.whl (wheel)
# Verify all subpackages are included (especially reports)
unzip -l dist/*.whl | grep "flakestorm/reports"
```
### Step 3: Check the Build
```bash
# Install twine for checking (if not already installed)
pip install twine
# Verify the package contents
twine check dist/*
# List files in the wheel
unzip -l dist/*.whl
# Ensure it contains:
# Ensure it contains all subpackages:
# - flakestorm/__init__.py
# - flakestorm/core/*.py
# - flakestorm/mutations/*.py
# - flakestorm/reports/*.py (important: check this exists!)
# - flakestorm/assertions/*.py
# - etc.
# Quick check for reports module:
unzip -l dist/*.whl | grep "flakestorm/reports"
```
### Step 4: Test on Test PyPI (Recommended)