apunkt/flakestorm
1
0
Fork 0
mirror of https://github.com/flakestorm/flakestorm.git synced 2026-04-24 16:26:35 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
77 commits 1 branch 4 tags 44 MiB
Commit graph

26 commits

Author SHA1 Message Date
Francisco M Humarang Jr.
f4d45d4053 Update documentation and configuration for Flakestorm V2, enhancing clarity on CI processes, report generation, and reproducibility features. Added details on the new --output option for saving reports, clarified the use of --min-score, and improved descriptions of the seed configuration for deterministic runs. Updated README and usage guides to reflect these changes and ensure comprehensive understanding of the CI pipeline and report outputs. 2026-03-12 20:05:51 +08:00
Francisco M Humarang Jr.
4a13425f8a Update documentation to reflect enhancements in Flakestorm V2, including detailed descriptions of new features such as resilience scores, chaos engineering capabilities, behavioral contracts, and replay regression. Clarified API key management via environment variables, updated CLI commands, and improved test scenarios. Adjusted mutation types count to 22+ and ensured all V2 gaps are closed as per the latest specifications. 2026-03-09 19:52:39 +08:00
Entropix
f1570628c3 Update TEST_SCENARIOS.md to include detailed descriptions of V2 scenarios, specifically focusing on the Research Agent with Search Tool and Support Agent with KB Tool. Enhanced documentation with actual tool calling examples, chaos and contract testing details, and configuration settings for improved clarity and usability. 2026-03-09 13:41:41 +08:00
Entropix
11489255e3 Remove the implementation checklist document and update README and TEST_SCENARIOS to reflect the latest V2 features, including detailed descriptions of environment chaos, behavioral contracts, and replay regression scenarios. Adjusted links and clarified configuration options for better usability. 2026-03-09 13:01:08 +08:00
Entropix
4b0ab63f97 Update README and usage guide to reflect changes in mutation types and clarify V1/V2 flow. Increased mutation types from 22 to 24 and added details on the new V2 features, including environment chaos and behavioral contracts. Enhanced documentation for clarity on scoring mechanisms and command usage. 2026-03-09 12:45:42 +08:00
Francisco M Humarang Jr.
4c1b43c5d5 Enhance documentation for Flakestorm V2 features, including detailed updates on behavioral contracts, context attacks, and scoring mechanisms. Added new configuration options for state isolation in agents, clarified context attack types, and improved the contract report generation with suggested actions for failures. Updated various guides to reflect the latest changes in chaos engineering capabilities and replay regression functionalities. 2026-03-08 20:29:48 +08:00
Francisco M Humarang Jr.
1bbe3a1f7b Enhance documentation and replay functionality in Flakestorm. Updated README to clarify V2 Spec and added references to LangSmith sources in configuration guide. Improved replay regression capabilities by allowing imports from LangSmith projects and runs, with filtering options. Added new classes for LangSmith project and run sources in the configuration. Updated replay loader to support project imports and refined session resolution logic. 2026-03-07 02:04:55 +08:00
Francisco M Humarang Jr.
9c3450a75d Update version to 2.0.0 and enhance chaos engineering features in Flakestorm. Added support for environment chaos, behavioral contracts, and replay regression. Expanded documentation and improved scoring mechanisms. Updated .gitignore to include new documentation files. 2026-03-06 23:33:21 +08:00
Francisco M Humarang Jr.
2f4f2270b5 Update README.md and CONTRIBUTING.md to enhance project visibility and support for new contributors. Added PyPI version and download badges, build status, and latest release information to README.md. Introduced a section in CONTRIBUTING.md for finding good first issues, providing guidance for beginners on how to contribute effectively. 2026-01-13 21:39:50 +08:00
Francisco M Humarang Jr.
d1aaa626c9 Enhance documentation to reflect the addition of 22+ mutation types in Flakestorm, including advanced prompt-level and system/network-level attacks. Update README.md, API_SPECIFICATION.md, CONFIGURATION_GUIDE.md, USAGE_GUIDE.md, and related files to improve clarity on mutation strategies, testing scenarios, and configuration options. Emphasize the importance of comprehensive testing for production AI agents and provide detailed descriptions for each mutation type. 2026-01-05 22:21:27 +08:00
Entropix
d339d5e436 Refactor README.md and USAGE_GUIDE.md to streamline installation instructions and enhance clarity on robustness scoring and mutation strategies. Removed outdated sections and added detailed explanations for mutation types and their applications in testing. This update aims to improve user understanding and facilitate easier setup and usage of Flakestorm. 2026-01-04 23:39:24 +08:00
Francisco M Humarang Jr.
0b8777c614 Update version to 0.9.0 in pyproject.toml and __init__.py, enhance CONFIGURATION_GUIDE.md and USAGE_GUIDE.md with aggressive mutation strategies and requirements for invariants, and add validation to ensure at least 3 invariants are configured in FlakeStormConfig. 2026-01-03 00:18:31 +08:00
Francisco M Humarang Jr.
e673b21b55 Update version to 0.8.0 in pyproject.toml, enhance README.md and USAGE_GUIDE.md with optional Rust extension installation instructions for improved performance, and remove outdated keywords_extractor_agent documentation. 2026-01-02 22:32:18 +08:00
Francisco M Humarang Jr.
2dcaf31712 Update model configuration and enhance documentation for improved user guidance - Change default model to "gemma3:1b" in flakestorm-generate-search-queries.yaml and increase mutation count from 3 to 20 - Revise README.md to include demo visuals and model recommendations based on system RAM - Expand USAGE_GUIDE.md with detailed model selection criteria and installation instructions - Enhance HTML report generation to include actionable recommendations for failed mutations and executive summary insights. 2026-01-02 20:01:12 +08:00
Entropix
661445c7b8 Add pre-flight validation, flexible response handling, and improved error detection - Add pre-flight check to validate agent with first golden prompt before mutations - Improve response extraction to handle various agent response formats automatically - Add support for non-JSON responses (plain text, HTML) - Enhance error detection for HTTP 200 responses with error fields - Add comprehensive auto-detection for common response field names - Improve JSON parsing error handling with graceful fallbacks - Add example YAML config for GenerateSearchQueries agent - Update documentation with build and installation fixes 2026-01-02 15:21:20 +08:00
entropix
c52a28377f Update documentation to clarify the integration process and enhance troubleshooting steps. Revise README.md and USAGE_GUIDE.md to include new integration examples and common error resolutions. Ensure consistency in terminology and provide additional context for users. 2026-01-01 17:46:53 +08:00
entropix
13d18e0428 Add Integrations Guide to README.md and outline Phase 7 roadmap in IMPLEMENTATION_CHECKLIST.md 2026-01-01 17:29:41 +08:00
entropix
844134920a Enhance mutation capabilities by adding three new types: encoding_attacks, context_manipulation, and length_extremes. Update configuration and documentation to reflect the addition of these types, including their weights and descriptions. Revise README.md, API_SPECIFICATION.md, CONFIGURATION_GUIDE.md, and other relevant documents to provide comprehensive coverage of the new mutation strategies and their applications. Ensure all tests are updated to validate the new mutation types. 2026-01-01 17:28:05 +08:00
Entropix
859566ee59 Implement flexible HTTP agent adapter with request templates and connection guides - Add request_template, response_path, method, query_params, and parse_structured_input to AgentConfig - Implement structured input parser for key-value extraction from golden prompts - Implement template engine with variable substitution for {prompt} and {field_name} - Implement response extractor supporting JSONPath and dot notation - Update HTTPAgentAdapter to support all HTTP methods (GET, POST, PUT, PATCH, DELETE) - Add comprehensive connection guide explaining localhost vs public endpoints - Update documentation with examples for TypeScript/JavaScript developers - Add tests for all new features 2025-12-31 23:04:47 +08:00
Entropix
050204ef42 Enhance installation and troubleshooting instructions in README.md, CONTRIBUTING.md, and USAGE_GUIDE.md. Clarified requirements for Python 3.10+, added detailed steps for creating virtual environments, and provided solutions for common issues related to Ollama and flakestorm installations. Updated license information in README.md. 2025-12-30 22:33:47 +08:00
Entropix
dbbdac9d43 Revise installation instructions in README.md, CONTRIBUTING.md, and USAGE_GUIDE.md to clarify the installation order for Ollama and flakestorm. Added detailed platform-specific installation steps for Ollama and emphasized the need for a virtual environment for Python packages. Included troubleshooting tips for common installation issues. 2025-12-30 18:36:42 +08:00
Entropix
cb034b41ff Enhance installation instructions across documentation to emphasize the use of virtual environments for Python. Added details for creating and activating virtual environments in README.md, CONTRIBUTING.md, TEST_SCENARIOS.md, TESTING_GUIDE.md, and USAGE_GUIDE.md. Included pipx installation instructions for CLI use in USAGE_GUIDE.md. 2025-12-30 18:02:36 +08:00
Entropix
1d45fb2981 - Updated class names and import statements to align with the new naming convention. 
- Adjusted test commands and report references to use FlakeStorm terminology.
- Ensured consistency in configuration and runner references throughout the documentation.
 2025-12-30 16:13:29 +08:00
Entropix
0c986e268a Refactor documentation and remove CI/CD integration references 
- Updated README.md to clarify local testing instructions and added error handling for low scores.
- Removed CI/CD configuration details from CONFIGURATION_GUIDE.md and other documentation files.
- Cleaned up MODULES.md by deleting references to the now-removed github_actions.py.
- Streamlined TEST_SCENARIOS.md and USAGE_GUIDE.md by eliminating CI/CD related sections.
- Adjusted CLI command help text in main.py for clarity on minimum score checks.
 2025-12-30 16:03:42 +08:00
Entropix
ee10da0b97 Add comprehensive documentation for flakestorm 
- Introduced multiple new documents including API Specification, Configuration Guide, Contributing Guide, Developer FAQ, Implementation Checklist, Module Documentation, Publishing Guide, Test Scenarios, Testing Guide, and Usage Guide.
- Each document provides detailed instructions, examples, and best practices for using and contributing to flakestorm.
- Enhanced overall project documentation to support users and developers in understanding and utilizing the framework effectively.
 2025-12-29 11:33:01 +08:00
Entropix
69e0f8deeb Fix .gitignore to allow docs files and add documentation files 
- Fix .gitignore pattern: un-ignore docs/ directory first, then ignore docs/*, then un-ignore specific files
- Add all documentation files referenced in README.md:
  - USAGE_GUIDE.md
  - CONFIGURATION_GUIDE.md
  - TEST_SCENARIOS.md
  - MODULES.md
  - DEVELOPER_FAQ.md
  - PUBLISHING.md
  - CONTRIBUTING.md
  - API_SPECIFICATION.md
  - TESTING_GUIDE.md
  - IMPLEMENTATION_CHECKLIST.md
- Pre-commit hooks fixed trailing whitespace and end-of-file formatting
 2025-12-29 11:32:50 +08:00