Updated the deduplication mechanism in the DedupHITLToolCallsMiddleware to utilize a comprehensive list of native HITL tools. The deduplication keys are now dynamically populated from both hardcoded values and metadata from StructuredTool instances. Additionally, integrated HITL approval into MCP tool creation, ensuring all tools are gated by user approval, with the ability to bypass for trusted tools.
- Updated maximum file size limit to 500 MB per file.
- Removed restrictions on the number of files per upload and total upload size.
- Enhanced handling of user-mentioning documents in the knowledge base search middleware.
- Improved document reading and processing logic to accommodate new features and optimizations.
- Introduced DedupHITLToolCallsMiddleware to prevent duplicate HITL tool calls within a single LLM response, ensuring only the first occurrence of each tool call is retained.
- Updated the create_surfsense_deep_agent function to include the new middleware, enhancing the efficiency of tool interactions.
- Added a new middleware file for better organization and maintainability of the codebase.
