8 commits

Author	SHA1	Message	Date
Anish Sarkar	d9df63f57e	refactor: enhance web crawling functionality with Firecrawl integration ... - Updated WebCrawlerConnector to prioritize Firecrawl API for crawling if an API key is provided, falling back to Chromium if Firecrawl fails. - Improved error handling to log failures from both Firecrawl and Chromium. - Enhanced link preview tool to use a random User-Agent for better compatibility with web servers. - Passed Firecrawl API key to the stream_new_chat function for improved configuration management.	2025-12-26 02:37:20 +05:30
CREDO23	64cd65bc1f	use trafilatura to extrack page content from the chromium result	2025-12-19 10:05:51 +02:00
CREDO23	1f60d1c22f	add user agent to AsyncChromiumLoader	2025-12-17 19:43:54 +02:00
CREDO23	4cfeffb38a	refactor: update the webcrawler connector formater	2025-12-17 18:42:37 +02:00
DESKTOP-RTLN3BA\$punk	0b1ca97acf	refactor(webcrawler): update scraping logic to use v2 API and improve error handling	2025-11-26 14:30:08 -08:00
DESKTOP-RTLN3BA\$punk	8f30cfd69a	chore(lint): ruff checks	2025-11-26 13:22:31 -08:00
samkul-swe	6d19e0fad8	Fixing search logic	2025-11-22 13:33:16 -08:00
samkul-swe	896e410e2a	Webcrawler connector draft	2025-11-21 23:27:21 -08:00