/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/chonkie/chunker/code.py:82: UserWarning: The language is set to `auto`. This would adversely affect the performance of the chunker. Consider setting the `language` parameter to a specific language to improve performance.
warnings.warn("The language is set to `auto`. This would adversely affect the performance of the chunker. " +
INFO: Will watch for changes in these directories: ['/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/app']
[92m12:08:54 - LiteLLM:INFO[0m: vertex_and_google_ai_studio_gemini.py:846 - Warning: Setting temperature < 1.0 for Gemini 3 models (gemini-3-flash-preview) can cause infinite loops, degraded reasoning performance, and failure on complex tasks. Strongly recommended to use temperature = 1.0 (default).
2026-01-31 12:08:54 - LiteLLM - INFO - Warning: Setting temperature < 1.0 for Gemini 3 models (gemini-3-flash-preview) can cause infinite loops, degraded reasoning performance, and failure on complex tasks. Strongly recommended to use temperature = 1.0 (default).
[92m12:08:56 - LiteLLM:INFO[0m: vertex_and_google_ai_studio_gemini.py:846 - Warning: Setting temperature < 1.0 for Gemini 3 models (gemini-3-pro-preview) can cause infinite loops, degraded reasoning performance, and failure on complex tasks. Strongly recommended to use temperature = 1.0 (default).
2026-01-31 12:08:56 - LiteLLM - INFO - Warning: Setting temperature < 1.0 for Gemini 3 models (gemini-3-pro-preview) can cause infinite loops, degraded reasoning performance, and failure on complex tasks. Strongly recommended to use temperature = 1.0 (default).
2026-01-31 12:08:57 - LiteLLM Router - INFO - Trying to fallback b/w models
[92m12:08:57 - LiteLLM Router:ERROR[0m: router.py:1431 - Fallback also failed: litellm.ServiceUnavailableError: litellm.MidStreamFallbackError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'. Received Model Group=auto
Available Model Group Fallbacks=None Original exception: RateLimitError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'
2026-01-31 12:08:57 - LiteLLM Router - ERROR - Fallback also failed: litellm.ServiceUnavailableError: litellm.MidStreamFallbackError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'. Received Model Group=auto
Available Model Group Fallbacks=None Original exception: RateLimitError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'
/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/integrations/custom_logger.py:440: RuntimeWarning: coroutine 'router_cooldown_event_callback' was never awaited
pass
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
[stream_new_chat] Error during chat: litellm.ServiceUnavailableError: litellm.MidStreamFallbackError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'. Received Model Group=auto
Available Model Group Fallbacks=None Original exception: RateLimitError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2071, in make_call
response = await client.post(
^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 190, in async_wrapper
result = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 464, in post
raise e
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 420, in post
response.raise_for_status()
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/httpx/_models.py", line 829, in raise_for_status
httpx.HTTPStatusError: Client error '429 Too Many Requests' for url 'https://generativelanguage.googleapis.com/v1alpha/models/gemini-3-pro-preview:streamGenerateContent?key=AIzaSyCEymJ2DxeFwCiLxC7CEXanyd6NI2BXTT0&alt=sse'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1812, in __anext__
await self.fetch_stream()
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1796, in fetch_stream
self.completion_stream = await self.make_call(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py", line 2077, in make_call
raise VertexAIError(
litellm.llms.vertex_ai.common_utils.VertexAIError: b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 2003, in __anext__
raise exception_type(
^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2340, in exception_type
raise e
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 1332, in exception_type
raise RateLimitError(
litellm.exceptions.RateLimitError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/app/tasks/chat/stream_new_chat.py", line 522, in stream_new_chat
async for event in agent.astream_events(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 1514, in astream_events
async for event in event_stream:
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/tracers/event_stream.py", line 1082, in _astream_events_implementation_v2
await task
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/tracers/event_stream.py", line 1037, in consume_astream
async for _ in event_streamer.tap_output_aiter(run_id, stream):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/tracers/event_stream.py", line 215, in tap_output_aiter
async for chunk in output:
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langgraph/pregel/main.py", line 2971, in astream
async for _ in runner.atick(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langgraph/pregel/_runner.py", line 304, in atick
await arun_with_retry(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langgraph/pregel/_retry.py", line 132, in arun_with_retry
async for _ in task.proc.astream(task.input, config):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langgraph/_internal/_runnable.py", line 839, in astream
output = await asyncio.create_task(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langgraph/_internal/_runnable.py", line 904, in _consume_aiter
async for chunk in it:
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/tracers/event_stream.py", line 192, in tap_output_aiter
first = await anext(output, sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 1587, in atransform
async for ichunk in input:
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 1168, in astream
yield await self.ainvoke(input, config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langgraph/_internal/_runnable.py", line 473, in ainvoke
ret = await self.afunc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 1199, in amodel_node
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 277, in final_normalized
final_result = await result(request, handler)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 261, in composed
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/middleware/todo.py", line 248, in awrap_model_call
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 257, in inner_handler
inner_result = await inner(req, handler)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 261, in composed
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/deepagents/middleware/filesystem.py", line 1029, in awrap_model_call
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 257, in inner_handler
inner_result = await inner(req, handler)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 261, in composed
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/deepagents/middleware/subagents.py", line 557, in awrap_model_call
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 257, in inner_handler
inner_result = await inner(req, handler)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_anthropic/middleware/prompt_caching.py", line 140, in awrap_model_call
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain/agents/factory.py", line 1167, in _execute_model_async
output = await model_.ainvoke(messages)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 425, in ainvoke
llm_result = await self.agenerate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 1132, in agenerate_prompt
return await self.agenerate(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 1090, in agenerate
raise exceptions[0]
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 1316, in _agenerate_with_cache
async for chunk in self._astream(messages, stop=stop, **kwargs):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/app/services/llm_router_service.py", line 483, in _astream
async for chunk in response:
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/router.py", line 1334, in __anext__
return await self._async_generator.__anext__()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/router.py", line 1434, in stream_with_fallbacks
raise fallback_error
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/router.py", line 1378, in stream_with_fallbacks
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/router.py", line 4299, in async_function_with_fallbacks_common_utils
raise original_exception
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/router.py", line 1338, in stream_with_fallbacks
async for item in model_response:
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 2013, in __anext__
raise MidStreamFallbackError(
litellm.exceptions.MidStreamFallbackError: litellm.ServiceUnavailableError: litellm.MidStreamFallbackError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'. Received Model Group=auto
Available Model Group Fallbacks=None Original exception: RateLimitError: litellm.RateLimitError: litellm.RateLimitError: vertex_ai_betaException - b'{\n "error": {\n "code": 429,\n "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_input_token_count, limit: 0, model: gemini-3-pro\\nPlease retry in 2.783688804s.",\n "status": "RESOURCE_EXHAUSTED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Learn more about Gemini API quotas",\n "url": "https://ai.google.dev/gemini-api/docs/rate-limits"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.QuotaFailure",\n "violations": [\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerDayPerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",\n "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerMinute-FreeTier",\n "quotaDimensions": {\n "model": "gemini-3-pro",\n "location": "global"\n }\n },\n {\n "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_input_token_count",\n "quotaId": "GenerateContentInputTokensPerModelPerDay-FreeTier",\n "quotaDimensions": {\n "location": "global",\n "model": "gemini-3-pro"\n }\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.RetryInfo",\n "retryDelay": "2s"\n }\n ]\n }\n}\n'
During task with name 'model' and id 'fb68e6ee-4add-3088-0d98-3607480636bb'
INFO: 127.0.0.1:49379 - "OPTIONS /api/v1/threads/4/full HTTP/1.1" 200 OK
INFO: 127.0.0.1:49381 - "OPTIONS /api/v1/threads/4 HTTP/1.1" 200 OK
INFO: 127.0.0.1:49377 - "GET /api/v1/searchspaces/2/members HTTP/1.1" 200 OK
INFO: 127.0.0.1:49383 - "GET /api/v1/threads/4 HTTP/1.1" 200 OK
INFO: 127.0.0.1:49379 - "GET /api/v1/threads/4/full HTTP/1.1" 200 OK
INFO: 127.0.0.1:49383 - "GET /api/v1/messages/25/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:49381 - "GET /api/v1/notifications/unread-count?search_space_id=2&type=connector_indexing HTTP/1.1" 200 OK
INFO: 127.0.0.1:49379 - "GET /api/v1/messages/23/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:49377 - "GET /api/v1/messages/27/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:49377 - "GET /api/v1/notifications?search_space_id=2&type=connector_indexing&limit=50 HTTP/1.1" 200 OK
INFO: 127.0.0.1:49391 - "POST /api/v1/new_chat HTTP/1.1" 200 OK
INFO: 127.0.0.1:49390 - "POST /api/v1/threads/4/messages HTTP/1.1" 200 OK
2026-01-31 12:09:23 - app.services.llm_router_service - INFO - ChatLiteLLMRouter initialized with 4 models
2026-01-31 12:09:23 - app.agents.new_chat.tools.mcp_tool - INFO - Loaded 0 MCP tools for search space 2
[92m12:09:33 - LiteLLM:INFO[0m: vertex_and_google_ai_studio_gemini.py:846 - Warning: Setting temperature < 1.0 for Gemini 3 models (gemini-3-flash-preview) can cause infinite loops, degraded reasoning performance, and failure on complex tasks. Strongly recommended to use temperature = 1.0 (default).
2026-01-31 12:09:33 - LiteLLM - INFO - Warning: Setting temperature < 1.0 for Gemini 3 models (gemini-3-flash-preview) can cause infinite loops, degraded reasoning performance, and failure on complex tasks. Strongly recommended to use temperature = 1.0 (default).
INFO: 127.0.0.1:58489 - "OPTIONS /users/me HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "OPTIONS /api/v1/global-new-llm-configs HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "OPTIONS /api/v1/search-source-connectors?search_space_id=2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58493 - "OPTIONS /api/v1/search-spaces/2/llm-preferences HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "OPTIONS /api/v1/searchspaces?limit=10&skip=0&owned_only=false HTTP/1.1" 200 OK
INFO: 127.0.0.1:58494 - "OPTIONS /api/v1/searchspaces/2/my-access HTTP/1.1" 200 OK
INFO: 127.0.0.1:58493 - "OPTIONS /api/v1/searchspaces/2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "OPTIONS /api/v1/threads?search_space_id=2&limit=40 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58497 - "OPTIONS /api/v1/threads?search_space_id=2&limit=1 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "OPTIONS /api/v1/searchspaces/2/members HTTP/1.1" 200 OK
INFO: 127.0.0.1:58494 - "OPTIONS /api/v1/new-llm-configs?search_space_id=2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58498 - "OPTIONS /api/v1/documents/type-counts?search_space_id=2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58497 - "OPTIONS /api/v1/messages/23/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58493 - "OPTIONS /api/v1/messages/25/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "OPTIONS /api/v1/messages/27/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "OPTIONS /api/v1/messages/29/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58498 - "OPTIONS /api/v1/messages/31/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "GET /api/v1/global-new-llm-configs HTTP/1.1" 200 OK
INFO: 127.0.0.1:58494 - "GET /users/me HTTP/1.1" 200 OK
INFO: 127.0.0.1:58497 - "GET /api/v1/search-source-connectors?search_space_id=2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "GET /api/v1/searchspaces?limit=10&skip=0&owned_only=false HTTP/1.1" 200 OK
INFO: 127.0.0.1:58494 - "GET /api/v1/searchspaces/2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "GET /api/v1/threads?search_space_id=2&limit=1 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "GET /api/v1/searchspaces/2/members HTTP/1.1" 200 OK
INFO: 127.0.0.1:58494 - "GET /api/v1/documents/type-counts?search_space_id=2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "GET /api/v1/new-llm-configs?search_space_id=2 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58493 - "GET /api/v1/search-spaces/2/llm-preferences HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "GET /api/v1/messages/27/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58493 - "GET /api/v1/messages/29/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58489 - "GET /api/v1/messages/25/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58494 - "GET /api/v1/messages/23/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:58498 - "GET /api/v1/searchspaces/2/my-access HTTP/1.1" 200 OK
INFO: 127.0.0.1:58497 - "GET /api/v1/threads?search_space_id=2&limit=40 HTTP/1.1" 200 OK
INFO: 127.0.0.1:58490 - "GET /api/v1/messages/31/comments HTTP/1.1" 200 OK
INFO: 127.0.0.1:50485 - "GET /health HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50506 - "POST /api/auth/register HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50535 - "POST /api/auth/login HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50544 - "GET /docs HTTP/1.1" 200 OK
INFO: 127.0.0.1:50554 - "GET /openapi.json HTTP/1.1" 200 OK
INFO: 127.0.0.1:50580 - "GET /openapi.json HTTP/1.1" 200 OK
INFO: 127.0.0.1:50625 - "POST /connectors/dexscreener/add HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50643 - "POST /api/v1/connectors/dexscreener/add HTTP/1.1" 404 Not Found
INFO: 127.0.0.1:50655 - "GET /openapi.json HTTP/1.1" 200 OK
INFO: Shutting down
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.
INFO: Finished server process [59799]
/Users/mac_1/.local/share/uv/python/cpython-3.12.9-macos-aarch64-none/lib/python3.12/multiprocessing/resource_tracker.py:255: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/chonkie/chunker/code.py:82: UserWarning: The language is set to `auto`. This would adversely affect the performance of the chunker. Consider setting the `language` parameter to a specific language to improve performance.
warnings.warn("The language is set to `auto`. This would adversely affect the performance of the chunker. " +
INFO: Will watch for changes in these directories: ['/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/app']
WHERE search_source_connectors.search_space_id = $1::INTEGER AND search_source_connectors.user_id = $2::UUID AND search_source_connectors.connector_type = $3::searchsourceconnectortype]
(Background on this error at: https://sqlalche.me/e/20/dbapi)
Traceback (most recent call last):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 545, in _prepare_and_execute
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
data = await self.__bind_execute(args, 0, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 267, in __bind_execute
data, status, _ = await self.__do_execute(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 256, in __do_execute
return await executor(protocol)
^^^^^^^^^^^^^^^^^^^^^^^^
File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncpg.exceptions.InvalidTextRepresentationError: invalid input value for enum searchsourceconnectortype: "DEXSCREENER_CONNECTOR"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1963, in _exec_single_context
self.dialect.do_execute(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute
cursor.execute(statement, parameters)
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
self._adapt_connection.await_(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
value = await result
^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 558, in _prepare_and_execute
self._handle_exception(error)
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 508, in _handle_exception
self._adapt_connection._handle_exception(error)
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 792, in _handle_exception
raise translated_error from error
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.Error: <class 'asyncpg.exceptions.InvalidTextRepresentationError'>: invalid input value for enum searchsourceconnectortype: "DEXSCREENER_CONNECTOR"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/app/routes/dexscreener_add_connector_route.py", line 79, in add_dexscreener_connector
result = await session.execute(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
result = await greenlet_spawn(
^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
result = context.throw(*sys.exc_info())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1415, in execute
return meth(
^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 523, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1637, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1982, in _exec_single_context
self._handle_dbapi_exception(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2351, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1963, in _exec_single_context
self.dialect.do_execute(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 943, in do_execute
cursor.execute(statement, parameters)
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
self._adapt_connection.await_(
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
value = await result
^^^^^^^^^^^^
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 558, in _prepare_and_execute
self._handle_exception(error)
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 508, in _handle_exception
self._adapt_connection._handle_exception(error)
File "/Users/mac_1/Documents/GitHub/SurfSense/surfsense_backend/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 792, in _handle_exception
raise translated_error from error
sqlalchemy.exc.DBAPIError: (sqlalchemy.dialects.postgresql.asyncpg.Error) <class 'asyncpg.exceptions.InvalidTextRepresentationError'>: invalid input value for enum searchsourceconnectortype: "DEXSCREENER_CONNECTOR"
WHERE search_source_connectors.search_space_id = $1::INTEGER AND search_source_connectors.user_id = $2::UUID AND search_source_connectors.connector_type = $3::searchsourceconnectortype]
INFO: 127.0.0.1:51170 - "POST /auth/jwt/login HTTP/1.1" 200 OK
2026-01-31 17:10:00 - app.routes.dexscreener_add_connector_route - INFO - Successfully created DexScreener connector for user f34b4612-6556-47c8-bd37-c0e1fd6c9b30 with ID 3
INFO: 127.0.0.1:51171 - "POST /api/v1/connectors/dexscreener/add HTTP/1.1" 200 OK
INFO: 127.0.0.1:51172 - "GET /api/v1/connectors/dexscreener HTTP/1.1" 405 Method Not Allowed
INFO: 127.0.0.1:51257 - "POST /auth/jwt/login HTTP/1.1" 200 OK
2026-01-31 17:10:42 - app.routes.dexscreener_add_connector_route - INFO - Updated existing DexScreener connector for user f34b4612-6556-47c8-bd37-c0e1fd6c9b30 in space 1
INFO: 127.0.0.1:51258 - "POST /api/v1/connectors/dexscreener/add HTTP/1.1" 200 OK
2026-01-31 17:10:43 - app.connectors.dexscreener_connector - INFO - Token not found: tokens/ethereum/0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2
INFO: 127.0.0.1:51259 - "GET /api/v1/connectors/dexscreener/test?space_id=1 HTTP/1.1" 400 Bad Request
INFO: 127.0.0.1:52249 - "POST /auth/jwt/login HTTP/1.1" 200 OK
2026-01-31 17:23:19 - app.routes.dexscreener_add_connector_route - INFO - Updated existing DexScreener connector for user f34b4612-6556-47c8-bd37-c0e1fd6c9b30 in space 1
INFO: 127.0.0.1:52250 - "POST /api/v1/connectors/dexscreener/add HTTP/1.1" 200 OK
2026-01-31 17:23:19 - app.connectors.dexscreener_connector - INFO - Token not found: tokens/ethereum/0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2
INFO: 127.0.0.1:52251 - "GET /api/v1/connectors/dexscreener/test?space_id=1 HTTP/1.1" 400 Bad Request
