feat: update Celery worker configuration and improve connector deletion process

- Added support for multiple queues in Celery worker configuration.
- Modified connector deletion to handle documents inline instead of using a background task.
- Updated response messages for document creation and connector deletion to reflect new processing status.
- Removed the obsolete connector deletion Celery task file.
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2026-02-16 00:07:23 -08:00
parent 81c70befcf
commit 1849b451a5
8 changed files with 66 additions and 300 deletions

View file

@ -278,8 +278,9 @@ In a new terminal window, start the Celery worker to handle background tasks:
# Make sure you're in the surfsense_backend directory
cd surfsense_backend
# Start Celery worker
uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo
# Start Celery worker (consume both default and connectors queues)
DEFAULT_Q="${CELERY_TASK_DEFAULT_QUEUE:-surfsense}"
uv run celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors"
```
**If using pip/venv:**
@ -293,8 +294,9 @@ source .venv/bin/activate # Linux/macOS
# OR
.venv\Scripts\activate # Windows
# Start Celery worker
celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo
# Start Celery worker (consume both default and connectors queues)
DEFAULT_Q="${CELERY_TASK_DEFAULT_QUEUE:-surfsense}"
celery -A celery_worker.celery_app worker --loglevel=info --concurrency=1 --pool=solo --queues="${DEFAULT_Q},${DEFAULT_Q}.connectors"
```
**Optional: Start Flower for monitoring Celery tasks:**

View file

@ -133,7 +133,10 @@ export const updateConnectorResponse = searchSourceConnector;
export const deleteConnectorRequest = searchSourceConnector.pick({ id: true });
export const deleteConnectorResponse = z.object({
message: z.literal("Search source connector deleted successfully"),
message: z.string(),
status: z.string().optional(),
connector_id: z.number().optional(),
connector_name: z.string().optional(),
});
/**