apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-07-20 23:21:06 +02:00
Code Issues Projects Releases Packages Wiki Activity

changed critical parts

Browse source
This commit is contained in:
tushar 2025-09-29 20:23:12 +05:30
parent cb8b6d04f5
commit a72ad218eb
2 changed files with 19 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

31
GITHUB_CONNECTOR_FIX.md
View file

@ -20,25 +20,20 @@ The GitHub indexing route was trying to update the `last_indexed_at` timestamp i
## Fixes Applied ## Fixes Applied
### 1. Added Missing Import and Function Call ### 1. Added Missing Import (for future use)
**File:** `surfsense_backend/app/tasks/connector_indexers/github_indexer.py` **File:** `surfsense_backend/app/tasks/connector_indexers/github_indexer.py`
```python ```python
# Added import # Added import for consistency with other indexers
from .base import ( from .base import (
check_duplicate_document_by_hash, check_duplicate_document_by_hash,
get_connector_by_id, get_connector_by_id,
logger, logger,
update_connector_last_indexed, # <- Added this update_connector_last_indexed, # <- Added this for future flexibility
) )
# Added before commit (around line 305) # Note: The update_last_indexed logic is handled by the route layer
# Update the last_indexed_at timestamp for the connector only if requested # following the established pattern used by other connectors
if update_last_indexed:
await update_connector_last_indexed(session, connector, update_last_indexed)
# Commit all changes at the end
await session.commit()
``` ```
### 2. Fixed Timezone-Aware Timestamps ### 2. Fixed Timezone-Aware Timestamps
@ -62,11 +57,13 @@ from datetime import UTC, datetime, timedelta
connector.last_indexed_at = datetime.now(UTC) # <- Added UTC connector.last_indexed_at = datetime.now(UTC) # <- Added UTC
``` ```
### 3. Fixed Session Management ### 3. Proper Session Management Pattern
**File:** `surfsense_backend/app/routes/search_source_connectors_routes.py` **File:** `surfsense_backend/app/routes/search_source_connectors_routes.py`
**Solution:** Follow the established pattern used by other connectors: let the route handle the timestamp update after successful indexing.
```python ```python
# Let the indexer handle timestamp updates within its own transaction # Follow the standard pattern used by other connectors
indexed_count, error_message = await index_github_repos( indexed_count, error_message = await index_github_repos(
session, session,
connector_id, connector_id,
@ -74,10 +71,16 @@ indexed_count, error_message = await index_github_repos(
user_id, user_id,
start_date, start_date,
end_date, end_date,
update_last_indexed=True, # <- Changed from False to True update_last_indexed=False, # Standard pattern: let route handle timestamp
) )
# Removed separate timestamp update logic since indexer handles it # Update timestamp only on success (like other connectors)
if error_message:
logger.error(f"GitHub indexing failed for connector {connector_id}: {error_message}")
else:
logger.info(f"GitHub indexing successful for connector {connector_id}. Indexed {indexed_count} documents.")
await update_connector_last_indexed(session, connector_id)
await session.commit()
``` ```
## Testing the Fix ## Testing the Fix

5
surfsense_backend/app/routes/search_source_connectors_routes.py
View file

@ -782,7 +782,7 @@ async def run_github_indexing(
user_id, user_id,
start_date, start_date,
end_date, end_date,
update_last_indexed=False, update_last_indexed=False, # Standard pattern: let route handle timestamp
) )
if error_message: if error_message:
logger.error( logger.error(
@ -792,9 +792,8 @@ async def run_github_indexing(
logger.info( logger.info(
f"GitHub indexing successful for connector {connector_id}. Indexed {indexed_count} documents." f"GitHub indexing successful for connector {connector_id}. Indexed {indexed_count} documents."
) )
# Update the last indexed timestamp only on success
await update_connector_last_indexed(session, connector_id) await update_connector_last_indexed(session, connector_id)
await session.commit() # Commit timestamp update await session.commit()
except Exception as e: except Exception as e:
await session.rollback() await session.rollback()
logger.error( logger.error(