Updates the GitHub Actions workflow (`.github/workflows/docker-publish.yml`)
to build and push Docker images for both `linux/amd64` and `linux/arm64`
architectures.
The `platforms` attribute has been added to the `docker/build-push-action`
step for both the backend and frontend jobs. This ensures that you on
different CPU architectures can use the published images from ghcr.io.
Adds a GitHub Actions workflow to automatically build and publish Docker images for the backend and frontend services.
The workflow (`.github/workflows/docker-publish.yml`) is triggered on pushes to the `main` branch. It includes two jobs:
1. `build_and_push_backend`: Builds the Docker image from `surfsense_backend/Dockerfile` and pushes it to `ghcr.io/<owner>/surfsense_backend:<commit_sha>`.
2. `build_and_push_frontend`: Builds the Docker image from `surfsense_web/Dockerfile` and pushes it to `ghcr.io/<owner>/surfsense_web:<commit_sha>`.
Both jobs include steps for:
- Checking out the repository.
- Setting up QEMU and Docker Buildx.
- Logging into the GitHub Container Registry (ghcr.io) using `secrets.GITHUB_TOKEN`.
- Building and pushing the respective Docker images, tagged with the commit SHA.
- Adding OCI labels for image source, creation date, and revision.
This CI pipeline automates the process of creating and distributing Docker images for the application, ensuring that new versions are available in the GitHub Container Registry upon changes to the main branch.
This commit addresses recurring `SyntaxError: invalid non-printable character U+001B`
errors in `surfsense_backend/app/connectors/slack_history.py`.
The file was cleaned to remove all occurrences of the
U+001B (ESCAPE) character. This ensures that previously introduced
problematic control characters are fully removed, allowing the application
to parse and load the module correctly.
Here's a rundown of what I did:
Fix: Robust Slack rate limiting, error handling & GitHub org repos
This update delivers comprehensive improvements to Slack connector stability and enhances the GitHub connector.
**Slack Connector (`slack_history.py`, `connectors_indexing_tasks.py`):**
- I've implemented proactive delays (1.2s for `conversations.history`, 3s for `conversations.list` pagination) and `Retry-After` header handling for 429 rate limit errors across `conversations.list`, `conversations.history`, and `users.info` API calls.
- I'll now gracefully handle `not_in_channel` errors when fetching conversation history by logging a warning and skipping the channel.
- I've refactored channel info fetching: `get_all_channels` now returns richer channel data (including `is_member`, `is_private`).
- I've removed direct calls to `conversations.info` from `connectors_indexing_tasks.py`, using the richer data from `get_all_channels` instead, to prevent associated rate limits.
- I corrected a `SyntaxError` (non-printable character) in `slack_history.py`.
- I've enhanced logging for rate limit actions, delays, and errors.
- I've updated unit tests in `test_slack_history.py` to cover all new logic.
**GitHub Connector (`github_connector.py`):**
- I've modified `get_user_repositories` to fetch all repositories accessible by you (owned, collaborated, organization) by changing the API call parameter from `type='owner'` to `type='all'`.
- I've included unit tests in `test_github_connector.py` for this change.
