apunkt/SurfSense
1
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2026-06-20 21:18:13 +02:00
Code Issues Projects Releases Packages Wiki Activity

feat: added celery beat to docker

Browse source
This commit is contained in:
DESKTOP-RTLN3BA\$punk 2025-10-23 00:56:28 -07:00
parent 8706df6716
commit aed8163aee
6 changed files with 134 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

31
surfsense_web/components/homepage/navbar.tsx
View file

@ -129,11 +129,18 @@ const MobileNav = ({ navItems, isScrolled }: any) => {
<Logo className="h-8 w-8 rounded-md" />
<span className="dark:text-white/90 text-gray-800 text-lg font-bold">SurfSense</span>
</div>
{open ? (
<IconX className="text-black dark:text-white" onClick={() => setOpen(!open)} />
) : (
<IconMenu2 className="text-black dark:text-white" onClick={() => setOpen(!open)} />
)}
<button
type="button"
onClick={() => setOpen(!open)}
className="relative z-50 flex items-center justify-center p-2 -mr-2 rounded-lg hover:bg-gray-100 dark:hover:bg-neutral-800 transition-colors touch-manipulation"
aria-label={open ? "Close menu" : "Open menu"}
>
{open ? (
<IconX className="h-6 w-6 text-black dark:text-white" />
) : (
<IconMenu2 className="h-6 w-6 text-black dark:text-white" />
)}
</button>
</div>
<AnimatePresence>
@ -155,10 +162,10 @@ const MobileNav = ({ navItems, isScrolled }: any) => {
))}
<div className="flex w-full items-center gap-2 pt-2">
<Link
href="https://discord.gg/your-server"
href="https://discord.gg/ejRNvftDp9"
target="_blank"
rel="noopener noreferrer"
className="flex items-center justify-center rounded-lg p-2 hover:bg-gray-100 dark:hover:bg-neutral-800 transition-colors"
className="flex items-center justify-center rounded-lg p-2 hover:bg-gray-100 dark:hover:bg-neutral-800 transition-colors touch-manipulation"
>
<IconBrandDiscord className="h-5 w-5 text-neutral-600 dark:text-neutral-300" />
</Link>
@ -166,7 +173,7 @@ const MobileNav = ({ navItems, isScrolled }: any) => {
href="https://github.com/MODSetter/SurfSense"
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1.5 rounded-lg px-3 py-2 hover:bg-gray-100 dark:hover:bg-neutral-800 transition-colors"
className="flex items-center gap-1.5 rounded-lg px-3 py-2 hover:bg-gray-100 dark:hover:bg-neutral-800 transition-colors touch-manipulation"
>
<IconBrandGithub className="h-5 w-5 text-neutral-600 dark:text-neutral-300" />
{loadingGithubStars ? (
@ -179,12 +186,12 @@ const MobileNav = ({ navItems, isScrolled }: any) => {
</Link>
<ThemeTogglerComponent />
</div>
<button
type="button"
className="w-full rounded-lg bg-black px-8 py-2 font-medium text-white shadow-[0px_-2px_0px_0px_rgba(255,255,255,0.4)_inset] dark:bg-white dark:text-black"
<Link
href="/contact"
className="w-full rounded-lg bg-black px-8 py-2 font-medium text-white shadow-[0px_-2px_0px_0px_rgba(255,255,255,0.4)_inset] dark:bg-white dark:text-black text-center touch-manipulation"
>
Book a call
</button>
</Link>
</motion.div>
)}
</AnimatePresence>

18
surfsense_web/content/docs/docker-installation.mdx
View file

@ -98,6 +98,8 @@ Before you begin, ensure you have:
| LLAMA_CLOUD_API_KEY | API key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD) |
| CELERY_BROKER_URL | Redis connection URL for Celery broker (e.g., `redis://localhost:6379/0`) |
| CELERY_RESULT_BACKEND | Redis connection URL for Celery result backend (e.g., `redis://localhost:6379/0`) |
| SCHEDULE_CHECKER_INTERVAL | (Optional) How often to check for scheduled connector tasks. Format: `<number><unit>` where unit is `m` (minutes) or `h` (hours). Examples: `1m`, `5m`, `1h`, `2h` (default: `1m`) |
| REGISTRATION_ENABLED | (Optional) Enable or disable new user registration (e.g., `TRUE` or `FALSE`, default: `TRUE`) |
**Optional Backend LangSmith Observability:**
@ -181,6 +183,22 @@ For more details, see the [Uvicorn documentation](https://www.uvicorn.org/#comma
- API Documentation: [http://localhost:8000/docs](http://localhost:8000/docs)
- pgAdmin: [http://localhost:5050](http://localhost:5050)
## Docker Services Overview
The Docker setup includes several services that work together:
- **Backend**: FastAPI application server
- **Frontend**: Next.js web application
- **PostgreSQL (db)**: Database with pgvector extension
- **Redis**: Message broker for Celery
- **Celery Worker**: Handles background tasks (document processing, indexing, etc.)
- **Celery Beat**: Scheduler for periodic tasks (enables scheduled connector indexing)
- The schedule interval can be configured using the `SCHEDULE_CHECKER_INTERVAL` environment variable in your backend `.env` file
- Default: checks every minute for connectors that need indexing
- **pgAdmin**: Database management interface
All services start automatically with `docker compose up`. The Celery Beat service ensures that periodic indexing functionality works out of the box.
## Using pgAdmin
pgAdmin is included in the Docker setup to help manage your PostgreSQL database. To connect:

20
surfsense_web/content/docs/manual-installation.mdx
View file

@ -86,6 +86,8 @@ Edit the `.env` file and set the following variables:
| LLAMA_CLOUD_API_KEY | API key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD) |
| CELERY_BROKER_URL | Redis connection URL for Celery broker (e.g., `redis://localhost:6379/0`) |
| CELERY_RESULT_BACKEND | Redis connection URL for Celery result backend (e.g., `redis://localhost:6379/0`) |
| SCHEDULE_CHECKER_INTERVAL | (Optional) How often to check for scheduled connector tasks. Format: `<number><unit>` where unit is `m` (minutes) or `h` (hours). Examples: `1m`, `5m`, `1h`, `2h` (default: `1m`) |
| REGISTRATION_ENABLED | (Optional) Enable or disable new user registration (e.g., `TRUE` or `FALSE`, default: `TRUE`) |
**(Optional) Backend LangSmith Observability:**
@ -249,7 +251,23 @@ uv run celery -A celery_worker.celery_app flower --port=5555
Access Flower at [http://localhost:5555](http://localhost:5555) to monitor your Celery tasks.
### 5. Run the Backend
### 5. Start Celery Beat (Scheduler)
In another new terminal window, start Celery Beat to enable periodic tasks (like scheduled connector indexing):
**Linux/macOS/Windows:**
```bash
# Make sure you're in the surfsense_backend directory
cd surfsense_backend
# Start Celery Beat
uv run celery -A celery_worker.celery_app beat --loglevel=info
```
**Important**: Celery Beat is required for the periodic indexing functionality to work. Without it, scheduled connector tasks won't run automatically. The schedule interval can be configured using the `SCHEDULE_CHECKER_INTERVAL` environment variable.
### 6. Run the Backend
Start the backend server: