mirror of
https://github.com/MODSetter/SurfSense.git
synced 2026-07-12 22:42:13 +02:00
feat(local-models): add documentation for connecting local model providers
This commit is contained in:
parent
97f004e7e1
commit
ceace003aa
11 changed files with 351 additions and 93 deletions
30
surfsense_web/content/docs/local-models/index.mdx
Normal file
30
surfsense_web/content/docs/local-models/index.mdx
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
title: Local Models
|
||||
description: Connect local model servers to SurfSense
|
||||
---
|
||||
|
||||
import { Card, Cards } from 'fumadocs-ui/components/card';
|
||||
|
||||
# Local Models
|
||||
|
||||
SurfSense can use local model servers such as Ollama and LM Studio.
|
||||
|
||||
The API Base URL is read by the SurfSense backend. If SurfSense runs in Docker, use an address the backend container can reach.
|
||||
|
||||
<Cards>
|
||||
<Card
|
||||
title="Ollama"
|
||||
description="Connect an Ollama server and discover local model tags"
|
||||
href="/docs/local-models/ollama"
|
||||
/>
|
||||
<Card
|
||||
title="LM Studio"
|
||||
description="Connect an LM Studio local server"
|
||||
href="/docs/local-models/lm-studio"
|
||||
/>
|
||||
<Card
|
||||
title="Other Local Servers"
|
||||
description="Connect llama.cpp, vLLM, LocalAI, LiteLLM Proxy, and more"
|
||||
href="/docs/local-models/other-local-servers"
|
||||
/>
|
||||
</Cards>
|
||||
92
surfsense_web/content/docs/local-models/lm-studio.mdx
Normal file
92
surfsense_web/content/docs/local-models/lm-studio.mdx
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
---
|
||||
title: LM Studio
|
||||
description: Connect LM Studio to SurfSense
|
||||
---
|
||||
|
||||
# Connect LM Studio
|
||||
|
||||
Connect to your LM Studio local server. Add it from
|
||||
**Search Space Settings > Models**.
|
||||
|
||||
## Base URL
|
||||
|
||||
LM Studio uses an OpenAI compatible server.
|
||||
|
||||
### SurfSense Runs in Docker
|
||||
|
||||
Use this when SurfSense is running from Docker and LM Studio is running on your computer.
|
||||
|
||||
```text
|
||||
http://host.docker.internal:1234/v1
|
||||
```
|
||||
|
||||
<Callout type="info">
|
||||
This is the default in SurfSense.
|
||||
</Callout>
|
||||
|
||||
### SurfSense Runs Without Docker
|
||||
|
||||
Use this when SurfSense and LM Studio both run directly on the same computer.
|
||||
|
||||
```text
|
||||
http://localhost:1234/v1
|
||||
```
|
||||
|
||||
### LM Studio Runs on Another Computer
|
||||
|
||||
Use this when LM Studio is running on another machine in your network.
|
||||
|
||||
```text
|
||||
http://<host>:1234/v1
|
||||
```
|
||||
|
||||
Replace `<host>` with the LAN IP or domain for that machine.
|
||||
|
||||
## LM Studio Setup
|
||||
|
||||
1. Open LM Studio.
|
||||
2. Load a model.
|
||||
3. Start the local server.
|
||||
4. Confirm the server listens on port `1234`.
|
||||
|
||||
## Add the Connection
|
||||
|
||||
1. Open Search Space Settings.
|
||||
2. Go to Models.
|
||||
3. Select LM Studio.
|
||||
4. Set API Base URL.
|
||||
5. Leave API Key empty unless your server requires one.
|
||||
6. Select the models you want to enable.
|
||||
7. Save the connection.
|
||||
|
||||
SurfSense discovers models from `/v1/models`. If you enter the URL without `/v1`, SurfSense adds it for requests.
|
||||
|
||||
## Verify
|
||||
|
||||
From the host:
|
||||
|
||||
```bash
|
||||
curl http://localhost:1234/v1/models
|
||||
```
|
||||
|
||||
From the SurfSense backend container:
|
||||
|
||||
```bash
|
||||
docker compose exec backend curl http://host.docker.internal:1234/v1/models
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection refused
|
||||
|
||||
LM Studio is not reachable from the backend.
|
||||
|
||||
Start the LM Studio server and confirm that port `1234` is open.
|
||||
|
||||
### No models found
|
||||
|
||||
Load a model in LM Studio, then refresh model discovery in SurfSense.
|
||||
|
||||
### Endpoint returned 404
|
||||
|
||||
Use an OpenAI compatible server URL. The models endpoint must be available at `/v1/models`.
|
||||
6
surfsense_web/content/docs/local-models/meta.json
Normal file
6
surfsense_web/content/docs/local-models/meta.json
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
{
|
||||
"title": "Local Models",
|
||||
"pages": ["ollama", "lm-studio", "other-local-servers"],
|
||||
"icon": "Cpu",
|
||||
"defaultOpen": false
|
||||
}
|
||||
102
surfsense_web/content/docs/local-models/ollama.mdx
Normal file
102
surfsense_web/content/docs/local-models/ollama.mdx
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
---
|
||||
title: Ollama
|
||||
description: Connect Ollama to SurfSense
|
||||
---
|
||||
|
||||
# Connect Ollama
|
||||
|
||||
Connect to your Ollama local server. Add it from
|
||||
**Search Space Settings > Models**.
|
||||
|
||||
## Base URL
|
||||
|
||||
Choose the URL from where the SurfSense backend runs.
|
||||
|
||||
### SurfSense Runs in Docker
|
||||
|
||||
Use this when SurfSense is running from Docker and Ollama is running on your computer.
|
||||
|
||||
```text
|
||||
http://host.docker.internal:11434
|
||||
```
|
||||
|
||||
<Callout type="info">
|
||||
This is the default in SurfSense.
|
||||
</Callout>
|
||||
|
||||
### Ollama Runs in Docker
|
||||
|
||||
Use this only when Ollama is a service in the same Compose stack as SurfSense.
|
||||
|
||||
```text
|
||||
http://ollama:11434
|
||||
```
|
||||
|
||||
### SurfSense Runs Without Docker
|
||||
|
||||
Use this when SurfSense and Ollama both run directly on the same computer.
|
||||
|
||||
```text
|
||||
http://localhost:11434
|
||||
```
|
||||
|
||||
### Ollama Runs on Another Computer
|
||||
|
||||
Use this when Ollama is running on another machine in your network.
|
||||
|
||||
```text
|
||||
http://<host>:11434
|
||||
```
|
||||
|
||||
Replace `<host>` with the LAN IP or domain for that machine.
|
||||
|
||||
## Add the Connection
|
||||
|
||||
1. Open Search Space Settings.
|
||||
2. Go to Models.
|
||||
3. Select Ollama.
|
||||
4. Set API Base URL.
|
||||
5. Leave API Key empty unless your endpoint needs one.
|
||||
6. Select the models you want to enable.
|
||||
7. Save the connection.
|
||||
|
||||
Do not add `/v1` to the URL. SurfSense uses Ollama native routes such as `/api/version` and `/api/tags`.
|
||||
|
||||
## Verify
|
||||
|
||||
From the host:
|
||||
|
||||
```bash
|
||||
curl http://localhost:11434/api/version
|
||||
```
|
||||
|
||||
From the SurfSense backend container:
|
||||
|
||||
```bash
|
||||
docker compose exec backend curl http://host.docker.internal:11434/api/version
|
||||
docker compose exec backend curl http://host.docker.internal:11434/api/tags
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Name or service not known
|
||||
|
||||
The backend cannot resolve the host name.
|
||||
|
||||
Use `http://host.docker.internal:11434` unless you run Ollama as a Compose service named `ollama`.
|
||||
|
||||
### Connection refused
|
||||
|
||||
Ollama is not reachable from the backend.
|
||||
|
||||
Start Ollama and confirm that port `11434` is open.
|
||||
|
||||
### No models found
|
||||
|
||||
Pull at least one model:
|
||||
|
||||
```bash
|
||||
ollama pull llama3.2
|
||||
```
|
||||
|
||||
Then refresh model discovery in SurfSense.
|
||||
109
surfsense_web/content/docs/local-models/other-local-servers.mdx
Normal file
109
surfsense_web/content/docs/local-models/other-local-servers.mdx
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
---
|
||||
title: Other Local Servers
|
||||
description: Connect local OpenAI compatible model servers
|
||||
---
|
||||
|
||||
# Connect Other Local Servers
|
||||
|
||||
Connect to llama.cpp, vLLM, LocalAI, LiteLLM Proxy, and other servers
|
||||
that expose OpenAI compatible routes.
|
||||
|
||||
SurfSense discovers models from:
|
||||
|
||||
```text
|
||||
/v1/models
|
||||
```
|
||||
|
||||
Chat requests use the same `/v1` base URL.
|
||||
|
||||
## Pick Your Setup
|
||||
|
||||
Use one of these URL patterns.
|
||||
|
||||
### SurfSense Runs in Docker
|
||||
|
||||
Use this when SurfSense is running from Docker and the model server is running on your computer.
|
||||
|
||||
```text
|
||||
http://host.docker.internal:<port>/v1
|
||||
```
|
||||
|
||||
Common ports:
|
||||
|
||||
| Server | Port |
|
||||
|---|---|
|
||||
| llama.cpp | `10000` |
|
||||
| vLLM | `8000` |
|
||||
| LocalAI | `8080` |
|
||||
| LiteLLM Proxy | `4000` |
|
||||
| text-generation-webui | `5000` |
|
||||
|
||||
### SurfSense Runs Without Docker
|
||||
|
||||
Use this when SurfSense and the model server both run directly on the same computer.
|
||||
|
||||
```text
|
||||
http://localhost:<port>/v1
|
||||
```
|
||||
|
||||
### Model Server Runs on Another Computer
|
||||
|
||||
Use this when the model server is running on another machine in your network.
|
||||
|
||||
```text
|
||||
http://<host>:<port>/v1
|
||||
```
|
||||
|
||||
## Add the Connection
|
||||
|
||||
1. Open Search Space Settings.
|
||||
2. Go to Models.
|
||||
3. Select OpenAI Compatible.
|
||||
4. Set API Base URL.
|
||||
5. Add an API Key only if your server requires one.
|
||||
6. Select the models you want to enable.
|
||||
7. Save the connection.
|
||||
|
||||
If you enter the URL without `/v1`, SurfSense adds `/v1` for requests.
|
||||
|
||||
## Verify
|
||||
|
||||
From the host:
|
||||
|
||||
```bash
|
||||
curl http://localhost:<port>/v1/models
|
||||
```
|
||||
|
||||
From the SurfSense backend container:
|
||||
|
||||
```bash
|
||||
docker compose exec backend curl http://host.docker.internal:<port>/v1/models
|
||||
```
|
||||
|
||||
A working server returns JSON with a `data` array.
|
||||
|
||||
## When Not to Use This
|
||||
|
||||
Use the Ollama provider for Ollama. It uses native routes such as `/api/tags`.
|
||||
|
||||
Use the LM Studio provider for LM Studio. Its default URL is already set.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Endpoint returned 404
|
||||
|
||||
The server does not expose `/v1/models`.
|
||||
|
||||
Enable the server's OpenAI compatible mode.
|
||||
|
||||
### Connection refused
|
||||
|
||||
The backend cannot reach the server.
|
||||
|
||||
Check that the server is running and that the port is open.
|
||||
|
||||
### No models found
|
||||
|
||||
The server returned an empty model list.
|
||||
|
||||
Load or serve a model, then refresh model discovery in SurfSense.
|
||||
Loading…
Add table
Add a link
Reference in a new issue