Add support for local llm (mistral 7b) (#31)

This commit is contained in:
Adil Hafeez 2024-08-06 23:40:06 -07:00 committed by GitHub
parent b49fc2f264
commit 445b1ea210
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
24 changed files with 703 additions and 51 deletions

View file

@ -4,12 +4,12 @@ This demo shows how you can use intelligent prompt gateway to provide realtime w
# Startig the demo
1. Ensure that submodule is up to date
```sh
$ git submodule sync --recursive
git submodule sync --recursive
```
1. Create `.env` file and set OpenAI key using env var `OPENAI_API_KEY`
1. Start services
```sh
$ docker compose up
docker compose up
```
1. Navigate to http://localhost:18080/
1. You can type in queries like "how is the weather in Seattle"