apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-04-25 16:56:24 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

updated all demo READMes and minor doc changes (#154)

Browse source 
* updated all demo READMes and minor doc changes

* minor typo fixes

* updated main Readme

* fixed README and docs

* fixed README and docs

---------

Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
This commit is contained in:
Salman Paracha 2024-10-08 23:58:55 -07:00 • committed by GitHub
parent b63a01fe82
commit 42d4a28e13
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
22 changed files with 324 additions and 1455 deletions
Download patch file Download diff file Expand all files Collapse all files

28
demos/function_calling/README.md
View file

@ -1,29 +1,20 @@
# Function calling
This demo shows how you can use intelligent prompt gateway to do function calling. This demo assumes you are using ollama running natively. If you want to run ollama running inside docker then please update ollama endpoint in docker-compose file.
This demo shows how you can use Arch's function calling capabilites.
# Starting the demo
1. Ensure that submodule is up to date
```sh
git submodule sync --recursive
```
1. Create `.env` file and set OpenAI key using env var `OPENAI_API_KEY`
1. Start services
2. Start Arch
```sh
docker compose up
archgw up arch_config.yaml
```
1. Download Bolt-FC model. This demo assumes we have downloaded [Arch-Function-Calling-1.5B:Q4_K_M](https://huggingface.co/katanemolabs/Arch-Function-Calling-1.5B.gguf/blob/main/Arch-Function-Calling-1.5B-Q4_K_M.gguf) to local folder.
1. If running ollama natively run
```sh
ollama serve
3. Start Network Agent
```sh
docker compose up
```
2. Create model file in ollama repository
```sh
ollama create Arch-Function-Calling-1.5B:Q4_K_M -f Arch-Function-Calling-1.5B-Q4_K_M.model_file
```
3. Navigate to http://localhost:18080/
4. Navigate to http://localhost:18080/
4. You can type in queries like "how is the weather in Seattle"
- You can also ask follow up questions like "show me sunny days"
5. To see metrics navigate to "http://localhost:3000/" (use admin/grafana for login)
6. To see metrics navigate to "http://localhost:3000/" (use admin/grafana for login)
- Open up dahsboard named "Intelligent Gateway Overview"
- On this dashboard you can see reuqest latency and number of requests
@ -38,6 +29,5 @@ Arch gateway publishes stats endpoint at http://localhost:19901/stats. In this d
1. From grafana left nav click on dashboards and select "Intelligent Gateway Overview" to view arch gateway stats
Here is sample interaction,
Here is a sample interaction,
<img width="575" alt="image" src="https://github.com/user-attachments/assets/e0929490-3eb2-4130-ae87-a732aea4d059">