mirror of
https://github.com/katanemo/plano.git
synced 2026-05-08 23:32:43 +02:00
Added Float type to the function parameter values (#77)
This commit is contained in:
parent
7505a0fc1f
commit
7f0fcb372b
26 changed files with 1505 additions and 45 deletions
24
demos/employee_details_copilot_arch/README.md
Normal file
24
demos/employee_details_copilot_arch/README.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# Function calling
|
||||
This demo shows how you can use intelligent prompt gateway as copilot to explore employee data by calling the correct api functions. It calls appropriate function and also engages with user to extract required parameters. This demo assumes you are using ollama natively.
|
||||
|
||||
# Starting the demo
|
||||
1. Create `.env` file and set OpenAI key using env var `OPENAI_API_KEY`
|
||||
1. Start services
|
||||
```sh
|
||||
docker compose up
|
||||
```
|
||||
1. Download Bolt-FC model. This demo assumes we have downloaded [Bolt-Function-Calling-1B:Q4_K_M](https://huggingface.co/katanemolabs/Bolt-Function-Calling-1B.gguf/blob/main/Bolt-Function-Calling-1B-Q4_K_M.gguf) to local folder.
|
||||
1. If running ollama natively run
|
||||
```sh
|
||||
ollama serve
|
||||
```
|
||||
2. Create model file in ollama repository
|
||||
```sh
|
||||
ollama create Bolt-Function-Calling-1B:Q4_K_M -f Bolt-FC-1B-Q4_K_M.model_file
|
||||
```
|
||||
3. Navigate to http://localhost:18080/
|
||||
4. You can type in queries like "show me the top 5 employees in each department with highest salary"
|
||||
- You can also ask follow up questions like "just show the top 2"
|
||||
5. To see metrics navigate to "http://localhost:3000/" (use admin/grafana for login)
|
||||
- Open up dahsboard named "Intelligent Gateway Overview"
|
||||
- On this dashboard you can see reuqest latency and number of requests
|
||||
Loading…
Add table
Add a link
Reference in a new issue