mirror of
https://github.com/katanemo/plano.git
synced 2026-05-08 23:32:43 +02:00
demos for network copilot and sql analyzer (#57)
* pulled from main branch after adding enums and made changes * added sql_analyzer folder and built a demo for Employee stats function calling. "top_employees" and "aggregate_stats". * sql_anayzer * After addressing PR comments * PR comments * PR comments * Addeed Network Analyzer FC Code * Added network Analyzer code for diff timeframes * Network Copilot and Employee Details demos are updated with their descriptions and resolved the PR comments * Added 2nd function in network copilot * Added 2nd function in network copilot * Added 2nd function in network copilot * Added 2nd function in network copilot * Added 2nd function in network copilot
This commit is contained in:
Sampreeth Sarma
GitHub
parent
a91fbdbf1c
commit
ed6a9139e6
11 changed files with 1052 additions and 1 deletions
28
demos/employee_details_copilot/README.md
Normal file
28
demos/employee_details_copilot/README.md
Normal file
|
|
@ -0,0 +1,28 @@
|
|||
# Function calling
|
||||
This demo shows how you can use intelligent prompt gateway to act a copilot for calling the correct proc by capturing the required and optional parametrs from the prompt. This demo assumes you are using ollama running natively. If you want to run ollama running inside docker then please update ollama endpoint in docker-compose file.
|
||||
|
||||
# Startig the demo
|
||||
1. Ensure that submodule is up to date
|
||||
```sh
|
||||
git submodule sync --recursive
|
||||
```
|
||||
1. Create `.env` file and set OpenAI key using env var `OPENAI_API_KEY`
|
||||
1. Start services
|
||||
```sh
|
||||
docker compose up
|
||||
```
|
||||
1. Download Bolt-FC model. This demo assumes we have downloaded [Bolt-Function-Calling-1B:Q4_K_M](https://huggingface.co/katanemolabs/Bolt-Function-Calling-1B.gguf/blob/main/Bolt-Function-Calling-1B-Q4_K_M.gguf) to local folder.
|
||||
1. If running ollama natively run
|
||||
```sh
|
||||
ollama serve
|
||||
```
|
||||
2. Create model file in ollama repository
|
||||
```sh
|
||||
ollama create Bolt-Function-Calling-1B:Q4_K_M -f Bolt-FC-1B-Q4_K_M.model_file
|
||||
```
|
||||
3. Navigate to http://localhost:18080/
|
||||
4. You can type in queries like "show me the top 5 employees in each department with highest salary"
|
||||
- You can also ask follow up questions like "just show the top 2"
|
||||
5. To see metrics navigate to "http://localhost:3000/" (use admin/grafana for login)
|
||||
- Open up dahsboard named "Intelligent Gateway Overview"
|
||||
- On this dashboard you can see reuqest latency and number of requests
|
||||
79
demos/employee_details_copilot/bolt_config.yaml
Normal file
79
demos/employee_details_copilot/bolt_config.yaml
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
default_prompt_endpoint: "127.0.0.1"
|
||||
load_balancing: "round_robin"
|
||||
timeout_ms: 5000
|
||||
|
||||
overrides:
|
||||
# confidence threshold for prompt target intent matching
|
||||
prompt_target_intent_matching_threshold: 0.8
|
||||
|
||||
# should not be here
|
||||
embedding_provider:
|
||||
name: "bge-large-en-v1.5"
|
||||
model: "BAAI/bge-large-en-v1.5"
|
||||
|
||||
llm_providers:
|
||||
|
||||
- name: open-ai-gpt-4
|
||||
api_key: $OPEN_AI_API_KEY
|
||||
model: gpt-4
|
||||
default: true
|
||||
|
||||
prompt_targets:
|
||||
|
||||
- type: function_resolver
|
||||
name: top_employees
|
||||
description: |
|
||||
Allows you to find the top employees in different groups, such as departments, locations, or position. You can rank the employees by different criteria, like salary, yoe, or rating. Returns the best-ranked employees for each group, helping you identify top n in the list.
|
||||
parameters:
|
||||
- name: grouping
|
||||
description: |
|
||||
Select how you'd like to group the employees. For example, you can group them by department, location, or their position. The tool will provide the top-ranked employees within each group you choose.
|
||||
required: true
|
||||
type: string
|
||||
enum: [department, location, position]
|
||||
- name: ranking_criteria
|
||||
required: true
|
||||
type: string
|
||||
description: |
|
||||
Choose how you'd like to rank the employees. You can rank them by their salary, their yoe, or their rating. The tool will sort the employees based on this ranking and return the best ones from each group.
|
||||
enum: [salary, yoe, rating]
|
||||
- name: top_n
|
||||
required: true
|
||||
type: integer
|
||||
description: |
|
||||
Enter how many of the top employees you want to see in each group. For example, if you enter 3, the tool will show you the top 3 employees for each group you selected.
|
||||
endpoint:
|
||||
cluster: databasehost
|
||||
path: /top_employees
|
||||
system_prompt: |
|
||||
You are responsible for retrieving the top N employees per group ranked by a constraint.
|
||||
|
||||
- type: function_resolver
|
||||
name: aggregate_stats
|
||||
description: |
|
||||
Calculate summary statistics for groups of employees. You can group employees by categories like department or location and then compute totals, averages, or other statistics for specific attributes such as salary or yoe.
|
||||
parameters:
|
||||
- name: grouping
|
||||
description: |
|
||||
Choose how you'd like to organize the employees. For example, you can group them by department, location, or position. The tool will calculate the summary statistics for each group.
|
||||
required: true
|
||||
enum: [department, location, position]
|
||||
- name: aggregate_criteria
|
||||
description: |
|
||||
Select the specific attribute you'd like to analyze. This could be something like salary, yoe, or rating. The tool will calculate the statistic you request for this attribute.
|
||||
required: true
|
||||
enum: [salary, yoe, rating]
|
||||
- name: aggregate_type
|
||||
description: |
|
||||
Choose the type of statistic you'd like to calculate for the selected attribute. For example, you can calculate the sum, average, minimum, or maximum value for each group.
|
||||
required: true
|
||||
enum: [SUM, AVG, MIN, MAX]
|
||||
endpoint:
|
||||
cluster: databasehost
|
||||
path: /aggregate_stats
|
||||
system_prompt: |
|
||||
You help calculate summary statistics for groups of employees. First, organize the employees by the specified grouping (e.g., department, location, or position). Then, compute the requested statistic (e.g., total, average, minimum, or maximum) for a specific attribute like salary, experience, or rating.
|
||||
|
||||
clusters:
|
||||
databasehost:
|
||||
address: model_server
|
||||
127
demos/employee_details_copilot/docker-compose.yaml
Normal file
127
demos/employee_details_copilot/docker-compose.yaml
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
services:
|
||||
|
||||
config_generator:
|
||||
build:
|
||||
context: ../../
|
||||
dockerfile: config_generator/Dockerfile
|
||||
volumes:
|
||||
- ../../envoyfilter/envoy.template.yaml:/usr/src/app/envoy.template.yaml
|
||||
- ./bolt_config.yaml:/usr/src/app/bolt_config.yaml
|
||||
- ./generated:/usr/src/app/out
|
||||
|
||||
bolt:
|
||||
build:
|
||||
context: ../../
|
||||
dockerfile: envoyfilter/Dockerfile
|
||||
hostname: bolt
|
||||
ports:
|
||||
- "10000:10000"
|
||||
- "19901:9901"
|
||||
volumes:
|
||||
- ./generated/envoy.yaml:/etc/envoy/envoy.yaml
|
||||
- /etc/ssl/cert.pem:/etc/ssl/cert.pem
|
||||
depends_on:
|
||||
config_generator:
|
||||
condition: service_completed_successfully
|
||||
model_server:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
- LOG_LEVEL=debug
|
||||
|
||||
model_server:
|
||||
build:
|
||||
context: ../../model_server
|
||||
dockerfile: Dockerfile
|
||||
ports:
|
||||
- "18081:80"
|
||||
healthcheck:
|
||||
test: ["CMD", "curl" ,"http://localhost:80/healthz"]
|
||||
interval: 5s
|
||||
retries: 20
|
||||
volumes:
|
||||
- ~/.cache/huggingface:/root/.cache/huggingface
|
||||
|
||||
function_resolver:
|
||||
build:
|
||||
context: ../../function_resolver
|
||||
dockerfile: Dockerfile
|
||||
ports:
|
||||
- "18082:80"
|
||||
healthcheck:
|
||||
test: ["CMD", "curl" ,"http://localhost:80/healthz"]
|
||||
interval: 5s
|
||||
retries: 20
|
||||
volumes:
|
||||
- ~/.cache/huggingface:/root/.cache/huggingface
|
||||
environment:
|
||||
# use ollama endpoint that is hosted by host machine (no virtualization)
|
||||
- OLLAMA_ENDPOINT=${OLLAMA_ENDPOINT:-host.docker.internal}
|
||||
# uncomment following line to use ollama endpoint that is hosted by docker
|
||||
# - OLLAMA_ENDPOINT=ollama
|
||||
|
||||
ollama:
|
||||
image: ollama/ollama
|
||||
container_name: ollama
|
||||
volumes:
|
||||
- ./ollama:/root/.ollama
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- '11434:11434'
|
||||
profiles:
|
||||
- manual
|
||||
|
||||
open-webui:
|
||||
image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG-main}
|
||||
container_name: open-webui
|
||||
volumes:
|
||||
- ./open-webui:/app/backend/data
|
||||
# depends_on:
|
||||
# - ollama
|
||||
ports:
|
||||
- 18090:8080
|
||||
environment:
|
||||
- OLLAMA_BASE_URL=http://${OLLAMA_ENDPOINT:-host.docker.internal}:11434
|
||||
- WEBUI_AUTH=false
|
||||
extra_hosts:
|
||||
- host.docker.internal:host-gateway
|
||||
restart: unless-stopped
|
||||
|
||||
chatbot_ui:
|
||||
build:
|
||||
context: ../../chatbot_ui
|
||||
dockerfile: Dockerfile
|
||||
ports:
|
||||
- "18080:8080"
|
||||
environment:
|
||||
- OPENAI_API_KEY=${OPENAI_API_KEY:?error}
|
||||
- CHAT_COMPLETION_ENDPOINT=http://bolt:10000/v1/chat/completions
|
||||
|
||||
prometheus:
|
||||
image: prom/prometheus
|
||||
container_name: prometheus
|
||||
command:
|
||||
- '--config.file=/etc/prometheus/prometheus.yaml'
|
||||
ports:
|
||||
- 9090:9090
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./prometheus:/etc/prometheus
|
||||
- ./prom_data:/prometheus
|
||||
profiles:
|
||||
- monitoring
|
||||
|
||||
grafana:
|
||||
image: grafana/grafana
|
||||
container_name: grafana
|
||||
ports:
|
||||
- 3000:3000
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- GF_SECURITY_ADMIN_USER=admin
|
||||
- GF_SECURITY_ADMIN_PASSWORD=grafana
|
||||
volumes:
|
||||
- ./grafana:/etc/grafana/provisioning/datasources
|
||||
- ./grafana/dashboard.yaml:/etc/grafana/provisioning/dashboards/main.yaml
|
||||
- ./grafana/dashboards:/var/lib/grafana/dashboards
|
||||
profiles:
|
||||
- monitoring
|
||||
Loading…
Add table
Add a link
Reference in a new issue