Adil Hafeez
962727f244
Infer port from protocol if port is not specified and add ability to override hostname in clusters def ( #389 )
2025-02-03 14:51:59 -08:00
Adil Hafeez
38f7691163
add support for custom llm with ssl support ( #380 )
...
* add support for custom llm with ssl support
Add support for using custom llm that are served through https protocol.
* add instructions on how to add custom inference endpoint
* fix formatting
* add more details
* Apply suggestions from code review
Co-authored-by: Salman Paracha <salman.paracha@gmail.com>
* Apply suggestions from code review
* fix precommit
---------
Co-authored-by: Salman Paracha <salman.paracha@gmail.com>
2025-01-24 17:14:24 -08:00
Adil Hafeez
2c67fa3bc0
Fix llm_routing provider element ( #382 )
...
* Fix llm_routing provider element
We replaced provider with provider_interface to make it more clear to developers about provider api/backend being used. During that upgrade we removed support for mistral in provider to encourage developers to start using provider_interface. But this demo was not updated to use provider_interface as it was using mistral. This code change fixes it by replacing provider with provider_interface.
Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com>
* fix the path
* move
* add more details
* fix
* Apply suggestions from code review
* fix
* fix
---------
Signed-off-by: Adil Hafeez <adil.hafeez@gmail.com>
2025-01-24 16:34:11 -08:00
Guy Stone
84af476c75
Add run_demo.sh for llm_routing demo ( #381 )
...
* Create run_demo.sh for llm_routing demo
* Update run_demo.sh
2025-01-24 14:41:51 -08:00
Adil Hafeez
6740a09952
add docker-compose file for honeycomb tracing ( #377 )
2025-01-22 14:02:59 -08:00
Aayush
fcd8cfb9fc
add in honeycomb support for weather-forecast demo ( #345 )
2025-01-21 17:15:27 -08:00
Adil Hafeez
07ef3149b8
add support for using custom upstream llm ( #365 )
2025-01-17 18:25:55 -08:00
Adil Hafeez
8407edae99
only test currency exchange in demo test ( #348 )
2024-12-21 11:33:08 -08:00
Shuguang Chen
ba7279becb
Use intent model from archfc to pick prompt gateway ( #328 )
2024-12-20 13:25:01 -08:00
Adil Hafeez
93d3d349a2
fix code bug ( #340 )
2024-12-06 17:20:59 -08:00
Adil Hafeez
a54db1a098
update getting started guide and add llm gateway and prompt gateway samples ( #330 )
2024-12-06 14:37:33 -08:00
Aayush
9d8fe02729
fix the README for the weather_forecasting demo ( #336 )
...
* README fix
* add missing colon
2024-12-06 14:02:41 -08:00
Aayush
885acc899f
322 add support for pydantic logfire for llm agent tracing ( #329 )
...
* set up otel-collector and implement sending to logfire
* moved rest of the files for the demo into the folder
* update docker-compose.yaml and run_demo.sh to properly check for LOGFIRE_API_KEY
* refactor weather_forecast demo to only be one demo
* add a default docker-compose for e2e tests
* update based on requested changes
* fix replace comma with colon in readme
* remove weather_forecast_service folder, and make logfire demo fail instantly if no key is set
* remove the unused weather forecast service folder
* Changed stop_demo to only stop one file at a time
* update readme with new demo stopping setup
* Revert changes to end behavior
* fix silly formatting mistake
2024-12-06 13:44:22 -08:00
Peter Jausovec
f5cdafb7c8
update alertmanager version to v2, remove the merge artifacts ( #309 )
...
Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
2024-11-27 11:41:31 -08:00
Adil Hafeez
0ff3d43008
remove dependency on docker-compose when starting up archgw ( #305 )
2024-11-26 13:13:02 -08:00
Adil Hafeez
726f1a3185
add schema change to use enum in arch_config ( #304 )
2024-11-25 17:51:25 -08:00
Adil Hafeez
9c6fcdb771
use fix prompt guards ( #303 )
2024-11-25 17:16:35 -08:00
Adil Hafeez
d3c17c7abd
move custom tracer to llm filter ( #267 )
2024-11-15 10:44:01 -08:00
Aayush
1d229cba8f
Add in tpot ( #269 )
...
* add in tpot and tokens per second
* add in debug logs for new stats and update integration tests
* update shared dashboard to include new stats
2024-11-14 15:03:08 -08:00
Adil Hafeez
31749bfc74
move grafana and prometheus to shared ( #265 )
2024-11-12 15:23:30 -08:00
Aayush
5993e36f22
Update arch stats ( #250 )
2024-11-12 15:03:26 -08:00
Adil Hafeez
30647fd508
Add service to stream custom otel traces to otel-collector ( #262 )
2024-11-12 11:09:40 -08:00
Adil Hafeez
d87105882b
update rust toolchain to 1.82 ( #255 )
...
* update rust to 1.82 pin it, also update envoy to 1.32 and python to 3.13
* use python:3.12
2024-11-12 10:35:14 -08:00
Salman Paracha
4b2b371876
removing depdency on mistral keys ( #256 )
2024-11-08 16:09:04 -08:00
Adil Hafeez
6b62662e01
update docs with weather_forecast path ( #253 )
2024-11-08 10:00:15 -08:00
Adil Hafeez
a72bb804eb
add support for jaeger tracing ( #229 )
2024-11-07 22:11:00 -06:00
Salman Paracha
21f4e7a5e4
fixing ports in arch_config for demos ( #241 )
...
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
2024-10-31 14:37:04 -07:00
Salman Paracha
3bff4a597b
fix ports and update README for paths to agent/chat ( #240 )
...
* fix ports and update README for paths to agent/chat
* minor fix
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
2024-10-31 09:25:24 -07:00
Salman Paracha
dab7a44053
several fixes to demos ( #238 )
...
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
2024-10-30 18:38:18 -07:00
Adil Hafeez
e462e393b1
Use large github action machine to run e2e tests ( #230 )
2024-10-30 17:54:51 -07:00
Salman Paracha
bb882fb59b
Updated hr_agent to be full stack: gradio + fastAPI ( #235 )
...
* commiting to remove
* fix
* updating hr_agent
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
Co-authored-by: Adil Hafeez <adil@katanemo.com>
2024-10-30 15:05:34 -07:00
Salman Paracha
bb9a774a72
moving chatbot-ui in demos and out of root project structure ( #228 )
...
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
2024-10-29 12:05:29 -07:00
Adil Hafeez
60299244b9
Improve Gradio UI and fix arch_state bug ( #227 )
2024-10-29 11:27:13 -07:00
José Ulises Niño Rivera
662a840ac5
Add support for streaming and fixes few issues (see description) ( #202 )
2024-10-28 17:05:06 -07:00
Salman Paracha
ff6e9bd9bd
add README for hr_agent ( #224 )
...
* add README for hr_agent
* fixed sample prompt for hr_agent in README
* added screenshot and updated README.md
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
2024-10-24 18:21:52 -07:00
Shuguang Chen
5f3aff4922
Update chatbot UI and update hallucination check ( #218 )
...
* update chatbot UI
* Update docker-compose for demos
* Fix bugs
* fix for emtadata (#219 )
* fix for emtadata
* fix
* revert
* merge main
---------
Co-authored-by: CTran <cotran2@utexas.edu>
2024-10-24 14:11:53 -07:00
Salman Paracha
708fa15a9b
HR agent demo ( #206 )
...
* commiting my hr_agent branch
* updating the HR agent config
* pushing to remote
* fix hr agent
* committing to merge with main
* updating to merge from main
* updating the demo and model-server-tests to pull from poetry
* updating the poetry.lock files
* updating based on feedback
* updated sysmte prompt for hr_agent
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
Co-authored-by: Adil Hafeez <adil@katanemo.com>
2024-10-23 14:32:40 -07:00
Adil Hafeez
faf64960df
update observability and dashboards ( #198 )
2024-10-18 15:07:49 -07:00
Adil Hafeez
f72588c6da
fix lint
2024-10-10 23:34:21 -07:00
Adil Hafeez
7b51cce2f7
fix demo broken links
2024-10-10 17:52:33 -07:00
Adil Hafeez
50bea9135c
update broken link
2024-10-10 17:50:57 -07:00
Adil Hafeez
7d5f760884
Improve cli ( #179 )
2024-10-10 17:44:41 -07:00
Aayush
ceca0dba28
fix prometheus target and update dashboard. ( #165 )
...
* fix prometheus target and update dashboard.
* Update envoy_overview.json with whitespace at the end
2024-10-10 14:57:20 -07:00
Co Tran
2c45de26e6
fix for linux ( #175 )
...
* fix for linux
* fix pre commit
* fix
* fix extra white space
* fix commit
2024-10-10 14:56:23 -07:00
Salman Paracha
95a0f1be5b
updated archgw cli to pull from archgw_modelserver from pypi ( #169 )
...
* updated archgw cli to pull from archgw_modelserver from pypi
* fix image name
* update rev
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
Co-authored-by: Adil Hafeez <adil@katanemo.com>
2024-10-09 21:00:26 -07:00
Adil Hafeez
6b70768170
make ratelimit section optional ( #168 )
2024-10-09 19:53:00 -07:00
Salman Paracha
dc3a9813c3
fixed function calling arch config yaml
2024-10-09 17:10:54 -07:00
Adil Hafeez
3e9327cf36
fix bug in jinja template for tracing
2024-10-09 16:44:50 -07:00
Salman Paracha
1acf43ff7a
fixed cli to use poetry as well. this way we make it easy to have the… ( #160 )
2024-10-09 15:53:12 -07:00
Adil Hafeez
e81ca8d5cf
llm listener split ( #155 )
2024-10-09 15:47:32 -07:00
