* pushing draft PR
* transformations are working. Now need to add some tests next
* updated tests and added necessary response transformations for Anthropics' message response object
* fixed bugs for integration tests
* fixed doc tests
* fixed serialization issues with enums on response
* adding some debug logs to help
* fixed issues with non-streaming responses
* updated the stream_context to update response bytes
* the serialized bytes length must be set in the response side
* fixed the debug statement that was causing the integration tests for wasm to fail
* fixing json parsing errors
* intentionally removing the headers
* making sure that we convert the raw bytes to the correct provider type upstream
* fixing non-streaming responses to tranform correctly
* /v1/messages works with transformations to and from /v1/chat/completions
* updating the CLI and demos to support anthropic vs. claude
* adding the anthropic key to the preference based routing tests
* fixed test cases and added more structured logs
* fixed integration tests and cleaned up logs
* added python client tests for anthropic and openai
* cleaned up logs and fixed issue with connectivity for llm gateway in weather forecast demo
* fixing the tests. python dependency order was broken
* updated the openAI client to fix demos
* removed the raw response debug statement
* fixed the dup cloning issue and cleaned up the ProviderRequestType enum and traits
* fixing logs
* moved away from string literals to consts
* fixed streaming from Anthropic Client to OpenAI
* removed debug statement that would likely trip up integration tests
* fixed integration tests for llm_gateway
* cleaned up test cases and removed unnecessary crates
* fixing comments from PR
* fixed bug whereby we were sending an OpenAIChatCompletions request object to llm_gateway even though the request may have been AnthropicMessages
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-4.local>
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-9.local>
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-10.local>
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-41.local>
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-136.local>
* add support for custom llm with ssl support
Add support for using custom llm that are served through https protocol.
* add instructions on how to add custom inference endpoint
* fix formatting
* add more details
* Apply suggestions from code review
Co-authored-by: Salman Paracha <salman.paracha@gmail.com>
* Apply suggestions from code review
* fix precommit
---------
Co-authored-by: Salman Paracha <salman.paracha@gmail.com>
* first commit to have model_server not be dependent on Docker
* making changes to fix the docker-compose file for archgw to set DNS_V4 and minor fixes with the build
* additional fixes for model server to be separated out in the build
* additional fixes for model server to be separated out in the build
* fix to get model_server to be built as a separate python process. TODO: fix the embeddings logs after cli completes
* fixing init to pull tempfile using the tempfile python package
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
* initial commit of the insurange agent demo, with the CLI tool
* committing the cli
* fixed some field descriptions for generate-prompt-targets
* CLI works with buil, up and down commands. Function calling example works stand-alone
* fixed README to install archgw cli
* fixing based on feedback
* fixing based on feedback
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
* change default stat_prefix from ingress_http to arch
* Update arch/envoy.template.yaml
Co-authored-by: Adil Hafeez <adil@katanemo.com>
---------
Co-authored-by: Adil Hafeez <adil@katanemo.com>
* Fixed build. Now, we have a bare bones version of the docker-compose file with only two services, archgw and archgw-model-server. Tested using CLI
* some pre-commit fixes
* fixed cargo formatting issues
* fixed model server conflict changes
---------
Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
