apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-06-08 14:55:14 +02:00
Code Issues Projects Releases Packages Wiki Activity
Plano is an AI-native proxy and data plane for agentic apps — with built-in orchestration, safety, observability, and smart LLM routing so you stay focused on your agents core logic. https://planoai.dev
85 commits 519 branches 67 tags 89 MiB
Find a file
Co Tran 79b1c5415f
[Kan-103] add support toxic/jailbreak model (#49) 
* add toxic/jailbreak model

* fix path loading model

* fix syntax

* fix bug,lint, format

* fix bug

* formatting

* add parallel + chunking

* fix bug

* working version

* fix onnnx name erorr

* device

* fix jailbreak config

* fix syntax error

* format

* add requirement + cli download for dockerfile

* add task

* add skeleton change for envoy filter for prompt guard

* fix hardware config

* fix bug

* add config changes

* add gitignore

* merge main

* integrate arch-guard with filter

* add hardware config

* nothing

* add hardware config feature

* fix requirement

* fix chat ui

* fix onnx

* fix lint

* remove non intel cpu

* remove onnx

* working version

* modify docker

* fix guard time

* add nvidia support

* remove nvidia

* add gpu

* add gpu

* add gpu support

* add gpu support for compose

* add gpu support for compose

* add gpu support for compose

* add gpu support for compose

* add gpu support for compose

* fix docker file

* fix int test

* correct gpu docker

* upgrad python 10

* fix logits to be gpu compatible

* default to cpu dockerfile

* resolve comments

* fix lint + unused parameters

* fix

* remove eetq install for cpu

* remove deploy gpu

---------

Co-authored-by: Adil Hafeez <adil@katanemo.com>
2024-09-23 12:07:31 -07:00
.github/workflows Add initial integration style tests (#20) 2024-07-25 14:41:36 -07:00
chatbot_ui move demo functions out of model_server (#67) 2024-09-20 14:38:10 -07:00
config_generator Add ability to define clusters in config (#61) 2024-09-18 20:03:26 -07:00
demos [Kan-103] add support toxic/jailbreak model (#49) 2024-09-23 12:07:31 -07:00
docs Docs branch - v1 of our tech docs (#69) 2024-09-20 17:08:42 -07:00
envoyfilter [Kan-103] add support toxic/jailbreak model (#49) 2024-09-23 12:07:31 -07:00
function_resolver use openai standard response in arch-fc and in gradio client (#62) 2024-09-19 12:19:14 -07:00
model_server [Kan-103] add support toxic/jailbreak model (#49) 2024-09-23 12:07:31 -07:00
open-message-format@1e838f3f40 update open-message-format (#30) 2024-07-31 15:56:13 -07:00
public_types [Kan-103] add support toxic/jailbreak model (#49) 2024-09-23 12:07:31 -07:00
.gitignore Docs branch - v1 of our tech docs (#69) 2024-09-20 17:08:42 -07:00
.gitmodules Add function calling support using bolt-fc-1b (#35) 2024-09-10 14:24:46 -07:00
.pre-commit-config.yaml Add ability to define clusters in config (#61) 2024-09-18 20:03:26 -07:00
gateway.code-workspace move demo functions out of model_server (#67) 2024-09-20 14:38:10 -07:00
README.md Update README.md 2024-09-10 14:27:14 -07:00

README.md

A open source project for developers to build and secure faster, more personalized generative AI apps. Katanemo is a high performance gateway designed with state of the art (SOTA) fast LLMs to process, route and evaluate prompts.

Demos

Complete

In progress

  • Network Co-pilot

Not Started

  • Show routing between different prompt targets (keyword search vs. top-k semantic search).
  • Show routing between different prompt-resolver vs RAG-based resolver targets.
  • Text Summarization Based on Lightweight vs. Thoughtful Dialogue using OpenAI
  • Show conversational and system observability metrics. This includes topic/intent detection
  • Show how we can help developers implement safeguards customized to their application requirements and responsible AI policies.

Dev setup

Pre-commit

Use instructions at pre-commit.com to set it up for your machine. Once installed make sure github hooks are setup, so that when you upstream your change pre-commit hooks can run and validate your change. Follow command below to setup github hooks,

$ brew install pre-commit
$ pre-commit install
pre-commit installed at .git/hooks/pre-commit