apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-05-02 04:12:56 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Refactor model server hardware config + add unit tests to load/request to the server (#189)

Browse source 
* remove mode/hardware

* add test and pre commit hook

* add pytest dependieces

* fix format

* fix lint

* fix precommit

* fix pre commit

* fix pre commit

* fix precommit

* fix precommit

* fix precommit

* fix precommit

* fix precommit

* fix precommit

* fix precommit

* fix precommit

* fix precommit

* fix precommit
This commit is contained in:
CTran 2024-10-16 16:58:10 -07:00 • committed by GitHub
parent 3bd2ffe9fb
commit 8e54ac20d8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 480 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

2
model_server/app/main.py
View file

@ -13,6 +13,7 @@ from app.commons.constants import embedding_model, zero_shot_model, arch_guard_h
from app.function_calling.model_utils import (
chat_completion as arch_function_chat_completion,
)
from unittest.mock import patch
logger = utils.get_model_server_logger()
@ -173,6 +174,7 @@ async def zeroshot(req: ZeroShotRequest, res: Response):
@app.post("/hallucination")
@patch("app.loader.glb.DEVICE", "cpu") # Mock the device to 'cpu'
async def hallucination(req: HallucinationRequest, res: Response):
"""
Take input as text and return the prediction of hallucination for each parameter