apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-05-15 11:02:39 +02:00
Code Issues Projects Releases Packages Wiki Activity

model server build (#127)

Browse source 
* first commit to have model_server not be dependent on Docker

* making changes to fix the docker-compose file for archgw to set DNS_V4 and minor fixes with the build

* additional fixes for model server to be separated out in the build

* additional fixes for model server to be separated out in the build

* fix to get model_server to be built as a separate python process. TODO: fix the embeddings logs after cli completes

* fixing init to pull tempfile using the tempfile python package

---------

Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local>
This commit is contained in:
Salman Paracha 2024-10-06 18:21:43 -07:00 • committed by GitHub
parent 7d21359f5b
commit b60ceb9168
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 3390 additions and 154 deletions
Download patch file Download diff file Expand all files Collapse all files

9
model_server/app/utils.py
View file

@ -2,6 +2,14 @@ import numpy as np
from concurrent.futures import ThreadPoolExecutor
import time
import torch
import pkg_resources
import yaml
def load_yaml_config(file_name):
# Load the YAML file from the package
yaml_path = pkg_resources.resource_filename('app', file_name)
with open(yaml_path, 'r') as yaml_file:
return yaml.safe_load(yaml_file)
def split_text_into_chunks(text, max_words=300):
@ -21,7 +29,6 @@ def split_text_into_chunks(text, max_words=300):
def softmax(x):
return np.exp(x) / np.exp(x).sum(axis=0)
class PredictionHandler:
def __init__(self, model, tokenizer, device, task="toxic", hardware_config="cpu"):
self.model = model