apunkt/plano
1
0
Fork 0
mirror of https://github.com/katanemo/plano.git synced 2026-04-29 10:56:35 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

hallucination with log probs (#281)

Browse source 
* first init

* fix

* fix test

* new implemenetation

* fix bug

* fix bug

* fix bug

* address issue

* address issues

* address comments

* fix test

* fix

* move constatns

* remove consts
This commit is contained in:
CTran 2024-11-27 15:17:02 -08:00 • committed by GitHub
parent f5cdafb7c8
commit cadd3cdaf9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 1269 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
model_server/app/commons/constants.py
View file

@ -19,6 +19,7 @@ arch_function_generation_params = {
"top_k": 50,
"max_tokens": 512,
"stop_token_ids": [151645],
# "top_logprobs": 10,
}
arch_guard_model_type = {
@ -34,3 +35,4 @@ zero_shot_model = loader.get_zero_shot_model()
prompt_guard_dict = loader.get_prompt_guard(arch_guard_model_type[glb.DEVICE])
arch_guard_handler = ArchGuardHanlder(model_dict=prompt_guard_dict)
# Patterns for function name and parameter parsing