I tested the new release and couldn't find any difference to the 0.9.1 version, which is good I guess :D
Hi @alpha-nerd, just returned from a sick-leave. I can't promise, but I will try to test it this week ;)
That is all right, just wanted to make sure that future tagged releases will stay available ;)
Thank you for the quick response. This explains the behavior, can this be fixed in a new release?
Yes a WRR describes this quite good. However, only if the model is already loaded.
I guess what I am describing could look like this:
if all(tracking_usage(ep) == 0 for ep in loaded_and_fr…
Firstly thank you for looking into this. I can imagine that an ideal load balancing for different endpoints is pretty tricky.
Maybe prioritization should only occur if all endpoints are…