cybermaggedon
0e03bc05a4
Refactor rate limit handling ( #280 )
...
* - Refactored retry for rate limits into the base class
- ConsumerProducer is derived from Consumer to simplify code
- Added rate_limit_count metrics for rate limit events
* Add rate limit events to VertexAI and Google AI Studio
* Added Grafana rate limit dashboard
* Add rate limit handling to all LLMs
2025-01-27 17:04:49 +00:00
cybermaggedon
56a9ac3ba9
Change LLM latency dashboard to be rate & bump version ( #92 )
2024-10-01 21:04:55 +01:00
cybermaggedon
ef1b8b5a13
Feature/metering dashboard ( #89 )
...
* Bump version
* Added Prom metrics to metering, added dashboard
* Update YAMLs
* Add $ on axis
* Tweak dashboard
2024-10-01 06:46:41 +01:00
cybermaggedon
f661791bbf
K8s ( #58 )
...
Added templates which produce K8s resources. With the provided GCP wrapper, it works on GCP K8s cluster. This isn't stable enough for other folks to use so will need more piloting before it can be documented and released.
2024-09-07 18:59:38 +01:00
cybermaggedon
0159e938a2
Update LLM text-completion duration metric ( #40 )
...
* Added LLM duration metric, better buckets
* Added heatmap to dashboard to replace 95/97/99 chart
* Bump version
2024-08-26 11:46:36 +01:00
cybermaggedon
b1b26a3f55
- Updated dashboard ( #27 )
...
- Adjusted limits everything works
- Bump version
2024-08-22 23:23:11 +01:00
cybermaggedon
25c390469f
Update dashboard with resources ( #25 )
2024-08-22 21:54:00 +01:00
cybermaggedon
19c826c387
Update dashboard for chunks & errors ( #21 )
2024-08-22 17:02:06 +01:00
cybermaggedon
0043b871ff
Added chunk_size metrics, and added metrics to dashboard ( #16 )
2024-08-22 00:20:24 +01:00
cybermaggedon
bdf4bc2bf5
Error is a heatmap ( #12 )
2024-08-19 23:35:16 +01:00
Cyber MacGeddon
9bcdee0f64
Fixed dashboard
2024-07-23 23:27:09 +01:00
Cyber MacGeddon
a7182b8f6f
Add pub/sub node graph to dashboard
2024-07-22 19:33:09 +01:00
Cyber MacGeddon
8401a7867e
Tweak dashboard
2024-07-19 12:49:52 +01:00
Cyber MacGeddon
bcbb493626
Add backlog visual
2024-07-18 19:25:10 +01:00
cybermaggedon
9ab7613e07
Metrics ( #3 )
...
* Basic metrics working
* Add consumer & producer metrics
* Grafana & Prometheus in docker compose
2024-07-18 17:20:42 +01:00
