cybermaggedon
c26ada08c2
Fix VertexAI package. Add Python packaging to Makefile. ( #85 )
...
Bump version & generate templates.
2024-09-30 20:50:20 +01:00
cybermaggedon
f00baab1b8
Maint/fix build env ( #84 )
...
* Put README placeholders for packages in place
* Bump version
2024-09-30 19:47:09 +01:00
cybermaggedon
9b91d5eee3
Feature/pkgsplit ( #83 )
...
* Starting to spawn base package
* More package hacking
* Bedrock and VertexAI
* Parquet split
* Updated templates
* Utils
2024-09-30 19:36:09 +01:00
cybermaggedon
3fb75c617b
Maint/auto pkg versions ( #82 )
...
* Remove need to manage setup.py version
* Update YAMLs
2024-09-30 16:38:50 +01:00
cybermaggedon
cdace22ee4
Feature/simpler subpackages ( #81 )
...
* Back to simpler directory structure
* Bump version, update templates
2024-09-30 16:16:20 +01:00
cybermaggedon
f081933217
Feature/subpackages ( #80 )
...
* Renaming what will become the core package
* Tweaking to get package build working
* Fix metering merge
* Rename to core directory
* Bump version. Use namespace searching for packaging trustgraph-core
* Change references to trustgraph-core
* Forming embeddings-hf package
* Reference modules in core package.
* Build both packages to one container, bump version
* Update YAMLs
2024-09-30 14:00:29 +01:00
cybermaggedon
14d79ef9f1
Streamline startup ( #79 )
...
* Separate Prom metrics, different processors as different jobs
* Create producers before consumers, may streamline startup.
* Bump version
* Add Pulsar init command, will replace pulsar-admin invocations.
* Integrate tg-init-pulsar with YAMLs
* Update YAMLs
2024-09-30 12:19:22 +01:00
Cyber MacGeddon
5e8a1520ee
Bump version & update templates
2024-09-30 00:00:36 +01:00
Jack Colquitt
2f23309f05
Metering for all LLMs ( #76 )
...
* Fixed VertexAI token counts
* Slight fix for Cohere token count
* Slight tweak to AzureAI
* Fix for prometheus for AzureAI
2024-09-29 23:59:17 +01:00
cybermaggedon
74a14639bd
Feature/track processor state ( #78 )
...
* Add a Prom metric to consumers & consumer/producers to track the running
state.
* New script, gets processor state using prometheus
* Bump version, add tg-processor-state to package
* Update templates
2024-09-29 23:50:57 +01:00
cybermaggedon
efc364583b
Fix/graph rag uses wrong prompt ( #77 )
...
* Fix queue name invocation, use correct names, not defaults
* Bump version
* Update templates
2024-09-29 20:38:50 +01:00
cybermaggedon
90127dfecb
Feature/rag metering ( #75 )
...
* Add metering-rag for RAG token metrics
* Make metering executable
* Add metering-rag to Prom metrics
* Metricate the right queue
* Update templates to include metering-rag
2024-09-29 20:15:03 +01:00
cybermaggedon
6d57b7a6f3
Add metering to templates ( #74 )
...
* Add metering to trustgraph-core component
* Regenerate templates to run metering
2024-09-29 19:48:35 +01:00
cybermaggedon
56b3b50e94
Fix/cohere temperature not string ( #73 )
...
* Temperature should be a string in the templates
* Fix YAMLs for Cohere
2024-09-29 19:37:49 +01:00
cybermaggedon
24b80f4789
Fix/template config vol broken ( #72 )
...
* Fix config volume invocation
* Regenerate YAMLs
2024-09-29 18:33:39 +01:00
Cyber MacGeddon
5951fb4e56
Bump version to 0.11.1
2024-09-29 18:15:34 +01:00
cybermaggedon
fa30544999
Fix/revert template change ( #71 )
...
* Ditched the deploy directory (going away in 0.11) and putting
YAML files in top-dir of Github (for now).
* Update Makefile for the template change
2024-09-29 18:13:34 +01:00
Jack Colquitt
88246742f3
Features/metering all llms ( #70 )
...
* Added Anthropic support and None logic
* Added Cohere API support
* Added support for Llamafile and OpenAI
* Added support for VertexAI
* Added AzureAI support
2024-09-29 18:11:48 +01:00
Cyber MacGeddon
e5249c2bac
Bump version
2024-09-29 18:03:32 +01:00
cybermaggedon
6cad3674c0
Update templates to 0.11 ( #69 )
2024-09-28 22:08:28 +01:00
cybermaggedon
2a49365482
Adds basic metering infrastructure ( #68 )
...
* Basic metering module structure
* Token counting working for Bedrock
* Price calc using price list
* Added more models to pricelist
* Added Ollama token counts
----
Authored-by: JackColquitt <daniel@kalntera.ai>
2024-09-28 20:48:20 +01:00
cybermaggedon
8085bb0118
Add version number to template generation ( #67 )
2024-09-28 19:44:29 +01:00
cybermaggedon
92a3e9816c
Script for deploy-bundle generation ( #65 )
...
* Added output config generation to ZIP file
* Added a README which is bundled with the ZIP file
* Remove CONFIGDIR reference
2024-09-28 17:46:41 +01:00
Cyber MacGeddon
1d89e466d5
Docker hub login
2024-09-28 11:55:30 +01:00
Cyber MacGeddon
2c8dd0bd39
Updated templates
2024-09-28 11:54:00 +01:00
Cyber MacGeddon
d6cacd3fb7
Bump version
2024-09-28 11:50:57 +01:00
Cyber MacGeddon
39cf256f5c
Merge branch 'master' into release/v0.10. Includes llamafile and
...
prompt modifications.
2024-09-28 11:26:52 +01:00
JackColquitt
a2a93ebaff
Updated agent messaging in readme
2024-09-24 14:24:16 -07:00
JackColquitt
78e3418dcb
Readme tweaks
2024-09-23 15:01:23 -07:00
JackColquitt
e2c326c626
Added sample files
2024-09-23 14:21:50 -07:00
JackColquitt
124cb20932
More readme tweaks
2024-09-23 14:14:30 -07:00
JackColquitt
74094d4471
Link tweak in readme
2024-09-23 14:07:17 -07:00
JackColquitt
99370670de
Readme tweaks
2024-09-23 14:05:01 -07:00
JackColquitt
a6f38fc214
Tidy up readme
2024-09-23 13:59:02 -07:00
JackColquitt
543c10090f
Readme tweaks
2024-09-22 10:41:49 -07:00
JackColquitt
28c2815ac3
Fixed readme typos
2024-09-16 17:53:47 -07:00
JackColquitt
28a28b9f0a
Updated readme
2024-09-16 17:52:02 -07:00
JackColquitt
7818178f41
Updated readme
2024-09-16 10:37:53 -07:00
Jack Colquitt
9612a11581
Added basic Llamafile integration ( #63 )
...
* Added basic Llamafile integration
* Added llamafile template support
* New templates following llamafile addition
---------
Co-authored-by: Cyber MacGeddon <cybermaggedon@gmail.com>
2024-09-16 16:18:01 +01:00
cybermaggedon
6af86fa09f
Topic templates for extractor ( #62 )
...
* Add topic prompt to templates
* Bump version
* Updated templates
2024-09-15 23:40:37 +01:00
Jack Colquitt
728ff7542a
Extraction upgrade ( #61 )
...
* Added KG Topics
* Updated prompt-template
* Fixed prompt-generic
2024-09-15 22:47:57 +01:00
JackColquitt
13f6b5d87f
Updated readme
2024-09-11 10:10:39 -07:00
cybermaggedon
054ea6c76a
Fix Docker Compose relative directory problem ( #60 )
...
- Fix Docker Compose relative directory problem
- Added CONFIGDIR environment variable for docker compose
- Correct Pulsar URL defaults for graph utils to be localhost again
2024-09-09 22:03:10 +01:00
cybermaggedon
0ae6feddb0
Added GCP and Minikube output ( #59 )
...
* Added a config to create Minikube k8s, uses hostpath volumes
* Reworked templater to produce docker compose and minikube output
* Fix config templates
2024-09-09 17:16:50 +01:00
cybermaggedon
f661791bbf
K8s ( #58 )
...
Added templates which produce K8s resources. With the provided GCP wrapper, it works on GCP K8s cluster. This isn't stable enough for other folks to use so will need more piloting before it can be documented and released.
2024-09-07 18:59:38 +01:00
JackColquitt
7af32b0eef
Doc updates for v0.9.3
2024-09-05 14:01:41 -07:00
cybermaggedon
375b213a54
Fix replace strings for JSON ( #57 )
...
* Fix replace strings for JSON
* Add JSON markdown preamble/postamble in prompts
* Remove JSON chicanery in LLM handling
* Bump version
2024-09-05 20:49:32 +01:00
cybermaggedon
3445759598
Fix bedrock replace ( #56 )
...
* Fix replace ignoring first output
* Bump version
2024-09-05 20:00:38 +01:00
cybermaggedon
ddd8cc90e5
Fix templates ( #55 )
...
* Fix template import problem
* Bump version
2024-09-05 18:24:51 +01:00
cybermaggedon
f7429f2f9b
Fix remaining templates ( #54 )
2024-09-05 18:17:47 +01:00
