apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-26 00:46:22 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
206 commits 76 branches 497 tags 113 MiB
Commit graph

75 commits

Author SHA1 Message Date
Cyber MacGeddon
5951fb4e56 Bump version to 0.11.1 2024-09-29 18:15:34 +01:00
Cyber MacGeddon
e5249c2bac Bump version 2024-09-29 18:03:32 +01:00
cybermaggedon
2a49365482
Adds basic metering infrastructure (#68) 
* Basic metering module structure
* Token counting working for Bedrock
* Price calc using price list
* Added more models to pricelist
* Added Ollama token counts
----
Authored-by: JackColquitt <daniel@kalntera.ai>
 2024-09-28 20:48:20 +01:00
Cyber MacGeddon
d6cacd3fb7 Bump version 2024-09-28 11:50:57 +01:00
Cyber MacGeddon
39cf256f5c Merge branch 'master' into release/v0.10. Includes llamafile and 
prompt modifications.
 2024-09-28 11:26:52 +01:00
Jack Colquitt
9612a11581
Added basic Llamafile integration (#63) 
* Added basic Llamafile integration
* Added llamafile template support
* New templates following llamafile addition
---------
Co-authored-by: Cyber MacGeddon <cybermaggedon@gmail.com>
 2024-09-16 16:18:01 +01:00
cybermaggedon
6af86fa09f
Topic templates for extractor (#62) 
* Add topic prompt to templates
* Bump version
* Updated templates
 2024-09-15 23:40:37 +01:00
Jack Colquitt
728ff7542a
Extraction upgrade (#61) 
* Added KG Topics
* Updated prompt-template
* Fixed prompt-generic
 2024-09-15 22:47:57 +01:00
cybermaggedon
0ae6feddb0
Added GCP and Minikube output (#59) 
* Added a config to create Minikube k8s, uses hostpath volumes
* Reworked templater to produce docker compose and minikube output
* Fix config templates
 2024-09-09 17:16:50 +01:00
cybermaggedon
375b213a54
Fix replace strings for JSON (#57) 
* Fix replace strings for JSON
* Add JSON markdown preamble/postamble in prompts
* Remove JSON chicanery in LLM  handling
* Bump version
 2024-09-05 20:49:32 +01:00
cybermaggedon
3445759598
Fix bedrock replace (#56) 
* Fix replace ignoring first output

* Bump version
 2024-09-05 20:00:38 +01:00
cybermaggedon
ddd8cc90e5
Fix templates (#55) 
* Fix template import problem

* Bump version
 2024-09-05 18:24:51 +01:00
Cyber MacGeddon
6fb118ba95 Bump version 2024-09-05 16:47:49 +01:00
cybermaggedon
6e4534e35c
Tidied scripts, added 2 query scripts (#53) 2024-09-05 16:45:22 +01:00
cybermaggedon
af5508ff68
Cannot access s error (#50) 
* Fix order of statements error
* Bump version
* Update templates
* Add missing script files
* Added missing DE support init
* Fixed bugs preventing definition extraction from working (#49).
 2024-09-03 22:10:48 +01:00
cybermaggedon
208c219962
Template rejig (#48) 
* document-rag / graph-rag refactor of templates

* Tweaking the docs and categories

* Clarify triple store vs RAG

* Tweak knowledge graph linkage

* Doc embedding for Qdrant

* Fix document RAG on Qdrant

* Fix templates

* Bump version

* Updated templates
 2024-09-03 00:09:15 +01:00
cybermaggedon
f7a30006ad
Make templating work more flexibly (#44) 
* Restructure directory
* Config loading
* Variable override points in JSONNET templates, separate pulsar-manager template
* Bump version
* Tidy chunking
* Simplified prompt overrides
* Update config loader
* Fix recursive chunker template
 2024-08-30 17:47:35 +01:00
Cyber MacGeddon
937dad3381 Version to 0.8.0 2024-08-27 23:40:40 +01:00
cybermaggedon
32b087fbf6
Switch Milvus for Qdrant in YAMLs (#43) 
* Qdrant working

* - Fix missing prompt templates
- Bump version
- Add Qdrant to packages

* Switch Milvus for Qdrant in config files
 2024-08-27 23:37:24 +01:00
cybermaggedon
e4c4774b5d
Extract rows and apply object embeddings (#42) 
* - Restructured the extract directories
- Added an extractor for 'rows' == a row of a table
- Added a row extractor prompt to prompter.
* Add row support to template prompter
* Row extraction working
* Bump version
* Emit extracted info
* Object embeddings store
* Invocation script
* Add script to package, remove cruft output
* Write rows to Cassandra
* Remove output cruft
 2024-08-27 21:55:12 +01:00
cybermaggedon
669aed0f8a
Added doc embedding support (#41) 
* document embedding writer & query
* Added test query for doc embeddings
* Bump version
* Added doc rag prompt
* Document RAG service
 2024-08-26 23:45:23 +01:00
cybermaggedon
0159e938a2
Update LLM text-completion duration metric (#40) 
* Added LLM duration metric, better buckets

* Added heatmap to dashboard to replace 95/97/99 chart

* Bump version
 2024-08-26 11:46:36 +01:00
cybermaggedon
d0e3fcf019
Catch LLM mismatches (#39) 
* Catch more upstream LLM issues

* Bump version
 2024-08-26 10:58:02 +01:00
cybermaggedon
acd60e95ec
Catch llm errors (#38) 
* Catch 'null' output from prompt for some values, presumably this is
caused by an upstream LLM error.

* Bump version
 2024-08-26 10:52:39 +01:00
cybermaggedon
cea8562ecf
Fix timeouts (#37) 
* Fix other timeout default settings

* Add storage-only YAML output

* Bump version
 2024-08-25 23:57:30 +01:00
cybermaggedon
3ca1defc88
Increase timeout (#36) 
* Increase timeout
* Bump version
 2024-08-25 20:45:04 +01:00
cybermaggedon
d69de52b04
Increase resources (#35) 
* More memory for Cassandra
* More memory/CPU for embeddings
* Bump version, regenerate templates
 2024-08-25 20:38:19 +01:00
cybermaggedon
6edc3f0ee1
Prompt templates (#33) 
* Added prompt-template, allows definiton, relationships and kg query
to be specified in config / command-line.

* Bump version & add prompt-templates to YAMLs

* Apply to graph rag flow

* Break out different templates
 2024-08-23 23:34:16 +01:00
cybermaggedon
6d0776c7bb
Fix OpenAI reporting (#32) 
* Fix OpenAI reporting

* bump version
 2024-08-23 14:02:06 +01:00
cybermaggedon
380bddeb90
Fix LLM output reporting (#31) 
* Fix LLM output reporting

* Bump version
 2024-08-23 13:44:42 +01:00
cybermaggedon
1e92b2048a
Fix client missing exception import (#30) 
* Fix missing import, remove cruft imports

* Bump version
 2024-08-23 12:59:14 +01:00
cybermaggedon
e7c498be92
Fix neo4j (#29) 
* - Fix Neo4j memory
- Fix neo4j query

* Version to 0.7.6
 2024-08-23 12:49:42 +01:00
cybermaggedon
8372ff0eb6
Fix exception import fail (#28) 
* Fix a missing import

* Fix missing  import, bump version
 2024-08-22 23:59:26 +01:00
cybermaggedon
b1b26a3f55
- Updated dashboard (#27) 
- Adjusted limits everything works
- Bump version
 2024-08-22 23:23:11 +01:00
cybermaggedon
a2ae1d8820
Generate all YAML files (#24) 
* All templates generated, added missing file

* Up version
 2024-08-22 21:20:17 +01:00
cybermaggedon
305dda4463
Fix errors in previous update (#23) 
* Increase some limits
* Fix msg errors and update version
 2024-08-22 20:58:44 +01:00
cybermaggedon
a01a72ba00
Set resource limits (#22) 
* Added resource limits to resources.

* Boost version number, rebuild YAMLs
 2024-08-22 17:54:00 +01:00
cybermaggedon
86cbe7f929
- Version to 0.7.0 (#19) 
- Tweak Containerfile to add more dependencies, speed up container
  build.
 2024-08-22 17:02:36 +01:00
Cyber MacGeddon
f09618081e Version 0.6.10 2024-08-22 00:21:42 +01:00
Jack Colquitt
c4bfd9fc8c
Parameters, Parsing, renaming YAMLs and Neo4j YAMLS (#15) 
* Added some params
* Parameter updates
* Fixed Neo4j issue
 2024-08-22 00:03:56 +01:00
cybermaggedon
7113d04f21
Add token chunker (#14) 2024-08-21 16:51:33 +01:00
cybermaggedon
0e2db095e3
Add a docker-compose for just the stores (#13) 
* - Added docker-compose-storage.yaml, just the infrastructure bits
- Tidied storage invocation

* Util, sits on chunker output and reports histogram of chunk sizes
 2024-08-21 16:20:21 +01:00
Cyber MacGeddon
b0fdb4f314 Version to 0.6.6 2024-08-20 22:14:13 +01:00
Cyber MacGeddon
ba056b93ed Catch JSON parse errors in prompt processor 2024-08-20 20:51:32 +01:00
Cyber MacGeddon
20f983eec9 - Change flawed _client timeout logic which was causing major lags 
- Moved clients to trustgraph.clients to tidy the parent directory
- Version bump
 2024-08-20 17:54:11 +01:00
cybermaggedon
a38f530c5f
Rate limit handling (#11) 
* Added a rate limit exception
* Reduce request/response timeouts because looks like there are major issues
* Add rate limit exception catch to all consumers
* Version to 0.6.3
 2024-08-19 22:15:32 +01:00
cybermaggedon
fa0b89b5d4
Simplify templates (#10) 
- Add component template files for all LLM types
- Top-level templates simplified to use just components
- Version to 0.6.2
 2024-08-14 20:56:57 +01:00
cybermaggedon
d3e213f194
Add Neo4j support (#9) 
- Add triples-write-neo4j and triples-query-neo4j to interact with neo4j
- Add docker-compose-openai-neo4j to demo Neo4j working
 2024-08-14 09:06:33 +01:00
cybermaggedon
a3ea1301d6
Breakout store queries (#8) 
- Break out store queries, so not locked into a Milvus/Cassandra backend
- Break out prompting into a separate module, so that prompts can be tailored to other LLMs
- Jsonnet used to generate docker compose templates
- Version to 0.6.0
 2024-08-13 17:30:59 +01:00
cybermaggedon
fd547f7762
OpenAI integration (#7) 
* Preliminary OpenAI support
* Version to 0.5.9

---------

Co-authored-by: JackColquitt <daniel@kalntera.ai>
 2024-08-12 15:37:04 +01:00