apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-25 00:16:23 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
1312 commits 76 branches 497 tags 113 MiB
Commit graph

121 commits

Author SHA1 Message Date
cybermaggedon
fa30544999
Fix/revert template change (#71) 
* Ditched the deploy directory (going away in 0.11) and putting
YAML files in top-dir of Github (for now).

* Update Makefile for the template change
 2024-09-29 18:13:34 +01:00
Cyber MacGeddon
e5249c2bac Bump version 2024-09-29 18:03:32 +01:00
Cyber MacGeddon
1d89e466d5 Docker hub login 2024-09-28 11:55:30 +01:00
Cyber MacGeddon
d6cacd3fb7 Bump version 2024-09-28 11:50:57 +01:00
Cyber MacGeddon
39cf256f5c Merge branch 'master' into release/v0.10. Includes llamafile and 
prompt modifications.
 2024-09-28 11:26:52 +01:00
Jack Colquitt
9612a11581
Added basic Llamafile integration (#63) 
* Added basic Llamafile integration
* Added llamafile template support
* New templates following llamafile addition
---------
Co-authored-by: Cyber MacGeddon <cybermaggedon@gmail.com>
 2024-09-16 16:18:01 +01:00
cybermaggedon
6af86fa09f
Topic templates for extractor (#62) 
* Add topic prompt to templates
* Bump version
* Updated templates
 2024-09-15 23:40:37 +01:00
cybermaggedon
0ae6feddb0
Added GCP and Minikube output (#59) 
* Added a config to create Minikube k8s, uses hostpath volumes
* Reworked templater to produce docker compose and minikube output
* Fix config templates
 2024-09-09 17:16:50 +01:00
cybermaggedon
f661791bbf
K8s (#58) 
Added templates which produce K8s resources.  With the provided GCP wrapper, it works on GCP K8s cluster.  This isn't stable enough for other folks to use so will need more piloting before it can be documented and released.
 2024-09-07 18:59:38 +01:00
cybermaggedon
375b213a54
Fix replace strings for JSON (#57) 
* Fix replace strings for JSON
* Add JSON markdown preamble/postamble in prompts
* Remove JSON chicanery in LLM  handling
* Bump version
 2024-09-05 20:49:32 +01:00
cybermaggedon
3445759598
Fix bedrock replace (#56) 
* Fix replace ignoring first output

* Bump version
 2024-09-05 20:00:38 +01:00
cybermaggedon
ddd8cc90e5
Fix templates (#55) 
* Fix template import problem

* Bump version
 2024-09-05 18:24:51 +01:00
Cyber MacGeddon
6fb118ba95 Bump version 2024-09-05 16:47:49 +01:00
cybermaggedon
65d7f6d261
Refactor templates (#52) 
* Switching from docker compose to abstract form - should be easier to k8s later
* Text loader util
* Recreate templates
 2024-09-05 16:40:47 +01:00
cybermaggedon
af5508ff68
Cannot access s error (#50) 
* Fix order of statements error
* Bump version
* Update templates
* Add missing script files
* Added missing DE support init
* Fixed bugs preventing definition extraction from working (#49).
 2024-09-03 22:10:48 +01:00
cybermaggedon
208c219962
Template rejig (#48) 
* document-rag / graph-rag refactor of templates

* Tweaking the docs and categories

* Clarify triple store vs RAG

* Tweak knowledge graph linkage

* Doc embedding for Qdrant

* Fix document RAG on Qdrant

* Fix templates

* Bump version

* Updated templates
 2024-09-03 00:09:15 +01:00
cybermaggedon
f7a30006ad
Make templating work more flexibly (#44) 
* Restructure directory
* Config loading
* Variable override points in JSONNET templates, separate pulsar-manager template
* Bump version
* Tidy chunking
* Simplified prompt overrides
* Update config loader
* Fix recursive chunker template
 2024-08-30 17:47:35 +01:00
Cyber MacGeddon
937dad3381 Version to 0.8.0 2024-08-27 23:40:40 +01:00
cybermaggedon
32b087fbf6
Switch Milvus for Qdrant in YAMLs (#43) 
* Qdrant working

* - Fix missing prompt templates
- Bump version
- Add Qdrant to packages

* Switch Milvus for Qdrant in config files
 2024-08-27 23:37:24 +01:00
cybermaggedon
e4c4774b5d
Extract rows and apply object embeddings (#42) 
* - Restructured the extract directories
- Added an extractor for 'rows' == a row of a table
- Added a row extractor prompt to prompter.
* Add row support to template prompter
* Row extraction working
* Bump version
* Emit extracted info
* Object embeddings store
* Invocation script
* Add script to package, remove cruft output
* Write rows to Cassandra
* Remove output cruft
 2024-08-27 21:55:12 +01:00
cybermaggedon
669aed0f8a
Added doc embedding support (#41) 
* document embedding writer & query
* Added test query for doc embeddings
* Bump version
* Added doc rag prompt
* Document RAG service
 2024-08-26 23:45:23 +01:00
cybermaggedon
0159e938a2
Update LLM text-completion duration metric (#40) 
* Added LLM duration metric, better buckets

* Added heatmap to dashboard to replace 95/97/99 chart

* Bump version
 2024-08-26 11:46:36 +01:00
cybermaggedon
d0e3fcf019
Catch LLM mismatches (#39) 
* Catch more upstream LLM issues

* Bump version
 2024-08-26 10:58:02 +01:00
cybermaggedon
acd60e95ec
Catch llm errors (#38) 
* Catch 'null' output from prompt for some values, presumably this is
caused by an upstream LLM error.

* Bump version
 2024-08-26 10:52:39 +01:00
cybermaggedon
cea8562ecf
Fix timeouts (#37) 
* Fix other timeout default settings

* Add storage-only YAML output

* Bump version
 2024-08-25 23:57:30 +01:00
cybermaggedon
3ca1defc88
Increase timeout (#36) 
* Increase timeout
* Bump version
 2024-08-25 20:45:04 +01:00
cybermaggedon
d69de52b04
Increase resources (#35) 
* More memory for Cassandra
* More memory/CPU for embeddings
* Bump version, regenerate templates
 2024-08-25 20:38:19 +01:00
cybermaggedon
6edc3f0ee1
Prompt templates (#33) 
* Added prompt-template, allows definiton, relationships and kg query
to be specified in config / command-line.

* Bump version & add prompt-templates to YAMLs

* Apply to graph rag flow

* Break out different templates
 2024-08-23 23:34:16 +01:00
cybermaggedon
6d0776c7bb
Fix OpenAI reporting (#32) 
* Fix OpenAI reporting

* bump version
 2024-08-23 14:02:06 +01:00
cybermaggedon
380bddeb90
Fix LLM output reporting (#31) 
* Fix LLM output reporting

* Bump version
 2024-08-23 13:44:42 +01:00
cybermaggedon
1e92b2048a
Fix client missing exception import (#30) 
* Fix missing import, remove cruft imports

* Bump version
 2024-08-23 12:59:14 +01:00
cybermaggedon
e7c498be92
Fix neo4j (#29) 
* - Fix Neo4j memory
- Fix neo4j query

* Version to 0.7.6
 2024-08-23 12:49:42 +01:00
cybermaggedon
8372ff0eb6
Fix exception import fail (#28) 
* Fix a missing import

* Fix missing  import, bump version
 2024-08-22 23:59:26 +01:00
cybermaggedon
b1b26a3f55
- Updated dashboard (#27) 
- Adjusted limits everything works
- Bump version
 2024-08-22 23:23:11 +01:00
cybermaggedon
2bc602fdcf
Fix, was putting Neo in Cassandra config (#26) 2024-08-22 22:02:57 +01:00
cybermaggedon
a2ae1d8820
Generate all YAML files (#24) 
* All templates generated, added missing file

* Up version
 2024-08-22 21:20:17 +01:00
cybermaggedon
305dda4463
Fix errors in previous update (#23) 
* Increase some limits
* Fix msg errors and update version
 2024-08-22 20:58:44 +01:00
cybermaggedon
a01a72ba00
Set resource limits (#22) 
* Added resource limits to resources.

* Boost version number, rebuild YAMLs
 2024-08-22 17:54:00 +01:00
cybermaggedon
86cbe7f929
- Version to 0.7.0 (#19) 
- Tweak Containerfile to add more dependencies, speed up container
  build.
 2024-08-22 17:02:36 +01:00
Cyber MacGeddon
f09618081e Version 0.6.10 2024-08-22 00:21:42 +01:00
cybermaggedon
d3cdb97528
Fix auto-templates (#17) 2024-08-22 00:19:55 +01:00
Jack Colquitt
c4bfd9fc8c
Parameters, Parsing, renaming YAMLs and Neo4j YAMLS (#15) 
* Added some params
* Parameter updates
* Fixed Neo4j issue
 2024-08-22 00:03:56 +01:00
cybermaggedon
7113d04f21
Add token chunker (#14) 2024-08-21 16:51:33 +01:00
cybermaggedon
0e2db095e3
Add a docker-compose for just the stores (#13) 
* - Added docker-compose-storage.yaml, just the infrastructure bits
- Tidied storage invocation

* Util, sits on chunker output and reports histogram of chunk sizes
 2024-08-21 16:20:21 +01:00
Cyber MacGeddon
b0fdb4f314 Version to 0.6.6 2024-08-20 22:14:13 +01:00
Cyber MacGeddon
ba056b93ed Catch JSON parse errors in prompt processor 2024-08-20 20:51:32 +01:00
Cyber MacGeddon
20f983eec9 - Change flawed _client timeout logic which was causing major lags 
- Moved clients to trustgraph.clients to tidy the parent directory
- Version bump
 2024-08-20 17:54:11 +01:00
cybermaggedon
a38f530c5f
Rate limit handling (#11) 
* Added a rate limit exception
* Reduce request/response timeouts because looks like there are major issues
* Add rate limit exception catch to all consumers
* Version to 0.6.3
 2024-08-19 22:15:32 +01:00
cybermaggedon
fa0b89b5d4
Simplify templates (#10) 
- Add component template files for all LLM types
- Top-level templates simplified to use just components
- Version to 0.6.2
 2024-08-14 20:56:57 +01:00
cybermaggedon
d3e213f194
Add Neo4j support (#9) 
- Add triples-write-neo4j and triples-query-neo4j to interact with neo4j
- Add docker-compose-openai-neo4j to demo Neo4j working
 2024-08-14 09:06:33 +01:00