apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-25 08:26:21 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
288 commits 76 branches 497 tags 113 MiB
Commit graph

45 commits

Author SHA1 Message Date
cybermaggedon
8a2126bba5
Fix neo4j schema - incompatible with extractors (#135) 
- Fixed schema invocation on Neo4j writer
- Fixed confusing typo name in cassandra template
- Unnecessary import on Cassandra removed
 2024-11-02 10:51:26 +00:00
cybermaggedon
7c0c471e55
Feature/prompt terms (#128) 
* Add prompt terms
* Remove dupe template code
 2024-10-27 15:04:53 +00:00
cybermaggedon
2942c4dcf0
Maint/neater prompt jsonnet (#127) 
* Changed the JSONNET for prompt-template so that prompt templates are
simpler.  This tidies default-prompts.jsonnet so it's easier to see
where the prompts are.

* Updated prompt override templates to match new structure
 2024-10-27 09:06:09 +00:00
cybermaggedon
b07d539802
- Fix topics extract (#126) 
- Add schemas to extraction
 2024-10-27 08:22:12 +00:00
cybermaggedon
1e137768ca
Prompt refactor (#125) 
* Prompt manager integrated and working with 6 tests
* Updated templates to for prompt-template update
 2024-10-26 22:17:43 +01:00
cybermaggedon
af0c8540e3
Feature/googleaistudio (#121) 
* Added jsonnet for GoogleAIStudio
* Port the template to 0.13, env secrets for k8s

---------

Co-authored-by: JackColquitt <daniel@kalntera.ai>
 2024-10-15 20:41:32 +01:00
cybermaggedon
25983d1557
Fix Bedrock (#119) 2024-10-15 19:21:43 +01:00
cybermaggedon
b76e9106dc
Use prompt templates by default, fix prompt template invocation (#117) 2024-10-15 17:14:20 +01:00
cybermaggedon
86288339cf
Feature/environment var creds (#116) 
- Change templates to interpolate environment variables in docker compose
- Change templates to invoke secrets for environment variable credentials in K8s configuration
- Update LLMs to pull in credentials from environment variables if not specified
 2024-10-15 00:34:52 +01:00
cybermaggedon
222dc9982c
Feature/azure openai templates (#104) 
* Azure OpenAI LLM templates
* Bump version, fix package versions
* Add azure-openai to template generation
 2024-10-04 15:47:46 +01:00
Cyber MacGeddon
8bd710972b Fix service 2024-10-02 22:25:37 +01:00
cybermaggedon
db9ed06b1c
Fix/broken kg extract topics (#97) 
* Add missing kg-extract-topics service

* Bump version
 2024-10-02 22:23:00 +01:00
cybermaggedon
6891053107
Maint/remove cruft (#94) 
* Remove cruft
 2024-10-02 00:09:11 +01:00
cybermaggedon
14d79ef9f1
Streamline startup (#79) 
* Separate Prom metrics, different processors as different jobs

* Create producers before consumers, may streamline startup.

* Bump version

* Add Pulsar init command, will replace pulsar-admin invocations.

* Integrate tg-init-pulsar with YAMLs

* Update YAMLs
 2024-09-30 12:19:22 +01:00
cybermaggedon
90127dfecb
Feature/rag metering (#75) 
* Add metering-rag for RAG token metrics

* Make metering executable

* Add metering-rag to Prom metrics

* Metricate the right queue

* Update templates to include metering-rag
 2024-09-29 20:15:03 +01:00
cybermaggedon
6d57b7a6f3
Add metering to templates (#74) 
* Add metering to trustgraph-core component

* Regenerate templates to run metering
 2024-09-29 19:48:35 +01:00
cybermaggedon
56b3b50e94
Fix/cohere temperature not string (#73) 
* Temperature should be a string in the templates

* Fix YAMLs for Cohere
 2024-09-29 19:37:49 +01:00
Cyber MacGeddon
39cf256f5c Merge branch 'master' into release/v0.10. Includes llamafile and 
prompt modifications.
 2024-09-28 11:26:52 +01:00
Jack Colquitt
9612a11581
Added basic Llamafile integration (#63) 
* Added basic Llamafile integration
* Added llamafile template support
* New templates following llamafile addition
---------
Co-authored-by: Cyber MacGeddon <cybermaggedon@gmail.com>
 2024-09-16 16:18:01 +01:00
cybermaggedon
6af86fa09f
Topic templates for extractor (#62) 
* Add topic prompt to templates
* Bump version
* Updated templates
 2024-09-15 23:40:37 +01:00
cybermaggedon
054ea6c76a
Fix Docker Compose relative directory problem (#60) 
- Fix Docker Compose relative directory problem
- Added CONFIGDIR environment variable for docker compose
- Correct Pulsar URL defaults for graph utils to be localhost again
 2024-09-09 22:03:10 +01:00
cybermaggedon
f661791bbf
K8s (#58) 
Added templates which produce K8s resources.  With the provided GCP wrapper, it works on GCP K8s cluster.  This isn't stable enough for other folks to use so will need more piloting before it can be documented and released.
 2024-09-07 18:59:38 +01:00
cybermaggedon
ddd8cc90e5
Fix templates (#55) 
* Fix template import problem

* Bump version
 2024-09-05 18:24:51 +01:00
cybermaggedon
f7429f2f9b
Fix remaining templates (#54) 2024-09-05 18:17:47 +01:00
cybermaggedon
65d7f6d261
Refactor templates (#52) 
* Switching from docker compose to abstract form - should be easier to k8s later
* Text loader util
* Recreate templates
 2024-09-05 16:40:47 +01:00
cybermaggedon
208c219962
Template rejig (#48) 
* document-rag / graph-rag refactor of templates

* Tweaking the docs and categories

* Clarify triple store vs RAG

* Tweak knowledge graph linkage

* Doc embedding for Qdrant

* Fix document RAG on Qdrant

* Fix templates

* Bump version

* Updated templates
 2024-09-03 00:09:15 +01:00
cybermaggedon
121f7bb776
Updates for config util repo (#47) 
* Fix pulsar manager template import
* Add pattern categories.  Update parameters with new schema.
* Add more parameters to config
 2024-09-01 21:29:29 +01:00
cybermaggedon
06a9e913dc
Enable model overrides, added a null option in config-loader (#45) 
Enable model overrides, added a null option in config-loader
 2024-08-30 21:33:03 +01:00
cybermaggedon
f7a30006ad
Make templating work more flexibly (#44) 
* Restructure directory
* Config loading
* Variable override points in JSONNET templates, separate pulsar-manager template
* Bump version
* Tidy chunking
* Simplified prompt overrides
* Update config loader
* Fix recursive chunker template
 2024-08-30 17:47:35 +01:00
cybermaggedon
32b087fbf6
Switch Milvus for Qdrant in YAMLs (#43) 
* Qdrant working

* - Fix missing prompt templates
- Bump version
- Add Qdrant to packages

* Switch Milvus for Qdrant in config files
 2024-08-27 23:37:24 +01:00
cybermaggedon
669aed0f8a
Added doc embedding support (#41) 
* document embedding writer & query
* Added test query for doc embeddings
* Bump version
* Added doc rag prompt
* Document RAG service
 2024-08-26 23:45:23 +01:00
cybermaggedon
d69de52b04
Increase resources (#35) 
* More memory for Cassandra
* More memory/CPU for embeddings
* Bump version, regenerate templates
 2024-08-25 20:38:19 +01:00
cybermaggedon
6edc3f0ee1
Prompt templates (#33) 
* Added prompt-template, allows definiton, relationships and kg query
to be specified in config / command-line.

* Bump version & add prompt-templates to YAMLs

* Apply to graph rag flow

* Break out different templates
 2024-08-23 23:34:16 +01:00
cybermaggedon
e7c498be92
Fix neo4j (#29) 
* - Fix Neo4j memory
- Fix neo4j query

* Version to 0.7.6
 2024-08-23 12:49:42 +01:00
cybermaggedon
b1b26a3f55
- Updated dashboard (#27) 
- Adjusted limits everything works
- Bump version
 2024-08-22 23:23:11 +01:00
cybermaggedon
305dda4463
Fix errors in previous update (#23) 
* Increase some limits
* Fix msg errors and update version
 2024-08-22 20:58:44 +01:00
cybermaggedon
a01a72ba00
Set resource limits (#22) 
* Added resource limits to resources.

* Boost version number, rebuild YAMLs
 2024-08-22 17:54:00 +01:00
cybermaggedon
ada7c405df
Update dependencies (#20) 2024-08-22 17:01:47 +01:00
cybermaggedon
d3cdb97528
Fix auto-templates (#17) 2024-08-22 00:19:55 +01:00
Cyber MacGeddon
7d00a53711 Some default values in yaml file 2024-08-21 17:17:24 +01:00
cybermaggedon
0e2db095e3
Add a docker-compose for just the stores (#13) 
* - Added docker-compose-storage.yaml, just the infrastructure bits
- Tidied storage invocation

* Util, sits on chunker output and reports histogram of chunk sizes
 2024-08-21 16:20:21 +01:00
cybermaggedon
a38f530c5f
Rate limit handling (#11) 
* Added a rate limit exception
* Reduce request/response timeouts because looks like there are major issues
* Add rate limit exception catch to all consumers
* Version to 0.6.3
 2024-08-19 22:15:32 +01:00
cybermaggedon
fa0b89b5d4
Simplify templates (#10) 
- Add component template files for all LLM types
- Top-level templates simplified to use just components
- Version to 0.6.2
 2024-08-14 20:56:57 +01:00
cybermaggedon
d3e213f194
Add Neo4j support (#9) 
- Add triples-write-neo4j and triples-query-neo4j to interact with neo4j
- Add docker-compose-openai-neo4j to demo Neo4j working
 2024-08-14 09:06:33 +01:00
Cyber MacGeddon
2f72fceaa2 Tidy templates directory 2024-08-13 19:22:57 +01:00