Commit graph

73 commits

Author SHA1 Message Date
cybermaggedon
8e227e256e
Fix bookie & pulsar service naming in K8s (#276) 2025-01-20 23:54:33 +00:00
cybermaggedon
ac7bf6bfd5
Fix Minikube launch issues (#275)
- Fix Pulsar service naming - broker -> pulsar
- Fix Google AI Studio secret name for RAG to be consistent with other
  text-completion.
2025-01-20 19:17:43 +00:00
cybermaggedon
8fc22155dd
Agent tweaks (#272)
- Removed unused LLM client configuration from agent-manager-react
- Change agent-manager-react template to use prompt-rag instead of
  prompt
- Changed TextCompletion tool to use 'question' instead of 'computation'
  for its parameter.
2025-01-20 14:05:51 +00:00
cybermaggedon
acdd3efe51
Fix argument usage not overriding defaults in Google AI Studio & VertexAI templates (#268) 2025-01-15 23:59:50 +00:00
cybermaggedon
1280af3eff
Fix/kg core save (#267)
* Add a 'localhost' Pulsar endpoint for docker env

* - Fix broken socket endpoint streamers
- Add unused listener endpoints for publisher/subscriber
- Fix graph embedding serialisation

* Fix GE load

* Remove Gossip settling delay, this is single-node Cassandra.
2025-01-13 14:42:33 +00:00
cybermaggedon
cec9e29222
Dual model templates (#263)
* Dual-mode templates

* Fixed generate-all so that works

* Fix ability to specify Claude model
2025-01-11 11:40:42 +00:00
JackColquitt
8f769b1e73 Fixed creds for GoogleAIStudio in K8s 2025-01-07 12:24:57 -08:00
JackColquitt
ffacf84c79 Fix for Claude key envvar 2025-01-07 08:54:35 -08:00
JackColquitt
970e824589 Fixed Zookeeper startup delay 2025-01-06 12:06:51 -08:00
JackColquitt
bb0d48c1b2 Hot fix for pulsar user issue 2025-01-06 10:03:22 -08:00
cybermaggedon
6aa212061d
Fix/document embeddings (#247)
* Update schema for doc embeddings

* Rename embeddings-vectorize to graph-embeddings

* Added document-embeddings processor (broken, needs fixing)

* Added scripts

* Fixed DE queue schema

* Add missing DE process

* Fix doc RAG processing, put graph-rag and doc-rag in appropriate component files.
2025-01-04 21:51:28 +00:00
cybermaggedon
fed7e4ca08
Pulsar resilience refactor (#238)
* Add userID control over compose containers
* Split Pulsar out, non-standalone
* Added bookie & zookeeper to Prometheus stats
2025-01-02 19:50:23 +00:00
cybermaggedon
f145d5c324
Add workbench-ui (#216) 2024-12-20 00:45:34 +00:00
cybermaggedon
a4afff59a0
wip integrate falkordb (#211) (#213)
Co-authored-by: Avi Avni <avi.avni@gmail.com>
2024-12-19 16:17:07 +00:00
cybermaggedon
bffaf62490
Feature/memgraph optim (#193)
* Separate memgraph query/write modules to optimise for memgraph
* Used 1GB memory for Memgraph
* Deployed specialised memgraph query/write processors, created memgraph indexes
* One triple is loaded as a single transaction
* Fixed index creation
2024-12-06 00:12:49 +00:00
cybermaggedon
f24eed3023
Fix/pinecone de (#187)
* Fix Goog AI studio settings
* Fix pinecone startup params
2024-12-03 09:51:33 +00:00
cybermaggedon
1b9c6be4fc
Feature/gateway auth (#186)
* Added auth module, just a simple token at this stage
* Pass auth token GATEWAY_SECRET through
* Auth token not mandatory, can be provided in env var
2024-12-02 19:57:21 +00:00
cybermaggedon
6d200c79c5
Feature/wikipedia ddg (#185)
API-side support for Wikipedia, DBpedia and internet search functions  This incorporates a refactor of the API code to break it up, separate classes for endpoints to reduce duplication
2024-12-02 17:41:30 +00:00
cybermaggedon
9c97ca32f6
Feature/memgraph (#182)
* Add database override to bolt output, default is neo4j

* Add memgraph templates
2024-11-28 19:21:28 +00:00
cybermaggedon
319f9ac04a
Feature/pinecone integration (#170)
* Added Pinecone for GE write & query

* Add templates

* Doc embedding support
2024-11-22 23:48:21 +00:00
cybermaggedon
ba6d6c13af
Fix API gateway integration, added to templates (#165) 2024-11-20 20:56:23 +00:00
cybermaggedon
efd13ae5f1
Added agent support to templates (#150) 2024-11-12 00:22:18 +00:00
cybermaggedon
7aab42a054
Fix prompt overrides (#147) 2024-11-10 18:24:19 +00:00
cybermaggedon
c308180b54
Fix prompt override - overriding a prompt with JSON output broke schema and (#142)
extraction settings.
2024-11-06 18:44:25 +00:00
cybermaggedon
2660b28ab9
Set default model (#140) 2024-11-05 23:58:36 +00:00
cybermaggedon
45091e01ec
Fix Google AI Studio model (#139) 2024-11-05 23:49:24 +00:00
cybermaggedon
ee9373d55e
Make temperature round to 3 dec places (#138) 2024-11-05 23:04:21 +00:00
cybermaggedon
53c958aaff
Fix: prompt template overrides (#136)
* Added single-target command-line config generator.  Mainly using for
testing ATM.

* Slightly tweak the config decode so that components can over-ride the
'with' method which injects parameters.

* Deliberately break the prompt-generic template.  Could do better, this
is temporary.

* Add 'prompt-overrides' component, injects new prompts.

* Removed prompt generic reference, not used

* prompt-generic is no longer supported
2024-11-05 21:17:34 +00:00
cybermaggedon
8a2126bba5
Fix neo4j schema - incompatible with extractors (#135)
- Fixed schema invocation on Neo4j writer
- Fixed confusing typo name in cassandra template
- Unnecessary import on Cassandra removed
2024-11-02 10:51:26 +00:00
cybermaggedon
7c0c471e55
Feature/prompt terms (#128)
* Add prompt terms
* Remove dupe template code
2024-10-27 15:04:53 +00:00
cybermaggedon
2942c4dcf0
Maint/neater prompt jsonnet (#127)
* Changed the JSONNET for prompt-template so that prompt templates are
simpler.  This tidies default-prompts.jsonnet so it's easier to see
where the prompts are.

* Updated prompt override templates to match new structure
2024-10-27 09:06:09 +00:00
cybermaggedon
b07d539802
- Fix topics extract (#126)
- Add schemas to extraction
2024-10-27 08:22:12 +00:00
cybermaggedon
1e137768ca
Prompt refactor (#125)
* Prompt manager integrated and working with 6 tests
* Updated templates to for prompt-template update
2024-10-26 22:17:43 +01:00
cybermaggedon
af0c8540e3
Feature/googleaistudio (#121)
* Added jsonnet for GoogleAIStudio
* Port the template to 0.13, env secrets for k8s

---------

Co-authored-by: JackColquitt <daniel@kalntera.ai>
2024-10-15 20:41:32 +01:00
cybermaggedon
25983d1557
Fix Bedrock (#119) 2024-10-15 19:21:43 +01:00
cybermaggedon
b76e9106dc
Use prompt templates by default, fix prompt template invocation (#117) 2024-10-15 17:14:20 +01:00
cybermaggedon
86288339cf
Feature/environment var creds (#116)
- Change templates to interpolate environment variables in docker compose
- Change templates to invoke secrets for environment variable credentials in K8s configuration
- Update LLMs to pull in credentials from environment variables if not specified
2024-10-15 00:34:52 +01:00
cybermaggedon
222dc9982c
Feature/azure openai templates (#104)
* Azure OpenAI LLM templates
* Bump version, fix package versions
* Add azure-openai to template generation
2024-10-04 15:47:46 +01:00
Cyber MacGeddon
8bd710972b Fix service 2024-10-02 22:25:37 +01:00
cybermaggedon
db9ed06b1c
Fix/broken kg extract topics (#97)
* Add missing kg-extract-topics service

* Bump version
2024-10-02 22:23:00 +01:00
cybermaggedon
6891053107
Maint/remove cruft (#94)
* Remove cruft
2024-10-02 00:09:11 +01:00
cybermaggedon
14d79ef9f1
Streamline startup (#79)
* Separate Prom metrics, different processors as different jobs

* Create producers before consumers, may streamline startup.

* Bump version

* Add Pulsar init command, will replace pulsar-admin invocations.

* Integrate tg-init-pulsar with YAMLs

* Update YAMLs
2024-09-30 12:19:22 +01:00
cybermaggedon
90127dfecb
Feature/rag metering (#75)
* Add metering-rag for RAG token metrics

* Make metering executable

* Add metering-rag to Prom metrics

* Metricate the right queue

* Update templates to include metering-rag
2024-09-29 20:15:03 +01:00
cybermaggedon
6d57b7a6f3
Add metering to templates (#74)
* Add metering to trustgraph-core component

* Regenerate templates to run metering
2024-09-29 19:48:35 +01:00
cybermaggedon
56b3b50e94
Fix/cohere temperature not string (#73)
* Temperature should be a string in the templates

* Fix YAMLs for Cohere
2024-09-29 19:37:49 +01:00
Cyber MacGeddon
39cf256f5c Merge branch 'master' into release/v0.10. Includes llamafile and
prompt modifications.
2024-09-28 11:26:52 +01:00
Jack Colquitt
9612a11581
Added basic Llamafile integration (#63)
* Added basic Llamafile integration
* Added llamafile template support
* New templates following llamafile addition
---------
Co-authored-by: Cyber MacGeddon <cybermaggedon@gmail.com>
2024-09-16 16:18:01 +01:00
cybermaggedon
6af86fa09f
Topic templates for extractor (#62)
* Add topic prompt to templates
* Bump version
* Updated templates
2024-09-15 23:40:37 +01:00
cybermaggedon
054ea6c76a
Fix Docker Compose relative directory problem (#60)
- Fix Docker Compose relative directory problem
- Added CONFIGDIR environment variable for docker compose
- Correct Pulsar URL defaults for graph utils to be localhost again
2024-09-09 22:03:10 +01:00
cybermaggedon
f661791bbf
K8s (#58)
Added templates which produce K8s resources.  With the provided GCP wrapper, it works on GCP K8s cluster.  This isn't stable enough for other folks to use so will need more piloting before it can be documented and released.
2024-09-07 18:59:38 +01:00