Commit graph

60 commits

Author SHA1 Message Date
cybermaggedon
b08db761d7
Fix config inconsistency (#609)
* Plural/singular confusion in config key

* Flow class vs flow blueprint nomenclature change

* Update docs & CLI to reflect the above
2026-01-14 12:31:40 +00:00
cybermaggedon
807f6cc4e2
Fix non streaming RAG problems (#607)
* Fix non-streaming failure in RAG services

* Fix non-streaming failure in API

* Fix agent non-streaming messaging

* Agent messaging unit & contract tests
2026-01-12 18:45:52 +00:00
cybermaggedon
53cf5fd7f9
Fix test async warnings (#601)
* Fix tracemalloc async warnings

* Comment out debug, left in for use if needed
2026-01-06 22:09:34 +00:00
cybermaggedon
f79d0603f7
Update to add streaming tests (#600) 2026-01-06 21:48:05 +00:00
cybermaggedon
ae13190093
Address legacy issues in storage management (#595)
* Removed legacy storage management cruft.  Tidied tech specs.

* Fix deletion of last collection

* Storage processor ignores data on the queue which is for a deleted collection

* Updated tests
2026-01-05 13:45:14 +00:00
cybermaggedon
5304f96fe6
Fix tests (#593)
* Fix unit/integration/contract tests which were broken by messaging fabric work
2025-12-19 08:53:21 +00:00
cybermaggedon
ba95fa226b
Gateway queue overrides (#584) 2025-12-06 11:01:20 +00:00
cybermaggedon
7d07f802a8
Basic multitenant support (#583)
* Tech spec

* Address multi-tenant queue option problems in CLI

* Modified collection service to use config

* Changed storage management to use the config service definition
2025-12-05 21:45:30 +00:00
cybermaggedon
789d9713a0
Fix API tests (#581)
* Fix RAG streaming CLIs

* Fixed, all tests pass
2025-12-04 21:11:56 +00:00
cybermaggedon
01aeede78b
Python API implements streaming interfaces (#577)
* Tech spec

* Python CLI utilities updated to use the API including streaming features

* Added type safety to Python API

* Completed missing auth token support in CLI
2025-12-04 17:38:57 +00:00
cybermaggedon
310a2deb06
Feature/streaming llm phase 1 (#566)
* Tidy up duplicate tech specs in doc directory

* Streaming LLM text-completion service tech spec.

* text-completion and prompt interfaces

* streaming change applied to all LLMs, so far tested with VertexAI

* Skip Pinecone unit tests, upstream module issue is affecting things, tests are passing again

* Added agent streaming, not working and has broken tests
2025-11-26 09:59:10 +00:00
cybermaggedon
3580e7a7ae
Remove some 'unnecessary' parameters from OpenAI invocation (#561)
* Remove some 'unnecessary' parameters from OpenAI invocation.  The OpenAI
API is getting complicated with the API and SDK changing on OpenAI's end,
but this not getting mapped through to other services which are 'compatible'
with OpenAI.

* Update OpenAI test for this change

* Trying running tests with Python 3.13
2025-11-20 17:56:31 +00:00
cybermaggedon
6c85038c75
Ontology extraction tests (#560) 2025-11-13 20:02:12 +00:00
cybermaggedon
4c3db4dbbe
MCP auth for the simple case (#557)
* MCP auth token header

* Mention limitations

* Fix AgentStep schema error by converting argument values to strings.

* Added tests for MCP auth and agent step parsing
2025-11-11 12:28:53 +00:00
cybermaggedon
d9d4c91363
Dynamic embeddings model (#556)
* Dynamic embeddings model selection

* Added tests

* HF embeddings are skipped, tests don't run with that package currently tests
2025-11-10 20:38:01 +00:00
cybermaggedon
6129bb68c1
Fix hard coded vector size (#555)
* Fixed hard-coded embeddings store size

* Vector store lazy-creates collections, different collections for
  different dimension lengths.

* Added tech spec for vector store lifecycle

* Fixed some tests for the new spec
2025-11-10 16:56:51 +00:00
cybermaggedon
51107008fd
master -> 1.5 (README updates) (#552) 2025-10-11 11:46:03 +01:00
cybermaggedon
52b133fc86
Collection delete pt. 3 (#542)
* Fixing collection deletion

* Fixing collection management param error

* Always test for collections

* Add Cassandra collection table

* Updated tech spec for explicit creation/deletion

* Remove implicit collection creation

* Fix up collection tracking in all processors
2025-09-30 16:02:33 +01:00
cybermaggedon
8929a680a1
Chunking dynamic params (#536)
* Chunking params are dynamic

* Update tests
2025-09-26 10:53:32 +01:00
cybermaggedon
43cfcb18a0
More LLM param test coverage (#535)
* More LLM tests

* Fixing tests
2025-09-26 01:00:30 +01:00
cybermaggedon
b0a3716b0e
Tests are failing (#534)
* Fix tests, update to new model parameter usage
2025-09-25 21:32:19 +01:00
cybermaggedon
45a14b5958
Graph rag optimisations (#527)
* Tech spec for GraphRAG optimisation

* Implement GraphRAG optimisation and update tests
2025-09-23 21:05:51 +01:00
cybermaggedon
fcd15d1833
Collection management part 2 (#522)
* Plumb collection manager into librarian

* Test end-to-end
2025-09-19 16:08:47 +01:00
cybermaggedon
d378db9370
Cassandra performance enhancement (#521)
* Tech spec

* Tech spec complete

* Cassandra multi-table for performance
2025-09-18 19:52:05 +01:00
cybermaggedon
13ff7d765d
Collection management (#520)
* Tech spec

* Refactored Cassanda knowledge graph for single table

* Collection management, librarian services to manage metadata and collection deletion
2025-09-18 15:57:52 +01:00
cybermaggedon
48016d8fb2
Added XML, JSON, CSV detection (#519)
* Improved XML detect, added schema selection

* Add schema select + tests

* API additions

* More tests

* Fixed tests
2025-09-16 23:53:43 +01:00
cybermaggedon
7f57bc6a0a
Feature/memgraph user collection isolation (#510)
* User/collection processing in memgraph

* Update tests
2025-09-10 22:11:35 +01:00
cybermaggedon
c694b12e9c
Feature/neo4j user collection isolation (#509)
* Tech spec

* User/collection separation

* Update tests
2025-09-10 22:11:21 +01:00
cybermaggedon
314ce76b81
Feature/fix milvus (#507)
- Remove object embeddings, were currently broken and not used
- Fixed Milvus collection names

* Updating tests

* Remove unused entrypoint
2025-09-09 21:44:55 +01:00
cybermaggedon
6ac8a7c2d9
Fix/document rag (#506)
* Fix missing document RAG user/collection params

* Added test
2025-09-09 15:30:11 +01:00
cybermaggedon
266454e75f
Feature/group state cli (#505) 2025-09-08 21:44:57 +01:00
cybermaggedon
f22bf13aa6
Extend use of user + collection fields (#503)
* Collection+user fields in structured query

* User/collection in structured query & agent
2025-09-08 18:28:38 +01:00
cybermaggedon
5537fac731
Structured data, minor features (#500)
- Sorted out confusing --auto mode with tg-load-structured-data
- Fixed tests & added CLI tests
2025-09-05 17:25:12 +01:00
cybermaggedon
0b7620bc04
Object batching (#499)
* Object batching

* Update tests
2025-09-05 15:59:06 +01:00
cybermaggedon
257a7951a7
Object import (#497)
* Object import dispatcher

* Add object import gateway test
2025-09-05 14:06:01 +01:00
cybermaggedon
f1d08969ec
Add more structured query support (#496)
* Updated tool cmd line

* Added a CLI test
2025-09-05 12:36:22 +01:00
cybermaggedon
50c37407c5
Fix/sys integration issues (#494)
* Fix integration issues

* Fix query defaults

* Fix tests
2025-09-05 08:38:15 +01:00
cybermaggedon
a6d9f5e849
Structured query support (#492)
* Tweak the structured query schema

* Structure query service

* Gateway support for nlp-query and structured-query

* API support

* Added CLI

* Update tests

* More tests
2025-09-04 16:06:18 +01:00
cybermaggedon
85e669c763
Fixing more Cassandra consistency issues (#488)
* Fixing more Cassandra work

* Fix tests
2025-09-04 00:58:11 +01:00
cybermaggedon
ccaec88a72
Feature/consolidate cassandra config (#483)
* Cassandra consolidation of parameters

* New Cassandra configuration helper

* Implemented Cassanda config refactor

* New tests
2025-09-03 23:41:22 +01:00
cybermaggedon
e74eb5d1ff
Feature/tool group (#484)
* Tech spec for tool group

* Partial tool group implementation

* Tool group tests
2025-09-03 23:39:49 +01:00
cybermaggedon
672e358b2f
Feature/graphql table query (#486)
* Tech spec

* Object query service for Cassandra

* Gateway support for objects-query

* GraphQL query utility

* Filters, ordering
2025-09-03 23:39:11 +01:00
cybermaggedon
38826c7de1
trustgraph-base .chunks / .documents confusion in the API (#481)
* trustgraph-base .chunks / .documents confusion in the API

* Added tests, fixed test failures in code

* Fix file dup error

* Fix contract error
2025-09-02 17:58:53 +01:00
cybermaggedon
96c2b73457
Fix import export graceful shutdown (#476)
* Tech spec for graceful shutdown

* Graceful shutdown of importers/exporters

* Update socket to include graceful shutdown orchestration

* Adding tests for conditions tracked in this PR
2025-08-28 13:39:28 +01:00
cybermaggedon
3e5d6ed3e4
Use collection field from request when loading a knowledge core (#472)
* Use collection field from request when loading a knowledge core

* Test core collection
2025-08-27 09:08:06 +01:00
cybermaggedon
28190fea8a
More config cli (#466)
* Extra config CLI tech spec

* Describe packaging

* Added CLI commands

* Add tests
2025-08-22 13:36:10 +01:00
cybermaggedon
5e71d0cadb
Object extraction schema validation (#465)
* Object schema validation in kg-extract-objects, prevents invalid data appearing in Pulsar messages

* Added tests for the above
2025-08-22 12:30:05 +01:00
cybermaggedon
865bb47349
Feature/mcp tool arguments (#462)
* Tech spec for MCP arguments

* Agent support for MCP tool arguments

* Extra tests for MCP arguments

* Fix tg-set-tool help and docs
2025-08-21 14:46:10 +01:00
cybermaggedon
79e16e65f6
Fix/agent tool resilience (#461)
* Fix incorrect tool initialisation in agent service

* Make Action: parsing more resient.  If there are quotation marks, strip them off.

* Added test case for this change
2025-08-21 13:00:33 +01:00
cybermaggedon
54948e567f
Increase agent ReACT tool coverage (#460)
* Extra multi-step tool invocation

* Multi-step reasoning test
2025-08-21 00:36:45 +01:00