cybermaggedon
ae13190093
Address legacy issues in storage management ( #595 )
...
* Removed legacy storage management cruft. Tidied tech specs.
* Fix deletion of last collection
* Storage processor ignores data on the queue which is for a deleted collection
* Updated tests
2026-01-05 13:45:14 +00:00
cybermaggedon
25563bae3c
Change MinIO integration options in librarian to be more generic - to support a Garage integration ( #594 )
...
* Tweak object store parameters to be more generic for other S3-type store integration
* Update librarian to have region & SSL params
* Update MinIO migration tech spec
2025-12-27 18:01:51 +00:00
cybermaggedon
5304f96fe6
Fix tests ( #593 )
...
* Fix unit/integration/contract tests which were broken by messaging fabric work
2025-12-19 08:53:21 +00:00
cybermaggedon
34eb083836
Messaging fabric plugins ( #592 )
...
* Plugin architecture for messaging fabric
* Schemas use a technology neutral expression
* Schemas strictness has uncovered some incorrect schema use which is fixed
2025-12-17 21:40:43 +00:00
Cyber MacGeddon
1865b3f3c8
Start 1.8 release branch
2025-12-17 21:32:13 +00:00
cybermaggedon
2b5ba68f00
Add model to meter metrics ( #589 )
2025-12-10 11:34:48 +00:00
cybermaggedon
727b6bc9d6
Add service ID to log entry instead of module name ( #588 )
2025-12-10 11:07:43 +00:00
cybermaggedon
aebdf9444b
Fix incorrect Cassandra config invocation ( #587 )
2025-12-10 10:55:14 +00:00
cybermaggedon
f12fcc2652
Loki logging ( #586 )
...
* Consolidate logging into a single module
* Added Loki logging
* Update tech spec
* Add processor label
* Fix recursive log entries, logging Loki"s internals
2025-12-09 23:24:41 +00:00
cybermaggedon
39f6a8b940
Fix/queue configurations ( #585 )
...
* Fix config-svc startup dupe CLI args
* Fix missing params on collection service
* Fix collection management handling
2025-12-06 14:54:47 +00:00
cybermaggedon
ba95fa226b
Gateway queue overrides ( #584 )
2025-12-06 11:01:20 +00:00
cybermaggedon
7d07f802a8
Basic multitenant support ( #583 )
...
* Tech spec
* Address multi-tenant queue option problems in CLI
* Modified collection service to use config
* Changed storage management to use the config service definition
2025-12-05 21:45:30 +00:00
cybermaggedon
789d9713a0
Fix API tests ( #581 )
...
* Fix RAG streaming CLIs
* Fixed, all tests pass
2025-12-04 21:11:56 +00:00
cybermaggedon
664bce6182
Fix Python streaming SDK issues ( #580 )
...
* Fix verify CLI issues
* Fixing content mechanisms in API
* Fixing error handling
* Fixing invoke_prompt, invoke_llm, invoke_agent
2025-12-04 20:42:25 +00:00
cybermaggedon
52ca74bbbc
System startup tracker ( #579 )
2025-12-04 18:01:47 +00:00
cybermaggedon
01aeede78b
Python API implements streaming interfaces ( #577 )
...
* Tech spec
* Python CLI utilities updated to use the API including streaming features
* Added type safety to Python API
* Completed missing auth token support in CLI
2025-12-04 17:38:57 +00:00
cybermaggedon
b957004db9
Feature/improve ontology extract ( #576 )
...
* Tech spec to change ontology extraction
* Ontology extract refactoring
2025-12-03 13:36:10 +00:00
cybermaggedon
517434c075
Merge master -> 1.7 for README changes ( #574 )
...
* All recent changes on master README
---------
Co-authored-by: Jack Colquitt <126733989+JackColquitt@users.noreply.github.com>
2025-12-03 09:53:35 +00:00
Cyber MacGeddon
98aaa4f67e
Configure for 1.7 release branch
2025-12-03 09:46:55 +00:00
cybermaggedon
c808d26b0b
Fix AWS bedrock issues with newer model invocation ( #572 )
...
- Fixed models so that global.* models work
- Fixed Claude 4.5 & 4.7 invocation by removing top_p top_k params
2025-12-01 21:50:05 +00:00
cybermaggedon
72cb1c98e0
Fix tests ( #571 )
2025-11-28 16:37:01 +00:00
cybermaggedon
e24de6081f
Fix streaming agent interactions ( #570 )
...
* Fix observer, thought streaming
* Fix end of message indicators
* Remove double-delivery of answer
2025-11-28 16:25:57 +00:00
cybermaggedon
1948edaa50
Streaming rag responses ( #568 )
...
* Tech spec for streaming RAG
* Support for streaming Graph/Doc RAG
2025-11-26 19:47:39 +00:00
cybermaggedon
b1cc724f7d
Streaming LLM part 2 ( #567 )
...
* Updates for agent API with streaming support
* Added tg-dump-queues tool to dump Pulsar queues to a log
* Updated tg-invoke-agent, incremental output
* Queue dumper CLI - might be useful for debug
* Updating for tests
2025-11-26 15:16:17 +00:00
cybermaggedon
310a2deb06
Feature/streaming llm phase 1 ( #566 )
...
* Tidy up duplicate tech specs in doc directory
* Streaming LLM text-completion service tech spec.
* text-completion and prompt interfaces
* streaming change applied to all LLMs, so far tested with VertexAI
* Skip Pinecone unit tests, upstream module issue is affecting things, tests are passing again
* Added agent streaming, not working and has broken tests
2025-11-26 09:59:10 +00:00
cybermaggedon
943a9d83b0
Fix Minio incompatible library change ( #565 )
2025-11-25 22:45:48 +00:00
cybermaggedon
97d8b84d7f
Open 1.6 release branch ( #564 )
2025-11-24 10:05:29 +00:00
cybermaggedon
7501db01f1
Reconcile master with 1.6 ( #563 )
...
- Reconcile all master changes (documentation) to the 1.6 release branch
2025-11-24 10:02:30 +00:00
cybermaggedon
3580e7a7ae
Remove some 'unnecessary' parameters from OpenAI invocation ( #561 )
...
* Remove some 'unnecessary' parameters from OpenAI invocation. The OpenAI
API is getting complicated with the API and SDK changing on OpenAI's end,
but this not getting mapped through to other services which are 'compatible'
with OpenAI.
* Update OpenAI test for this change
* Trying running tests with Python 3.13
2025-11-20 17:56:31 +00:00
cybermaggedon
6c85038c75
Ontology extraction tests ( #560 )
2025-11-13 20:02:12 +00:00
cybermaggedon
2baf21c5e1
Collection deletion batch error fixed ( #559 )
...
Collection deletion caused a batch error, this reduces batch size on deletion
2025-11-13 16:29:40 +00:00
cybermaggedon
db4e842df3
Update tech spec ( #558 )
2025-11-13 16:29:20 +00:00
cybermaggedon
c69f5207a4
OntoRAG: Ontology-Based Knowledge Extraction and Query Technical Specification ( #523 )
...
* Onto-rag tech spec
* New processor kg-extract-ontology, use 'ontology' objects from config to guide triple extraction
* Also entity contexts
* Integrate with ontology extractor from workbench
This is first phase, the extraction is tested and working, also GraphRAG with the extracted knowledge works
2025-11-12 20:38:08 +00:00
cybermaggedon
4c3db4dbbe
MCP auth for the simple case ( #557 )
...
* MCP auth token header
* Mention limitations
* Fix AgentStep schema error by converting argument values to strings.
* Added tests for MCP auth and agent step parsing
2025-11-11 12:28:53 +00:00
cybermaggedon
d9d4c91363
Dynamic embeddings model ( #556 )
...
* Dynamic embeddings model selection
* Added tests
* HF embeddings are skipped, tests don't run with that package currently tests
2025-11-10 20:38:01 +00:00
cybermaggedon
6129bb68c1
Fix hard coded vector size ( #555 )
...
* Fixed hard-coded embeddings store size
* Vector store lazy-creates collections, different collections for
different dimension lengths.
* Added tech spec for vector store lifecycle
* Fixed some tests for the new spec
2025-11-10 16:56:51 +00:00
cybermaggedon
05b9063fea
Feature/python3.13 ( #553 )
...
* Python to 3.13
* cassandra-driver -> scylla-driver
(cassandra-driver doesn't work with Python 3.13)
2025-10-11 12:19:26 +01:00
cybermaggedon
51107008fd
master -> 1.5 (README updates) ( #552 )
2025-10-11 11:46:03 +01:00
cybermaggedon
ad35656811
Prepare 1.5 release branch ( #550 )
2025-10-11 11:44:00 +01:00
cybermaggedon
77fdec2c2d
Fix chunk params not converted ( #549 )
2025-10-07 00:04:34 +01:00
cybermaggedon
09930bdb86
Fix deletion error in Cassandra object store ( #546 )
...
* Iteration algorithm modified to look for tables using schema definitions rather than iterate over all tables
2025-10-06 17:45:02 +01:00
cybermaggedon
35ef9bd71c
Apply fix, incorrect statement invocation ( #545 )
2025-10-06 14:24:46 +01:00
cybermaggedon
3e23d3c3ed
Fix collection management sync prob ( #544 )
...
* Address creation/deletion sync problems
* Fix object writer management
* Get Milvus to use ANN
2025-09-30 23:04:28 +01:00
cybermaggedon
b7bf7388aa
Reduce log spam ( #543 )
2025-09-30 16:04:07 +01:00
cybermaggedon
52b133fc86
Collection delete pt. 3 ( #542 )
...
* Fixing collection deletion
* Fixing collection management param error
* Always test for collections
* Add Cassandra collection table
* Updated tech spec for explicit creation/deletion
* Remove implicit collection creation
* Fix up collection tracking in all processors
2025-09-30 16:02:33 +01:00
cybermaggedon
dc79b10552
Feaature/flow default params ( #541 )
...
* Flow creation uses parameter defaults in API and CLI
* Submit strings for flow parameters
2025-09-30 14:06:08 +01:00
cybermaggedon
d1456e547c
Fix label issue in metrics ( #540 )
2025-09-26 14:13:22 +01:00
cybermaggedon
10806caac2
Fix label names ( #539 )
2025-09-26 12:34:40 +01:00
cybermaggedon
209d0de0b8
Add missing file ( #538 )
2025-09-26 11:28:15 +01:00
cybermaggedon
8354ea1276
Update flow parameter tech spec for advanced params ( #537 )
...
* Add advanced mode to tech spec, fix enum description in tech spec
* Updated tech-spec for controlled-by relationship between parameters
* Update tg-show-flows CLI
* Update tg-show-flows, tg-show-flow-classes, tg-start-flow CLI
* Add tg-show-parameter-types
2025-09-26 10:55:10 +01:00
