Commit graph

18 commits

Author SHA1 Message Date
cybermaggedon
1948edaa50
Streaming rag responses (#568)
* Tech spec for streaming RAG

* Support for streaming Graph/Doc RAG
2025-11-26 19:47:39 +00:00
cybermaggedon
45a14b5958
Graph rag optimisations (#527)
* Tech spec for GraphRAG optimisation

* Implement GraphRAG optimisation and update tests
2025-09-23 21:05:51 +01:00
cybermaggedon
48016d8fb2
Added XML, JSON, CSV detection (#519)
* Improved XML detect, added schema selection

* Add schema select + tests

* API additions

* More tests

* Fixed tests
2025-09-16 23:53:43 +01:00
cybermaggedon
3d783f4bd4
Structure data diagnosis service (#518)
* Import flow tech spec

* Structured diag service

* Plumbed into API gateway

* Type detector

* Diag service

* Added entry point
2025-09-16 21:43:23 +01:00
cybermaggedon
6ac8a7c2d9
Fix/document rag (#506)
* Fix missing document RAG user/collection params

* Added test
2025-09-09 15:30:11 +01:00
cybermaggedon
f22bf13aa6
Extend use of user + collection fields (#503)
* Collection+user fields in structured query

* User/collection in structured query & agent
2025-09-08 18:28:38 +01:00
cybermaggedon
a92050c411
Fix Prometheus incorrect metric name (#502)
* Fix Prometheus incorrect metric name

* Remove unnecessary changes
2025-09-06 18:37:01 +01:00
cybermaggedon
50c37407c5
Fix/sys integration issues (#494)
* Fix integration issues

* Fix query defaults

* Fix tests
2025-09-05 08:38:15 +01:00
cybermaggedon
a6d9f5e849
Structured query support (#492)
* Tweak the structured query schema

* Structure query service

* Gateway support for nlp-query and structured-query

* API support

* Added CLI

* Update tests

* More tests
2025-09-04 16:06:18 +01:00
cybermaggedon
8d4aa0069c
NLP query to GraphQL service (#491) 2025-09-04 13:39:47 +01:00
cybermaggedon
dd70aade11
Implement logging strategy (#444)
* Logging strategy and convert all prints() to logging invocations
2025-07-30 23:18:38 +01:00
cybermaggedon
5364b1fad5
Concurrency implemented in more services (#409) 2025-06-04 11:45:21 +01:00
cybermaggedon
a9197d11ee
Feature/configure flows (#345)
- Keeps processing in different flows separate so that data can go to different stores / collections etc.
- Potentially supports different processing flows
- Tidies the processing API with common base-classes for e.g. LLMs, and automatic configuration of 'clients' to use the right queue names in a flow
2025-04-22 20:21:38 +01:00
cybermaggedon
ef845d6c9b
Feature/rag parameters (#311)
* Change document-rag and graph-rag processing so that the user can
specify parameters.  Changes in Pulsar services, Pulsar message
schemas, gateway and command-line tools.  User-visible changes in
new parameters on command-line tools.

* Fix bugs, graph-rag working

* Get subgraph truncation in the right place

* Graph RAG and document RAG working and configurable

* Multi-hop path traversal GraphRAG

* Add safety valve for path_size set too high
2025-03-13 00:38:18 +00:00
cybermaggedon
617eb7efd5
Feature/pulsar api key support (#308)
* Add pulsar API token check

* Added missing api_key references

---------

Co-authored-by: Tyler O <4535788+toliver38@users.noreply.github.com>
2025-02-15 11:22:48 +00:00
cybermaggedon
f350abb415
Maint/asyncio (#305)
* Move to asyncio services, even though everything is largely sync
2025-02-11 23:24:46 +00:00
cybermaggedon
b0f4c58200
Feature / collections (#96)
* Update schema defs for source -> metadata
* Migrate to use metadata part of schema, also add metadata to triples & vecs
* Add user/collection metadata to query
* Use user/collection in RAG
* Write and query working on triples
2024-10-02 18:14:29 +01:00
cybermaggedon
9b91d5eee3
Feature/pkgsplit (#83)
* Starting to spawn base package
* More package hacking
* Bedrock and VertexAI
* Parquet split
* Updated templates
* Utils
2024-09-30 19:36:09 +01:00