* Working mux socket
* Change API to incorporate flow
* Add Flow ID to all relevant CLIs, not completely implemented
* Change tg-processor-state to use API gateway
* Updated all CLIs
* New tg-show-flow-state command
* tg-show-flow-state shows classes too
- Keeps processing in different flows separate so that data can go to different stores / collections etc.
- Potentially supports different processing flows
- Tidies the processing API with common base-classes for e.g. LLMs, and automatic configuration of 'clients' to use the right queue names in a flow
- prompt-template takes config from the config-svc, dynamically reloads
as new config appears.
- agent-react takes config from config-svc, dynamically reloads
- Fixed lack of data in config queue, needed to take the Earliest, not the
Latest values.
- Changed text-completion and knowledge-query tool to both use 'query'
as the argument.
- Prompt and agent no longer have command line args to supply
configuration.
* - Fixed error reporting in config
- Updated tg-init-pulsar to be able to load initial config to config-svc
- Tweaked API naming and added more config calls
* Tools to dump out prompts and agent tools
Configuration service provides an API to change configuration. Complete configuration is pushed down a config queue so that users have a complete copy of config object.
* Change document-rag and graph-rag processing so that the user can
specify parameters. Changes in Pulsar services, Pulsar message
schemas, gateway and command-line tools. User-visible changes in
new parameters on command-line tools.
* Fix bugs, graph-rag working
* Get subgraph truncation in the right place
* Graph RAG and document RAG working and configurable
* Multi-hop path traversal GraphRAG
* Add safety valve for path_size set too high
* Bring QDrant up-to-date
* Tables for data from queue outputs
- Pass single Pulsar client to everything in gateway & librarian
- Pulsar listener-name support in gateway
- PDF and text load working in librarian
* Complete Cassandra schema
* Add librarian support to templates
* - Refactored retry for rate limits into the base class
- ConsumerProducer is derived from Consumer to simplify code
- Added rate_limit_count metrics for rate limit events
* Add rate limit events to VertexAI and Google AI Studio
* Added Grafana rate limit dashboard
* Add rate limit handling to all LLMs
* Make schema changes
* Core entity context flow in place
* extract-def outputs entity contexts
* Refactored qdrant write
* Refactoring of all vector stores in place
* Fix invalid variable name invocation
* Fix error responses in websockets
* Increase websocket limits to 50MB max message. Turn on Pulsar chunking by default.
API-side support for Wikipedia, DBpedia and internet search functions This incorporates a refactor of the API code to break it up, separate classes for endpoints to reduce duplication
* Acknowledge messaages from Pulsar, doh!
* Change API to deliver a boolean e if value is an entity
* Change loaders to use new API
* Changes, entity-aware API is complete
* Add schema load util
* Added a sample schema turtle file will be useful for future testing and
tutorials.
* Fixed graph label metadata confusion, was created incorrect subjectOf
edges.
* Rework metadata structure in processing messages to be a subgraph
* Add subgraph creation for tg-load-pdf and tg-load-text based on command-line passing of doc attributes
* Document metadata is added to knowledge graph with subjectOf linkage to extracted entities
* Update schema defs for source -> metadata
* Migrate to use metadata part of schema, also add metadata to triples & vecs
* Add user/collection metadata to query
* Use user/collection in RAG
* Write and query working on triples
