Commit graph

54 commits

Author SHA1 Message Date
cybermaggedon
8828c00cf6
Fix sample doc loader (#400) 2025-05-28 11:30:53 +01:00
cybermaggedon
ef34d951fe
Renamed default flow from 0000 to default (#395) 2025-05-24 12:27:56 +01:00
cybermaggedon
9e9b21068c
ID field is used (#375) 2025-05-08 15:28:27 +01:00
cybermaggedon
31b7ade44d
Feature/knowledge load (#372)
* Switch off retry in Cassandra until we can differentiate retryable errors

* Fix config getvalues

* Loading knowledge cores works
2025-05-08 00:41:45 +01:00
cybermaggedon
fdd9a9a9ae
Fix kg-core downloading (#371) 2025-05-07 14:38:48 +01:00
cybermaggedon
f7123ac57f
tg-put-kg-core command (#369) 2025-05-07 11:13:21 +01:00
cybermaggedon
8080b54328
Knowledge core CLI (#368) 2025-05-07 00:20:59 +01:00
cybermaggedon
d0da122bed
Fix/llms (#366)
* Fix LMStudio, cache documents with tg-load-sample-documents

* Fix Mistral
2025-05-06 16:17:16 +01:00
cybermaggedon
54e475fa3a
Sample docs loader (#365) 2025-05-06 13:43:17 +01:00
cybermaggedon
9e4eb634a4
Fix/chunking not enabled (#364)
* Enable chunking by default in producer

* Fix some issues including uploading large docs
2025-05-06 00:28:20 +01:00
cybermaggedon
844547ab5f
Feature/library cli (#363)
* Major Python client API rework, break down API & colossal class

* Complete rest of library API

* Library CLI support
2025-05-05 11:09:18 +01:00
cybermaggedon
6bf485788a
Fix broken CLI (#360) 2025-05-03 11:07:17 +01:00
cybermaggedon
b3c9dda57f
Fix broken build (#359) 2025-05-03 10:45:42 +01:00
cybermaggedon
3b8b9ea866
Feature/flow api 3 (#358)
* Working mux socket

* Change API to incorporate flow

* Add Flow ID to all relevant CLIs, not completely implemented

* Change tg-processor-state to use API gateway

* Updated all CLIs

* New tg-show-flow-state command

* tg-show-flow-state shows classes too
2025-05-03 10:39:53 +01:00
cybermaggedon
a70ae9793a
Flow API - update gateway (#357)
* Altered API to incorporate Flow IDs, refactored for dynamic start/stop of flows
* Gateway: Split endpoint / dispatcher for maintainability
2025-05-02 21:11:50 +01:00
cybermaggedon
9508ac6c69
Port metering to new API, not tested. (#354)
- Port metering to new API
- Moved price list to configuration
- Added tg-set-token-costs and tg-show-token-costs utils.
2025-04-28 21:26:38 +01:00
cybermaggedon
342f555948
Feature/interface descriptions (#348) 2025-04-25 15:59:34 +01:00
cybermaggedon
76e85d895b
Add missing files (#347) 2025-04-24 19:02:51 +01:00
cybermaggedon
3b021720c5
Feature/flow management cli (#346)
Flow management API + various flow management commands

trustgraph-cli/scripts/tg-delete-flow-class
trustgraph-cli/scripts/tg-get-flow-class
trustgraph-cli/scripts/tg-put-flow-class
trustgraph-cli/scripts/tg-show-flow-classes
trustgraph-cli/scripts/tg-show-flows
trustgraph-cli/scripts/tg-start-flow
trustgraph-cli/scripts/tg-stop-flow
2025-04-24 18:57:33 +01:00
cybermaggedon
a759a1771e
Added CLI tg-set-prompt command, can be used to update prompts dynamically. (#338)
Added CLI tg-set-prompt command, can be used to update prompts dynamically. (#338)
2025-04-04 15:43:40 +01:00
cybermaggedon
298d09f388
Prompt and agent manager, dynamically load configuration from config-svc
- prompt-template takes config from the config-svc, dynamically reloads
  as new config appears.
- agent-react takes config from config-svc, dynamically reloads
- Fixed lack of data in config queue, needed to take the Earliest, not the
  Latest values.
- Changed text-completion and knowledge-query tool to both use 'query'
  as the argument.
- Prompt and agent no longer have command line args to supply
  configuration.
2025-04-02 16:37:08 +01:00
cybermaggedon
1d222235d3
Configuration initialisation (#335)
* - Fixed error reporting in config
- Updated tg-init-pulsar to be able to load initial config to config-svc
- Tweaked API naming and added more config calls

* Tools to dump out prompts and agent tools
2025-04-02 13:52:33 +01:00
cybermaggedon
a2c64cad4a
Added config reload handler mechanism, calls a Python method on config (#334) 2025-04-02 00:23:30 +01:00
cybermaggedon
fa09dc319e
Feature/config service (#332)
Configuration service provides an API to change configuration. Complete configuration is pushed down a config queue so that users have a complete copy of config object.
2025-04-01 19:47:05 +01:00
cybermaggedon
ef845d6c9b
Feature/rag parameters (#311)
* Change document-rag and graph-rag processing so that the user can
specify parameters.  Changes in Pulsar services, Pulsar message
schemas, gateway and command-line tools.  User-visible changes in
new parameters on command-line tools.

* Fix bugs, graph-rag working

* Get subgraph truncation in the right place

* Graph RAG and document RAG working and configurable

* Multi-hop path traversal GraphRAG

* Add safety valve for path_size set too high
2025-03-13 00:38:18 +00:00
cybermaggedon
617eb7efd5
Feature/pulsar api key support (#308)
* Add pulsar API token check

* Added missing api_key references

---------

Co-authored-by: Tyler O <4535788+toliver38@users.noreply.github.com>
2025-02-15 11:22:48 +00:00
cybermaggedon
bed7423c26
- Added load/save API for document embeddings (#269)
- tg-load-doc-embeds and tg-save-doc-embeds command line utils
2025-01-16 00:00:59 +00:00
cybermaggedon
1280af3eff
Fix/kg core save (#267)
* Add a 'localhost' Pulsar endpoint for docker env

* - Fix broken socket endpoint streamers
- Add unused listener endpoints for publisher/subscriber
- Fix graph embedding serialisation

* Fix GE load

* Remove Gossip settling delay, this is single-node Cassandra.
2025-01-13 14:42:33 +00:00
cybermaggedon
44f8ce8834
Migrate cli utils to REST API (#239)
* Port a number of commands to use API gateway instead of Pulsar

* Ported tg-invoke-agent to websockets API

* Rename the 2 RAG commands: tg-query-... to tg-invoke-...
2025-01-02 19:49:22 +00:00
cybermaggedon
9942f63773
tg-dump-msgpack previously output Python 'representation', now does JSON. (#233) 2024-12-30 10:51:35 +00:00
cybermaggedon
61031270e4
Fix loop logic flaws in loader (#203) 2024-12-09 12:44:30 +00:00
cybermaggedon
803f110891
Timeout alias error (#202) 2024-12-09 00:31:03 +00:00
cybermaggedon
cf8c76b5c6
Fix/save core hang (#201)
* Working around an exception class change in Python 3.11
2024-12-09 00:01:01 +00:00
cybermaggedon
fd3db3c925
Feature/tidy kg load save (#198)
* Clean exit on ctrl-C
* More functionality in dump
* Dump some metadata
2024-12-06 15:16:09 +00:00
cybermaggedon
2818ec9f23
Fix header (#194) 2024-12-06 08:50:49 +00:00
cybermaggedon
6d200c79c5
Feature/wikipedia ddg (#185)
API-side support for Wikipedia, DBpedia and internet search functions  This incorporates a refactor of the API code to break it up, separate classes for endpoints to reduce duplication
2024-12-02 17:41:30 +00:00
cybermaggedon
cf564ed147
PDF loader symbol error fixed (#183) 2024-11-28 21:26:36 +00:00
cybermaggedon
c52b70c286
Fix metadata load format (#181) 2024-11-28 19:21:41 +00:00
cybermaggedon
2f7ccb2ef8
- Reduce back-pressure on tg-load-kg-core (#179)
- Save entity in correct format in tg-save-core
2024-11-27 17:24:27 +00:00
cybermaggedon
99e3e43f7b
Fix/kg cli help (#173)
* Fix kg-core-help
2024-11-26 16:58:47 +00:00
cybermaggedon
887fafcf8c
Fix/core save api (#172)
* Acknowledge messaages from Pulsar, doh!
* Change API to deliver a boolean e if value is an entity
* Change loaders to use new API
* Changes, entity-aware API is complete
2024-11-26 16:46:38 +00:00
cybermaggedon
340d7a224f
Feature/rework kg core (#171)
* Knowledge cores with msgpack
* Put it in the cli package
* Tidy up msgpack dumper
* Created a loader
2024-11-25 20:46:35 +00:00
cybermaggedon
dc0f54f236
API supports doc & text load (#167) 2024-11-21 14:53:53 +00:00
cybermaggedon
36cdeab588
Feature/agent manager (#146)
* Agent schema
* Agent working through client
* Add agent-manager-react command line
* test-agent test script
* Add tg-invoke-agent CLI
2024-11-10 11:44:01 +00:00
cybermaggedon
ae8661fe2e
Fix/improve command line help (#145)
* Make command line consistent, fix incorrect documentation.

* Improve tg-invoke-prompt help
2024-11-08 18:14:14 +00:00
cybermaggedon
f97856245c
Fix command name typo (#144) 2024-11-07 21:06:13 +00:00
cybermaggedon
66d273cf5a
Fix/cli docs (#143)
* Improved documentation in tg-invoke-prompt.  Added tg-invoke-llm.
These are diagnostics for the text-completion and prompt services.

* Fix command name
2024-11-07 21:01:51 +00:00
cybermaggedon
24d099793d
Feature/doc metadata labels (#130)
* Add schema load util

* Added a sample schema turtle file will be useful for future testing and
tutorials.

* Fixed graph label metadata confusion, was created incorrect subjectOf
edges.
2024-10-29 21:18:02 +00:00
cybermaggedon
dedb66379d
Added tg-invoke-prompt utility to test / diagnose / interact with the (#129) prompt service 2024-10-27 17:15:10 +00:00
cybermaggedon
7954e863cc
Feature: document metadata (#123)
* Rework metadata structure in processing messages to be a subgraph
* Add subgraph creation for tg-load-pdf and tg-load-text based on command-line passing of doc attributes
* Document metadata is added to knowledge graph with subjectOf linkage to extracted entities
2024-10-23 18:04:04 +01:00