apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-25 08:26:21 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
trustgraph/trustgraph-cli/trustgraph/cli
History
cybermaggedon 9f84891fcc
Flow service lifecycle management (#822) 
feat: separate flow service from config service with explicit queue
lifecycle management

The flow service is now an independent service that owns the lifecycle
of flow and blueprint queues. System services own their own queues.
Consumers never create queues.

Flow service separation:
- New service at trustgraph-flow/trustgraph/flow/service/
- Uses async ConfigClient (RequestResponse pattern) to talk to config
  service
- Config service stripped of all flow handling

Queue lifecycle management:
- PubSubBackend protocol gains create_queue, delete_queue,
  queue_exists, ensure_queue — all async
- RabbitMQ: implements via pika with asyncio.to_thread internally
- Pulsar: stubs for future admin REST API implementation
- Consumer _connect() no longer creates queues (passive=True for named
  queues)
- System services call ensure_queue on startup
- Flow service creates queues on flow start, deletes on flow stop
- Flow service ensures queues for pre-existing flows on startup

Two-phase flow stop:
- Phase 1: set flow status to "stopping", delete processor config
  entries
- Phase 2: retry queue deletion, then delete flow record

Config restructure:
- active-flow config replaced with processor:{name} types
- Each processor has its own config type, each flow variant is a key
- Flow start/stop use batch put/delete — single config push per
  operation
- FlowProcessor subscribes to its own type only

Blueprint format:
- Processor entries split into topics and parameters dicts
- Flow interfaces use {"flow": "topic"} instead of bare strings
- Specs (ConsumerSpec, ProducerSpec, etc.) read from
  definition["topics"]

Tests updated
2026-04-16 17:19:39 +01:00
..
__init__.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
add_library_document.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
delete_collection.py Collection management (#520) 2025-09-18 15:57:52 +01:00
delete_config_item.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
delete_flow_blueprint.py Fix config inconsistency (#609) 2026-01-14 12:31:40 +00:00
delete_kg_core.py Fix CLI docs (#470) 2025-08-27 09:08:20 +01:00
delete_mcp_tool.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
delete_tool.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
dump_msgpack.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
dump_queues.py RabbitMQ pub/sub backend with topic exchange architecture (#752) 2026-04-02 12:47:16 +01:00
get_config_item.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
get_document_content.py Fix/librarian broken (#674) 2026-03-09 13:36:24 +00:00
get_flow_blueprint.py Fix config inconsistency (#609) 2026-01-14 12:31:40 +00:00
get_kg_core.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
graph_to_turtle.py Feature/streaming triples (#676) 2026-03-09 15:46:33 +00:00
init_pulsar_manager.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
init_trustgraph.py Refactor: Derive consumer behaviour from queue class (#772) 2026-04-09 09:55:41 +01:00
invoke_agent.py Add agent explainability instrumentation and unify envelope field naming (#795) 2026-04-13 16:16:42 +01:00
invoke_document_embeddings.py Fix doc embeddings invocation (#672) 2026-03-09 11:07:32 +00:00
invoke_document_rag.py Expose LLM token usage across all service layers (#782) 2026-04-13 14:38:34 +01:00
invoke_embeddings.py Batch embeddings (#668) 2026-03-08 18:36:54 +00:00
invoke_graph_embeddings.py Fix doc embeddings invocation (#672) 2026-03-09 11:07:32 +00:00
invoke_graph_rag.py Expose LLM token usage across all service layers (#782) 2026-04-13 14:38:34 +01:00
invoke_llm.py Expose LLM token usage across all service layers (#782) 2026-04-13 14:38:34 +01:00
invoke_mcp_tool.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
invoke_nlp_query.py Structured query support (#492) 2025-09-04 16:06:18 +01:00
invoke_prompt.py Expose LLM token usage across all service layers (#782) 2026-04-13 14:38:34 +01:00
invoke_row_embeddings.py Row embeddings APIs exposed (#646) 2026-02-23 21:52:56 +00:00
invoke_rows_query.py Structured data 2 (#645) 2026-02-23 15:56:29 +00:00
invoke_sparql_query.py SPARQL CLI reports errors from service (#794) 2026-04-13 14:31:33 +01:00
invoke_structured_query.py Extend use of user + collection fields (#503) 2025-09-08 18:28:38 +01:00
list_collections.py Fix/queue configurations (#585) 2025-12-06 14:54:47 +00:00
list_config_items.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
list_explain_traces.py Persistent websocket connections for socket clients and CLI tools (#723) 2026-03-26 16:46:28 +00:00
load_doc_embeds.py Document chunks not stored in vector store (#665) 2026-03-07 23:10:45 +00:00
load_kg_core.py Fix CLI docs (#470) 2025-08-27 09:08:20 +01:00
load_knowledge.py Feature/more cli diags (#624) 2026-02-04 14:10:30 +00:00
load_sample_documents.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
load_structured_data.py Structured data 2 (#645) 2026-02-23 15:56:29 +00:00
load_turtle.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
monitor_prompts.py Refactor: Derive consumer behaviour from queue class (#772) 2026-04-09 09:55:41 +01:00
put_config_item.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
put_flow_blueprint.py Fix config inconsistency (#609) 2026-01-14 12:31:40 +00:00
put_kg_core.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
query_graph.py Add unified explainability support and librarian storage for (#693) 2026-03-12 21:40:09 +00:00
remove_library_document.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
save_doc_embeds.py Document chunks not stored in vector store (#665) 2026-03-07 23:10:45 +00:00
set_collection.py Fix/queue configurations (#585) 2025-12-06 14:54:47 +00:00
set_mcp_tool.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
set_prompt.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
set_token_costs.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
set_tool.py Row embeddings APIs exposed (#646) 2026-02-23 21:52:56 +00:00
show_config.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
show_explain_trace.py Split Analysis into Analysis+ToolUse and Observation, add message_id (#747) 2026-03-31 17:51:22 +01:00
show_extraction_provenance.py Subgraph provenance (#694) 2026-03-13 11:37:59 +00:00
show_flow_blueprints.py Persistent websocket connections for socket clients and CLI tools (#723) 2026-03-26 16:46:28 +00:00
show_flow_state.py Fix config inconsistency (#609) 2026-01-14 12:31:40 +00:00
show_flows.py Flow service lifecycle management (#822) 2026-04-16 17:19:39 +01:00
show_graph.py Terminology Rename, and named-graphs for explainability (#682) 2026-03-10 14:35:21 +00:00
show_kg_cores.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
show_library_documents.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
show_library_processing.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
show_mcp_tools.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
show_parameter_types.py Persistent websocket connections for socket clients and CLI tools (#723) 2026-03-26 16:46:28 +00:00
show_processor_state.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
show_prompts.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
show_token_costs.py Fix config inconsistency (#609) 2026-01-14 12:31:40 +00:00
show_token_rate.py Migrate from setup.py to pyproject.toml (#440) 2025-07-23 21:22:08 +01:00
show_tools.py Row embeddings APIs exposed (#646) 2026-02-23 21:52:56 +00:00
start_flow.py Fix config inconsistency (#609) 2026-01-14 12:31:40 +00:00
start_library_processing.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
stop_flow.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
stop_library_processing.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
unload_kg_core.py Python API implements streaming interfaces (#577) 2025-12-04 17:38:57 +00:00
verify_system_status.py Remove Pulsar healthcheck from tg-verify-system-health (#809) 2026-04-14 16:12:24 +01:00