trustgraph/trustgraph-flow/trustgraph/gateway/dispatch
cybermaggedon 4fb0b4d8e8
Pub/sub abstraction: decouple from Pulsar (#751)
Remove Pulsar-specific concepts from application code so that
the pub/sub backend is swappable via configuration.

Rename translators:
- to_pulsar/from_pulsar → decode/encode across all translator
  classes, dispatch handlers, and tests (55+ files)
- from_response_with_completion → encode_with_completion
- Remove pulsar.schema.Record from translator base class

Queue naming (CLASS:TOPICSPACE:TOPIC):
- Replace topic() helper with queue() using new format:
  flow:tg:name, request:tg:name, response:tg:name, state:tg:name
- Queue class implies persistence/TTL (no QoS in names)
- Update Pulsar backend map_topic() to parse new format
- Librarian queues use flow class (persistent, for chunking)
- Config push uses state class (persistent, last-value)
- Remove 15 dead topic imports from schema files
- Update init_trustgraph.py namespace: config → state

Confine Pulsar to pulsar_backend.py:
- Delete legacy PulsarClient class from pubsub.py
- Move add_args to add_pubsub_args() with standalone flag
  for CLI tools (defaults to localhost)
- PulsarBackendConsumer.receive() catches _pulsar.Timeout,
  raises standard TimeoutError
- Remove Pulsar imports from: async_processor, flow_processor,
  log_level, all 11 client files, 4 storage writers, gateway
  service, gateway config receiver
- Remove log_level/LoggerLevel from client API
- Rewrite tg-monitor-prompts to use backend abstraction
- Update tg-dump-queues to use add_pubsub_args

Also: pubsub-abstraction.md tech spec covering problem statement,
design goals, as-is requirements, candidate broker assessment,
approach, and implementation order.
2026-04-01 20:16:53 +01:00
..
agent.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
collection_management.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
config.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
core_export.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
core_import.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
document_embeddings_export.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
document_embeddings_import.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
document_embeddings_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
document_load.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
document_rag.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
document_stream.py Fix/doc streaming proto (#673) 2026-03-09 12:36:10 +00:00
embeddings.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
entity_contexts_export.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
entity_contexts_import.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
flow.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
graph_embeddings_export.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
graph_embeddings_import.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
graph_embeddings_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
graph_rag.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
knowledge.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
librarian.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
manager.py Fix/doc streaming proto (#673) 2026-03-09 12:36:10 +00:00
mcp_tool.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
mux.py Fix websocket error responses in Mux dispatcher (#726) 2026-03-28 10:58:28 +00:00
nlp_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
prompt.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
requestor.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
row_embeddings_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
rows_import.py Remove redundant metadata (#685) 2026-03-11 10:51:39 +00:00
rows_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
sender.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
serialize.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
streamer.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
structured_diag.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
structured_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
text_completion.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
text_load.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00
triples_export.py Messaging fabric plugins (#592) 2025-12-17 21:40:43 +00:00
triples_import.py The id field in pipeline Metadata was being overwritten at each processing (#686) 2026-03-11 12:16:39 +00:00
triples_query.py Pub/sub abstraction: decouple from Pulsar (#751) 2026-04-01 20:16:53 +01:00