apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-04-25 08:26:21 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Pub/sub abstraction: decouple from Pulsar (#751)

Browse source 
Remove Pulsar-specific concepts from application code so that
the pub/sub backend is swappable via configuration.

Rename translators:
- to_pulsar/from_pulsar → decode/encode across all translator
  classes, dispatch handlers, and tests (55+ files)
- from_response_with_completion → encode_with_completion
- Remove pulsar.schema.Record from translator base class

Queue naming (CLASS:TOPICSPACE:TOPIC):
- Replace topic() helper with queue() using new format:
  flow:tg:name, request:tg:name, response:tg:name, state:tg:name
- Queue class implies persistence/TTL (no QoS in names)
- Update Pulsar backend map_topic() to parse new format
- Librarian queues use flow class (persistent, for chunking)
- Config push uses state class (persistent, last-value)
- Remove 15 dead topic imports from schema files
- Update init_trustgraph.py namespace: config → state

Confine Pulsar to pulsar_backend.py:
- Delete legacy PulsarClient class from pubsub.py
- Move add_args to add_pubsub_args() with standalone flag
  for CLI tools (defaults to localhost)
- PulsarBackendConsumer.receive() catches _pulsar.Timeout,
  raises standard TimeoutError
- Remove Pulsar imports from: async_processor, flow_processor,
  log_level, all 11 client files, 4 storage writers, gateway
  service, gateway config receiver
- Remove log_level/LoggerLevel from client API
- Rewrite tg-monitor-prompts to use backend abstraction
- Update tg-dump-queues to use add_pubsub_args

Also: pubsub-abstraction.md tech spec covering problem statement,
design goals, as-is requirements, candidate broker assessment,
approach, and implementation order.
This commit is contained in:
cybermaggedon 2026-04-01 20:16:53 +01:00 • committed by GitHub
parent dbf8daa74a
commit 4fb0b4d8e8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
106 changed files with 1269 additions and 788 deletions
Download patch file Download diff file Expand all files Collapse all files

24
tests/contract/test_translator_completion_flags.py
View file

@ -33,7 +33,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is True, "is_final must be True when end_of_session=True"
@ -57,7 +57,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is False, "is_final must be False when end_of_session=False"
@ -80,7 +80,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is False
@ -103,7 +103,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is False, "end_of_stream=True should NOT make is_final=True"
@ -125,7 +125,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is True, "is_final must be True when end_of_session=True"
@ -147,7 +147,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is False, "end_of_stream=True should NOT make is_final=True"
@ -168,7 +168,7 @@ class TestRAGTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is False, "is_final must be False when end_of_stream=False"
@ -195,7 +195,7 @@ class TestAgentTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is True, "is_final must be True when end_of_dialog=True"
@ -217,7 +217,7 @@ class TestAgentTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is False, "is_final must be False when end_of_dialog=False"
@ -240,7 +240,7 @@ class TestAgentTranslatorCompletionFlags:
)
# Act
thought_dict, thought_is_final = translator.from_response_with_completion(thought_response)
thought_dict, thought_is_final = translator.encode_with_completion(thought_response)
# Assert
assert thought_is_final is False, "Thought message must not be final"
@ -254,7 +254,7 @@ class TestAgentTranslatorCompletionFlags:
)
# Act
obs_dict, obs_is_final = translator.from_response_with_completion(observation_response)
obs_dict, obs_is_final = translator.encode_with_completion(observation_response)
# Assert
assert obs_is_final is False, "Observation message must not be final"
@ -275,7 +275,7 @@ class TestAgentTranslatorCompletionFlags:
)
# Act
response_dict, is_final = translator.from_response_with_completion(response)
response_dict, is_final = translator.encode_with_completion(response)
# Assert
assert is_final is True, "Streaming format must use end_of_dialog for is_final"