mirror of
https://github.com/trustgraph-ai/trustgraph.git
synced 2026-05-14 01:32:36 +02:00
Config push notify pattern: replace stateful pub/sub with signal+ fetch (#760)
Replace the config push mechanism that broadcast the full config blob on a 'state' class pub/sub queue with a lightweight notify signal containing only the version number and affected config types. Processors fetch the full config via request/response from the config service when notified. This eliminates the need for the pub/sub 'state' queue class and stateful pub/sub services entirely. The config push queue moves from 'state' to 'flow' class — a simple transient signal rather than a retained message. This solves the RabbitMQ late-subscriber problem where restarting processes never received the current config because their fresh queue had no historical messages. Key changes: - ConfigPush schema: config dict replaced with types list - Subscribe-then-fetch startup with retry: processors subscribe to notify queue, fetch config via request/response, then process buffered notifies with version comparison to avoid race conditions - register_config_handler() accepts optional types parameter so handlers only fire when their config types change - Short-lived config request/response clients to avoid subscriber contention on non-persistent response topics - Config service passes affected types through put/delete/flow operations - Gateway ConfigReceiver rewritten with same notify pattern and retry loop Tests updated New tests: - register_config_handler: without types, with types, multiple types, multiple handlers - on_config_notify: old/same version skipped, irrelevant types skipped (version still updated), relevant type triggers fetch, handler without types always called, mixed handler filtering, empty types invokes all, fetch failure handled gracefully - fetch_config: returns config+version, raises on error response, stops client even on exception - fetch_and_apply_config: applies to all handlers on startup, retries on failure
This commit is contained in:
parent
d4723566cb
commit
4acd853023
37 changed files with 1449 additions and 406 deletions
|
|
@ -148,18 +148,7 @@ class Configuration:
|
|||
|
||||
async def handle_delete(self, v):
|
||||
|
||||
# for k in v.keys:
|
||||
# if k.type not in self or k.key not in self[k.type]:
|
||||
# return ConfigResponse(
|
||||
# version = None,
|
||||
# values = None,
|
||||
# directory = None,
|
||||
# config = None,
|
||||
# error = Error(
|
||||
# type = "key-error",
|
||||
# message = f"Key error"
|
||||
# )
|
||||
# )
|
||||
types = list(set(k.type for k in v.keys))
|
||||
|
||||
for k in v.keys:
|
||||
|
||||
|
|
@ -167,20 +156,22 @@ class Configuration:
|
|||
|
||||
await self.inc_version()
|
||||
|
||||
await self.push()
|
||||
await self.push(types=types)
|
||||
|
||||
return ConfigResponse(
|
||||
)
|
||||
|
||||
async def handle_put(self, v):
|
||||
|
||||
types = list(set(k.type for k in v.values))
|
||||
|
||||
for k in v.values:
|
||||
|
||||
await self.table_store.put_config(k.type, k.key, k.value)
|
||||
|
||||
await self.inc_version()
|
||||
|
||||
await self.push()
|
||||
await self.push(types=types)
|
||||
|
||||
return ConfigResponse(
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue