SwissDataScienceCenter / renku-data-services / 10715718710

86%

Build Type

push

github

Committed by web-flow

Commit Message

chore: Send all search synchronisation events to a single queue (#377)

* chore: Send all search synchronisation events to a single queue

Instead of using multiple queues/streams all the events are now sent
to a single stream `search.sync`. They are only intended to
synchronize data between data-services and search-service.

This requires a change to the `Event.queue` field. It was used for two
things: the redis qeue/stream name and the message type. Now these two
are different now and so the event must carry both separately. Events
are first stored to the postgres database as single value (its avro
payload) and the qeue name is stored alongside. When re-creating the
event object from the database, we don't have the `(header, payload)`
tuple, but only a `dict[str, Any]` (the raw message). This is
different to when creating the event from a `message-type` and
structured avro object. Since we never need to read the structured
avro object of an event, the `Event` class is now simplified to only
carry the raw message. This makes the generic type param obsolete and
removes the early return in the `serialize` method which checked the
payloads runtime type (as when read from the db).

The `serialize` method is still there to keep the interface, allowing
perhaps later to add more structure to the class.

Additional to the canonical constructor of the event, there is a
second one, `create`, for creating an event from a structured avro
object and message type.

NOTE: This change requires search-services to be at least version 0.5.0 (as
 in renku 0.57.0).

Run Details

30 of 30 new or added lines in 3 files covered. (100.0%)

7 existing lines in 5 files now uncovered.

9302 of 10270 relevant lines covered (90.57%)

1.61 hits per line