Retained message handling: add "prefix" option

[smurfix]

I want to outsource retained message handling.

Background: I'm working on a distributed, master-less, eventually-consistent key-value storage system. The storage server wants to see the messages that a client wants to have retained before they're forwarded, so that it can guarantee the eventually-consistent part.

My idea is to add a "prefix" option to retained_messages_mode: retained messages arriving for the foo/bar/baz topic would be sent to $RETAIN/foo/bar/baz instead, where my server could pick them up, check and store them, then forward them to the original topic (without the retain flag obviously).

[halfgaar]

It sounds like something for a plugin, maybe?

What does the 'checking' part entail? Because you can have your storage subscriber subscribe with 'retain as published', so it can see which messages were originally published retained. Then perhaps we need one more retained messages mode: enabled_without_retaining (better name pending). This is like downgrade, except the retain bit will still be present on the messages for subscribers that do 'retain as published'.

[smurfix]

checking Ah, yes, a plugin might work for this.

Checking may require not publishing the message and telling the client that there was a conflict, so I do need to redirect it initially.

That being said, I still like your enabled_without_retaining flag idea because I'm also linking multiple MQTT servers. Thus when messages from another server arrive my storage service still needs to know whether to save them.

[halfgaar]

Perhaps a programmatic solution is (also) welcome. Like here, to be able to control what to do with a published retained message. I'd need to review a bit what the consequences are.

That success_without_retained_delivery would partly already solve your problem, that new clients that susbcribe don't get retained messages, but of course you will still have all the overhead.

[halfgaar]

I can have a test version of enabled_without_retaining and plugin AuthResult::success_without_setting_retained ready soon. How do you want it? Just a github branch? A .deb file?

[smurfix]

Branch is sufficient. Take your time, I'm on holiday for two weeks.

[halfgaar]

I included it in the release, because it turned out that there was a bug. The downgrade mode behaved like the new enabled_without_retaining. So, that had to be fixed, and I included this new option.

I can also be done in the plugin.