Subscribe homie/+/# is effectively what’s happening though as it’s every device
Possibly limited to the Homey homie tree
homie/homeyname/+/#
Subscribe homie/+/# is effectively what’s happening though as it’s every device
Possibly limited to the Homey homie tree
homie/homeyname/+/#
@robertklep Can you reproduce the actual problem?
If you restart the hub, does it cause any flow to be disabled?
I’ve got 1 device in Homey (Homey itself), so it’s hard to test
I’ll check too later when I get back home. I have a few devices …Got to go
I have not seen this issue but don’t use incoming MQTT flow triggers really
Interesting though
I hardly have any devices in Homey, and not running the app, so not sure why I said that I could reproduce the actual problem…
Not sure if this helps but whenever I try to discover new devices in Tasmota MQTT I get the too many flows error occur , they then all get disabled and I then have to manually re-enable them again. All these flows use MQTT client …
I have tried hard to utilise the in-built 433MHZ feature in Homey (see my other posts) but I gave up in the end and now use a SonOff RF bridge thats been flashed with Tasmota and communicate to all my 433Hhz devices via MQTT…
When listening for MQTT events from my 433Mhz PIR sensors I use this method to capture all the incoming messages .
See flow (below)…
Basically I just get the system to scan and listen for a specific piece of text in all of the incoming MQTT data (identifying the unique device identity) and then make trigger depending on that devices identity…
This is the only on MQTT communication I use in my flows or system but I get the flow overload error when ever I get Tasmota MQTT to discover new devices . Not sure if this helps but just thought I’d share it …
The cause of the problem is line 37-40 in messagehandling.js
It triggers a flow card for each incoming message.
@scanno Can we match the incoming message topic against ‘ref.triggers.broker.getTopicArray().getTriggerTopics()’ before actually executing the trigger?
So the rate limiter is tied to .trigger()
being called, and not the run listener for that trigger that may (subsequently) decide that the trigger isn’t meant for it? Not sure if that makes sense…
FWIW, I already tested this in my PoC and I wasn’t able to trigger the rate limiter:
const trigger = this.homey.flow.getTriggerCard('test');
trigger.registerRunListener((args, state) => {
console.log('listener triggered', args, state);
return Promise.resolve(false);
});
for (let i = 0; i < 500; i++) {
trigger.trigger({ idx: i }, {});
}
I can reproduce the problem by adding a flow card trigger on a broad wildcard topic like ‘#’ or ‘homie/#’. Such a flow will be disabled in no time due the many triggers, but that’s expected behaviour.
Found something else, I see these log messages a lot:
20210409-18:18:25 MQTT Offline
20210409-18:18:25 De MQTT broker is ONBESCHIKBAAR
20210409-18:18:29 Broker State: DISCONNECTED
20210409-18:18:35 MQTT Closed
20210409-18:18:35 Broker State: DISCONNECTED
20210409-18:18:37 MQTT Reconnect
20210409-18:18:37 Broker State: RECONNECTING
20210409-18:18:42 De MQTT broker is weer BESCHIKBAAR
20210409-18:18:45 MQTT client connected
20210409-18:18:46 Broker State: CONNECTED
Each time the broker is reconneced, all retained messages on the broker (thousands in my case) are send to the client. This causes an endless stream of the same messages.
@Zimo To answer your initial question about the log:
LOG explained:
The next line says: We are not waiting for this topic
meaning: The flow card is not triggered by this message, because the pattern does not match.
It does not explain why the flow gets disabled though…
NOTE: Your log is from the MQTT client, not from the Hub.
That would not solve all problems… You need to iterate through the trigger cards anyway to see if a topic matches.
But a topic match can be checked before even calling the trigger? The flowcard trigger topics are known on beforehand. I know it shouldn’t matter because the trigger response will be false anyway, but who knows…
You only prevent iterating through the trigger cards when there is no registered topic. You still have to go through them all to do the matching when the topic is registered.
The best solution would be to have a list of topics and the corresponding trigger card id’s and then be able to trigger that specific card.
Sorry Harrie for dumb question, is there a way to get better logs then copy-pasting log-lines from app interface? Because it trims logs and I can see only last 30 seconds after hub restart and there is much more log lines which I cannot capture…
Was just curious as to how this went. Was the problem figured out ?
@Russell_S Well…
I made a pull request with some improvements to the MQTT Client.
@scanno Needs to look into it before it can be send to the store.
But maybe someone is willing to test already by installing via CLI?
branch: GitHub - harriedegroot/nl.scanno.mqtt at feature/topic-subscription
DETAILS
Commit 1
Topic registration:
Commit 2
Commit 3
Flow card triggers:
Commit 4
pre-check to prevent uneccesairy calls to the Homey trigger system
prevents disabling of random flows not even listening to these topics (Homey BUG?)…
this is also a performance boost, since the Homey trigger system won’t be called for all api topics
the small downside is the duplicate topic matching, if there is a match on a trigger topic
Commit 5
(added missing file ‘Trigger.js’ for commit 3
Commit 6
On change of app settings:
Commit 7
added unsubscribe api
Commit 8
Introduced a TriggerQueue to handle the rate limiting of flow card triggers and prevent disabling of flows
Commit 9
Added broker online/offline flow card triggers
Commit 10
Added flow card action to send an empty message to be able to remove retained messages from the broker
Commit 11
Introduced a TopicsRegistry to register topics by reference (e.g. app id). The unsubscribe api can only be called for topics with the reference provided during topic subscription. This prevents unsubscription of topics used by other apps/processes.
Also optimized the topic matching by using a topic map (instead of array lookup).
I’ve got this running for a few days now with several thousands of (retained) messages. Also have a test flow with a trigger on topic ‘#’, which forwards the data to another mqtt topic. Non of the flows has been disabled yet…
As soon as I can find some spare time, I will have a look at the pull request. Work and thesis consume all my time right now.
Thanks for the effort.