NetworkChuck is exactly one of the few people I know of who try this stuff and to some extent inspired me. He does take a bit of a different route where he lets N8B AI agents orchestrate his home lab, but more from a technical, IT, perspective. That’s not what I am aiming for. I also gave the situation some thought and tried some stuff. The difference is fundamental.
Every event that occurs has a trigger. The sum of all triggers, and prior knowledge, is context and what I hope to achieve is that with enough context an algorithm can derive intent. This is what I call a “Probabilistic Connect Engine” that generates a “State Probability Fabric” of all properties of all devices in my home. The algorithm for that is based on Bayesian Inference. It might go a bit deep to explain that here. In essence prior knowledge (triggers) can generate probabilistic, posteriori hypotheses (intents). And intents can be translated into actions.
Based on inferred intent, many actions can easily be determined even without the help of AI, which is way faster and cheaper in resources. There are, however, some situations that are ambiguous. In those cases I intend to use AI to resolve a solution.
An example: it is warm inside, the sun is shining, but the wind is increasing. The first two triggers suggest to put the sun screens down (comfort and energy saving). However, the increased wind requires te sun screens to pull up (safety). So, depending on the triggers, the intents that are inferred are “comfort” and “energy saving”, but blocked by the inferred intent “safety”.
Or an example where the homey setup is expanded:
Lets say I have a switch in a room that switches on all the lights in that room in a certain lighting scene or mood. Now, I buy an additional light, perhaps a freestanding light, and add it to the room. In the old way I would have to add this light to all pre-defined automations. This is fine in small setups, but quickly become unmanageable when the devices and capabilities grow. The number of possible permutations explode. In my setup it would be automatically inferred that if all lights go on, also this new light might have to go on. Unless I manually override it, for instance through an app, in which case it would be inferred that this light is operated differently. The system would learn by constantly adjusting its inference weightings.
Like Chuck, I conclude the AI resolution part to be done in a stepped approach: first an AI router/orchestrator decides which specialized AI agent needs to be involved (security, comfort, energy, etc). Than the outputs of these specialist agents need to be resolved for conflicts. And finally, I intend to run it past a “governance” agent, that specifically has the task to check if it is within guardrails (I don’t intend to have a “human in the loop”).
I’ve written a high level architecture document and published a bit on my LinkedIn profile. See architecture diagram below. Obviously there is more to it, like heuristics, that can greatly reduce the workload of an automation engine. And I have done some partial performance and scaling tests and cost calculations. My home had 317 devices (
) and more than 1500 capabilities (sensors and actuators). Some constantly generate triggers (like energy sensors), others hardly ever. I had Homey pull all states of all capabilities every minute and the amount of data that you can generate quickly becomes a serious challenge and cost. There are all kinds of practical considerations to be taken into account: hardware, performance, storage, AI, data privacy, etc, etc.
I’v not been able to work any further on it. I was work-constrained and I was on holiday… Not sure if I will bring it further, because it stretches my IT skills quite a bit. I am more of a conceptual guy (physicist) than an engineer. And there are probably practical hurdles to be solved. But I believe it could work.
And Philips’ recent announcement of MotionAware (using RF Sensing in their Hue light bulbs) opens an enormous tsunami of possibilities, only necessitating even more an approach like the probabilistic one I took.
Let me know if you want a more detailed document I wrote explaining it conceptually (23 pages).
