[APP][Pro] Gemini AI - Control your Smart Home with Google's Gemini

Hi @DimitriEbben, @J273 !

are you referring to the “send a prompt with image” action card:

or to the action card “send a command to smart home” with a command like “check if the entrance cam frames a person”?

Hi yeah that’s right.

For me, I have an advanced flow which sends me a notification to my phone telling me who’s at my front door from my doorbell camera (send a prompt with image) The image doesn’t match the analyzed answer.

Hi @s_dimaio,

I’m referring to the “send a prompt with image” action card.

Dimitri

Tried the app. With my first question from Gemini via flow I tried to get a device listing, but got more than fifty notifications in a row. When I read them, the notifications formed a flash fiction story that made no sense. I must have done something wrong :thinking:

Hi @DimitriEbben, it’s certainly possible to modify the ‘send a prompt with image’ action card to return a second token with the image parsed by Gemini. However, I don’t think it’s strictly necessary. Wouldn’t a Flow like this already solve the problem? Or have I misunderstood?

Hi @J273, this is strange. Try using a more advanced model like the Gemini 3.1 Flash-Lite, although the 2.5 Flash-Lite should be more than adequate for analyzing images. When you’re done, send me a diagnostic report and I’ll try to take a look.

Hi @Jari_Peuhkurinen , to get a list of devices connected to Homey (or more generally for any smart home-related command), you must use the “send a command to smart home” action card:

Furthermore, managing your smart home is more complex than simply generating text or images, so (i) you must use a more sophisticated model for it to respond properly (I recommend Gemini 3.1 Flash-Lite, which offers the best value for money) and (ii) you must associate billing with your API Key for extended use of this feature.

Hi @s_dimaio,

The flow you created here is exactly how I set it up as well. The problem you run into is the following: the camera image (photo 1) with the prompt gets sent, then a few seconds later Gemini responds, after which a new snapshot is taken (photo 2 in the notification card), the response from Gemini is added, and everything is sent via the notification.

Because there is a time difference between photo 1 and photo 2—sometimes a few seconds—the analyzed image (photo1) doesn’t match the second photo (photo 2). Example: photo 1 shows a person a few seconds later, photo 2 is taken for the notification card, and by then the person is already out of the frame.

That’s why the image sent to Gemini should be retrievable as a tag, so that exactly that same image can be included in the notification together with Gemini’s response.

In my case, every time the cam’s action card “request a new camera snapshot” is called, it generates an image, and this same image is passed to gemini and the notification (so the image is generated once at the start of the flow, and this same image is taken within the flow as many times as needed). I checked https://tools.developer.homey.app/ in the images section.

Anyway, if you think it might be useful, I’ll work on it. I’ll publish the test version this week!

Bye!

@s_dimaio

That would be great. I think this is caused by the ONVIF camera app taking a new snapshot. Maybe you could also look into the possibility of adding an “if” card (see my earlier response), so we can trigger other flows once Gemini has given a response, including the response tag and photo tag within it.

I’m going to test it when it’s ready and will also make a donation for your work. Thanks again!

Hi @s_dimaio

Yeah what @DimitriEbben describes here is what happens for me as well.