[APP][Pro] AI Voice Assistant

Apps
, , ,
1

large
large500×350 200 KB

:studio_microphone: AI Voice Assistant — talk to your Homey

Now certified and live in the Homey App Store!

This app turns small, inexpensive ESPHome voice devices into a natural voice interface for Homey. Say the wake word (e.g. “Okay Nabu”), speak normally, and the assistant controls your devices, answers questions, plays music, sets timers and speaks back — in your language. It keeps listening after it answers, so you can ask follow-up questions without repeating the wake word.

What you can do by voice

  • Control your smart home — lights, plugs, dimming, thermostats, in any room/zone
  • Lock & unlock doors — locking always works; unlocking is off by default for safety (opt-in in settings, and never more than one lock per command)
  • Weather — current conditions and forecasts for your location
  • Timers & alarms — “set a 10 minute timer” — the device counts down on its LED ring and chimes when time’s up
  • Search the web — “what’s playing at the cinema tonight?” — via OpenAI web search or the Brave Search API
  • Manage your Bring! shopping list — “add milk”, “what’s on the list?” (opt-in)
  • Play music — “play Abbey Road by the Beatles”, “next song”, “play some Queen in the kitchen” — through a Music Assistant server on your network; the audio streams from the server straight to the speaker
  • Ask anything — general questions, time & date, and “what can you do?” for help

The assistant understands and replies in English, Dutch, German, French, Italian, Swedish, Norwegian, Spanish, Danish, Russian, Polish and Korean.

You choose the AI engine

  • OpenAI Realtime — low-latency cloud speech-to-speech (full or mini model), with advanced tuning for speech detection sensitivity and pause length
  • Google Gemini Live — the same real-time pipeline, powered by Gemini
  • Mistral (Voxtral) — the European alternative on one API key: streaming Voxtral speech recognition that transcribes while you talk, a Mistral chat model, and Voxtral voices
  • Custom / self-hosted pipeline — Whisper, Ollama, LM Studio, Piper, Wyoming, and any OpenAI-compatible server (Groq, OpenRouter, llama.cpp, vLLM, …). Every stage (speech-to-text, language model, text-to-speech) is independently pluggable, with test buttons in the settings — and with a fully local setup, no audio ever leaves your network

llm-providers-transparent-small
llm-providers-transparent-small800×100 15.3 KB

Supported hardware

  • Home Assistant Voice Preview Edition — stock ESPHome firmware, no Home Assistant needed
  • ThirdReality Voice & Music Assistant — works out of the box, no flashing at all, and doubles as a Music Assistant multi-room speaker
  • Seeed ReSpeaker XVF3800 (XIAO ESP32S3) — for the tinkerers: a DIY satellite with a 4-mic XMOS array doing echo cancellation and beamforming on-chip; you compile and flash the community ESPHome firmware yourself
  • XiaoZhi AI devices — the cheap ESP32-S3 gadgets in many shapes, running RealDeco’s ESPHome firmware

large
large500×500 41 KB

large
large500×500 39.9 KB

large
large500×500 26.3 KB

large
large500×500 67.6 KB

The app talks to the devices directly over your LAN using the ESPHome native API — no Home Assistant installation is needed. Devices with an ESPHome API encryption key (e.g. previously adopted by Home Assistant) are fully supported with encrypted connections.

Pairing is easy, even for brand-new devices: the pairing wizard can put an out-of-the-box Voice PE or ThirdReality onto your Wi-Fi via Bluetooth — no vendor apps or cables. Devices already on the network are found automatically, and there’s manual IP entry as a fallback for networks where mDNS doesn’t reach the Homey.

Flow support

Flow cards include Ask the assistant (answer as a text tag, or spoken on the device), Say (text-to-speech), Play an audio URL, timer start/cancel plus timer triggers, and Heard something / Thinking triggers that show what the assistant heard, which tools it used and what it replied — great for following a whole conversation on the timeline.

Requirements

  • A compatible ESPHome voice device (see above)
  • An API key for your chosen cloud engine (OpenAI, Google Gemini or Mistral), or your own local AI services
  • Optional: a Music Assistant server for music playback

Links

Feedback, questions and device reports are very welcome in this topic!

2

I am waiting, hopefully it will be soon I’m also testing homey-Homeassistant-esp home and my Synology nas I have 4 servers working now.

Please let me know if you know how.

3

A test version is now available:

Instructions on how to setup can be found here: Readme

Please let me know how testing goes :slight_smile:

4

Working like a charm. I will make a demonstration with it on Researcher Night.

5

Hi @Arve_Bjornerud,

ist this app still under active development or is it abandoned?
Does anybody use it and what are the experiences with it?

After all the troubles with all the Amazon Alexa integrations in the last weeks I will definitely switch to Home Assistant Voice PE. But I do not want to do the Homey integration based on a “dead” app.

Any advice for me?

6

Ehhh relax a bit, it’s been only 2 months since release. It’s not Widows software needing updates every day.

I do however get your concerns,

but that’s a bit useless, isn’t it?
Imagine the app is not “dead”, so you buy your stuff and install everything.
And then, the next day life happens to the dev and he/she has to stop maintaining it.
You seem to forget apps keep running, even when “dead”, to use your unpleasant words.

7

Hi. It’s not dead, i basically thought nobody was interested in this project so I just put it on hold. But now I’ll get back into it.

8

First of all, I want to thank the developer Arve_Bjornerud, who like most developers uses their skill, abilities, and time to give us new features and compatibility for our Homey.
I came across this project by chance, and it seems very interesting.

9

Hi all,

I really didn´t mean to insult @Arve_Bjornerud or anybody else. - So pls take my apologies. This is a great project and it is well alive :wink:

Just dropping a hint on all the currently desperate Amazon Alexa/Echo threads should create some interest in this alternative, I imagine.

10

I will try this later this week.

Can it does the same as when connected to Home Assistant?

11

Is this app still active? I’m trying to connect a home assistent voice preview edition speaker to homey following the installation notes from the github site but my speaker continues to flash blue and is not found by homey. Any advice how to proceed?

12

Hey. Sorry I’ve been busy with life, so there hasn’t been any development on this for a long time. I’m planning to get back to this. As i see that having this type of connection with homey would be real useful. Basically you can just ask homey to do something for you without pulling out your phone. Just in the same way as Google Nest, Alexa and so on. I had a lot of issues with the OpenAI API last i worked on this. But at that point the API was new and in Beta, perhaps it’s a bit more reliable now. I’ll see if i can find some time to look at this again.

13

@Per-Erik_Sundberg What firmware have you flashed on the PE? Can you ping the PE on the static IP that you set? Does Homey find the PE on your network?

14

Hi @Arve_Bjornerud,

Thank you for the work you are doing on this app, it looks fantastic!

I have installed the latest stable version (v26.4.0) on the PE and set a static IP.

I can ping the PE, but when I try to add the PE to homey I get the message “No new units found”. The PE is flashing blue after startup.

15

I just noticed that when I try to add the PE and get the message “No new units found”, the PE change from flashing blue to flashing red.

16

Hmm.. Then Homey is finding the PE on your network, but is having some issue finalizing the connection. Basically there is no easy way to check if the device it found on the network is an voice enabled device. So Homey has to “ask” the device what its capable of doing. If it doesn’t support the stuff it needs it’s rejected and you get a “No new unit found”. There might be a change in the PE firmware. I’ll try updating mine and see what happens.

17

Hi @Arve_Bjornerud

I would like to stay (if not local then rather) european. - Would it be feasible to make Mistral available as an alternative AI?

Can it be used as a local TTS device?

18

Sorry I thought I was in the Dutch forum. Likewise, why not use Ollama or other open source, since OpenAI and all other parties want to recoup their money, something that will never happen, but so be it.

By the way, I don’t begrudge those idiotic Americans our money from Europe!

I would really appreciate it if Ollama were used, because I am not planning to pay for the use of APIs.

Thank you again for all the effort being put in.

19

Could you pls post in english?

20

You can use the Ollama app for Homey to generate responses with Ollama (optionally with images as well, very useful for analyzing camera snapshots)

I don’t know if this is what you’re looking for though. It doesn’t support controlling Homey devices from a prompt.