[APP][Pro] AI Voice Assistant - In TEST phase

Apps
, , ,
41

AI Voice Assistant v1.4.1

This one grew into a much bigger release than the version number suggests. Highlights: a new supported device, two new voice engines (including a fully local one), music control, Wi-Fi setup over Bluetooth, and a long list of stability fixes.

:new_button: New device: ThirdReality Voice & Music Assistant

  • Full support as a voice satellite — pairing, wake word, conversations and music playback.
  • New Flow trigger for the device’s physical button (single, double, triple and long press).
  • Tuned microphone gain and a connection keepalive so the link stays stable.

:speaking_head: New voice engines

  • Mistral joins OpenAI and Google Gemini as a voice engine — Voxtral realtime speech recognition and Voxtral voices, with full smart-home tool support.
  • Custom pipeline — run the assistant on your own hardware, with no cloud required. Mix and match each stage:
    • Speech-to-text: Whisper, Wyoming faster-whisper (the Home Assistant add-on), Mistral Voxtral (including realtime streaming), or any OpenAI-compatible server.
    • Language model: Ollama, LM Studio, Mistral, or any OpenAI-compatible server (Groq, OpenRouter, DeepSeek, llama.cpp, vLLM…).
    • Text-to-speech: Piper, Wyoming Piper, Mistral Voxtral, or any OpenAI-compatible server.
    • The assistant starts speaking sentence-by-sentence while the language model is still writing, and streaming speech recognition means the transcript is ready the moment you stop talking.
    • Small local models are protected against context-window overflow, with a configurable context size for Ollama.
  • OpenAI users are automatically migrated off the deprecated gpt-realtime models.

:antenna_bars: Wi-Fi setup over Bluetooth

Brand-new devices can now be put on your Wi-Fi directly from the pairing wizard in the Homey app, over Bluetooth (Improv BLE). No Home Assistant, ESPHome tooling or USB cable needed — power the device, pair, enter your Wi-Fi credentials, done. Works for both the Voice PE and the ThirdReality.

:musical_note::shopping_cart::magnifying_glass_tilted_right: New assistant abilities (all optional, off by default)

  • Music — control Music Assistant by voice on both devices: play artists/albums/playlists, pause, volume, “what’s playing?”. Compatible with Music Assistant 2.9.
  • Shopping list — manage your Bring! shopping list by voice.
  • Web search — let the assistant look things up on the web (OpenAI or Brave as search provider).

:gear: Settings page redesign

  • Reorganized into sections with a dropdown, one section per feature.
  • A live token-budget meter shows what each enabled feature costs in prompt tokens — with a per-feature breakdown and toggles, and a green/amber/red verdict against your local model’s context window.
  • Per-stage Test buttons for the custom pipeline, so you can verify each server befo
  • Choose the active wake word, and pick voices per TTS backend (Piper voices, Voxtral voice library, or free-text for OpenAI-compatible servers).

:speaker_high_volume: Better audio feedback

  • The device now plays a welcome sound when it connects after pairing, so you know the link works.
  • If a reply fails mid-conversation (connection drop, service error) you now hear a short error sound instead of silence.
  • All spoken feedback clips re-recorded to be provider-neutral (they no longer assume you use OpenAI).

:stethoscope: Flows & diagnostics

  • New Flow triggers “Heard something” (with the transcript) and “Thinking” — han flows around the assistant.
  • Optional remote logging to your own syslog collector (e.g. VictoriaLogs) for troubleshooting.

:hammer_and_wrench: Reliability

Two full external code reviews were done on the app, with dozens of fixes landing in this release. The most noticeable ones:

  • The assistant now recovers automatically if the connection dies right as you wake it.
  • Speech recognition accuracy improved (dedicated transcription model), and replies are anchored to what was actually transcribed — fewer answers to a mis-heard question.
  • Fixed very short replies being dropped, reply audio playing out of order, reconnect stseveral rare crashes.
  • All outbound network calls (weather, geocoding, etc.) now have timeouts, and settings saves are transactional.

As always: feedback, bug reports and feature ideas are very welcome!

42

Hi, thanks for the app! I wanted to explain something I ran into and ask whether a change might be possible.

I have a ThirdReality Voice & Music speaker — an ESPHome-based device with the native API on port 6053. When I add it through your AI Voice Assistant app and choose “Find it on my network,” the scan runs and then ends with “Timeout after 30000ms” — it never finds the speaker.

After a lot of digging, the cause turned out to be discovery, not the device itself. My Homey Pro is on Wi-Fi only, and the speaker’s mDNS multicast isn’t reaching it — other clients on my network can see the device’s _esphomelib._tcp advertisement fine, but Homey doesn’t receive it. So the app’s network scan finds nothing, even though the speaker is fully reachable. As proof, I was able to add the exact same speaker without any issue in the ESPHome Controller app by simply typing in its IP address — it connected instantly over a direct connection, no discovery involved.

So my question is really whether it would be possible to add an option to enter the device’s IP address manually (with port, and an encryption key if needed), as a fallback for when mDNS discovery doesn’t work. That would let people on Wi-Fi-only Homeys, or on networks where multicast doesn’t propagate well, still add these speakers.

Totally understand if it’s not straightforward — just wanted to flag what’s happening and ask. Happy to help test on my end. Thanks!

43

Hi Bruno. Thank you for the feedback, adding a device by manually typing in the IP is a good idea. Will look into it :slight_smile: A question, do you use encryption on your device? My app doesn’t support that yet.

44

My device doesn’t use encryption.