I failed developing a Google Assistant app

DanielvanBreda · April 5, 2024, 9:28pm

Hi all,

I spent two days trying to build a Google Assistant app, to use Homey to send a text command to GA.

I got a bit frustrated no being able to let my Hubs and Nest play Spotify music, so i figured i could make a app that could send a textQuery (Play album X on cast device Y) with this library:

But i couldn’t get the oAuth to work because of my lacky experience with NodeJS. Maybe someone else has the same need for a “missing link” app like this, so thats why i’m sharing my frustrated development experience

I figured it shouldn’t be to hard for a more experienced developer to make this work:

Cheers, Daniel

RonnyW · April 6, 2024, 6:52am

Hi. just linking to the other similar post.
Use this documentation and npm package from Athom to get all oAuth logic for free

DanielvanBreda · April 9, 2024, 12:55pm

Hi @RonnyW,

Thanks for your reply, i did try those links but i couldn’t get it to work for Google I hope that someone would check out my git repository and see what i’m doing wrong:

I think i’m messing up in the driver.ts file:

github.com

thadrax/nl.danielvanbreda.gaquery/blob/main/drivers/google-assistant-driver/driver.ts

import Homey from 'homey';

const { OAuth2Driver } = require('homey-oauth2app');

module.exports = class MyBrandDriver extends OAuth2Driver {

  async onOAuth2Init() {
    // Register Flow Cards etc.
  }

  async onPairListDevices({ oAuth2Client }) {

    const settings = this.homey.settings;
    const CLIENT_ID = settings.get('clientid');
		const CLIENT_SECRET = settings.get('clientsecret');
		const TOKEN = settings.get('token');

    const SCOPES = [ 'https://www.googleapis.com/auth/userinfo.profile','https://www.googleapis.com/auth/userinfo.email' ];
    const REDIRECT_URI = 'https://callback.athom.com/oauth2/callback'; // Default: 'https://callback.athom.com/oauth2/callback'

This file has been truncated. show original

RonnyW · April 9, 2024, 2:57pm

Where did you get an error or when?
Did you debug onto the auth/code exchange functions of the oAuthClient?
Is teh oAuth popup shown? Can you proceed?

What I’ve seen:

static API_URL = 'https://accounts.google.com/o/oauth2/auth?response_type=code&';
API_URL is the URL to call API services with your bearer token. It’s not for oAuth. I think this should be the Server for your assistant API.

An in your oAuthClient you call:

await this.post({
      path: '/api/1/...',
      json:{}
    });

and the API_URL is used as host.

DanielvanBreda · April 15, 2024, 8:56am

Hi @RonnyW,

I get an popup, but i get this response:

When i copy the response from google from the console, i get this html:

But it actually receives these parameters:

PARAMS {
client_id: ‘361248910627-sbt2ehglhgptp0ksjqfgk1cv32f1162l.apps.googleusercontent.com’,
redirect_uri: ‘https://callback.athom.com/oauth2/callback’,
scope: ‘https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email’
}

RonnyW · April 15, 2024, 10:19am

You defined the redirect URL in your MyBrandOAuth2Client.ts. Looks ok.
But your driver raises an execption fot “things”. The “things” function is only an example where you can call your API.

I’m not sure what you are doing here: nl.danielvanbreda.gaquery/drivers/google-assistant-driver/driver.ts at main · thadrax/nl.danielvanbreda.gaquery · GitHub

This list devices function is calles after oAuth process (oAuth is valid and you got your token) to read available devices you can add to Homey. I don’t know what “devices” you have for this API.

You oAuthClient redefinition should provide functions to read the API. The oAuthClient has access to the token etc.
Example: homey.tesla/lib/TeslaOAuth2Client.js at main · RonnyWinkler/homey.tesla · GitHub
In your driver, just call the method of your oAuthClient like this: homey.tesla/drivers/car/driver.js at main · RonnyWinkler/homey.tesla · GitHub

DanielvanBreda · April 15, 2024, 9:01pm

Hi @RonnyW

Thank you so much for all your help, i’m beginning to understand more and more of how this works thanks to you. NodeJS is a bit out of my comfort zone, but its nice to learn.

I now got the oauth authorization working, now i’m gonna figure out how to call that authorization in my app.ts or how i can move my query flow to the virtual Google Assistant device.

github.com

thadrax/nl.danielvanbreda.gaquery/blob/main/app.ts

'use strict';

import Homey from 'homey';

const { OAuth2App } = require('homey-oauth2app');
const MyBrandOAuth2Client = require('./lib/MyBrandOAuth2Client');
const path = require('path');
const GoogleAssistant = require('google-assistant');


var CLIENT_ID = "";
var CLIENT_SECRET = "";
var TOKEN = "";

module.exports = class MyBrandApp extends OAuth2App {

  static OAUTH2_CLIENT = MyBrandOAuth2Client; // Default: OAuth2Client
  static OAUTH2_DEBUG = true; // Default: false
  static OAUTH2_MULTI_SESSION = false; // Default: false
  static OAUTH2_DRIVERS = [ 'google-assistant-driver' ]; // Default: all drivers

This file has been truncated. show original

Thanks once more

RonnyW · April 16, 2024, 5:16am

You’re welcome
And I know the difficulty to learn a nuw language, the SKD and oAuth on top

DanielvanBreda · April 17, 2024, 9:33am

Hi @RonnyW

Would you happen to know how to fix this grpc error:

github.com

thadrax/nl.danielvanbreda.gaquery/blob/main/drivers/google-assistant-driver/driver.ts

import Homey from 'homey';
import { json } from 'stream/consumers';

const { OAuth2Driver } = require('homey-oauth2app');
const path = require('path');
const GoogleAssistant = require('google-assistant');

module.exports = class MyBrandDriver extends OAuth2Driver {

  async onOAuth2Init() {
    // Register Flow Cards etc.

    await super.onOAuth2Init();

  }

  async onPairListDevices({ oAuth2Client }) {

    const settings = this.homey.settings;
    const CLIENT_ID = settings.get('clientid');

This file has been truncated. show original

grpc is deprecated and now grpc-js is used. But google-assistant uses grpc, and now throws this error:

“googleCredentials.getRequestMetadata is not a function”

Thank you

robertklep · April 17, 2024, 9:38am

Does it?

DanielvanBreda · April 17, 2024, 10:15am

uhm no it doesn’t
i’m sorry for the confusion, i did not read well.

This is my terminal error when i pass MyBrandOAuth2Client as oauth2Client. it contains:

MyBrandOAuth2Client {
    _events: [Object: null prototype] {
      log: [Function (anonymous)],
      error: [Function (anonymous)],
      debug: [Function (anonymous)],
      save: [Function (anonymous)],
      destroy: [Function (anonymous)]
    },
    _eventsCount: 5,
    _maxListeners: undefined,
    homey: Homey {
      _events: [Object: null prototype],
      _eventsCount: 4,
      _maxListeners: undefined,
      dir: '/app',
      tmpdir: '/app/tmp',
      version: '10.3.4',
      platform: 'local',
      platformVersion: 2,
      env: {},
      manifest: [Object],
      app: [MyBrandApp],
      __readyResolve: [Function (anonymous)],
      __readyPromise: [Promise],
      _timers: Set(0) {},
      _destroyed: false,
      apps: [ManagerApps],
      arp: [ManagerArp],
      audio: [ManagerAudio],
      ble: [ManagerBLE],
      cloud: [ManagerCloud],
      clock: [ManagerClock],
      drivers: [ManagerDrivers],
      discovery: [ManagerDiscovery],
      flow: [ManagerFlow],
      geolocation: [ManagerGeolocation],
      i18n: [ManagerI18n],
      images: [ManagerImages],
      insights: [ManagerInsights],
      ledring: [ManagerLedring],
      nfc: [ManagerNFC],
      notifications: [ManagerNotifications],
      rf: [ManagerRF],
      settings: [ManagerSettings],
      speechInput: [ManagerSpeechInput],
      speechOutput: [ManagerSpeechOutput],
      zigbee: [ManagerZigBee],
      zwave: [ManagerZwave],
      api: [ManagerApi],
      __strings: [Object],
      __: [Function: __],
      [Symbol(kCapture)]: false
    },
    _tokenConstructor: [class OAuth2Token],
    _clientId: '???',
    _clientSecret: '???',
    _apiUrl: 'https://accounts.google.com/o/oauth2/auth?response_type=code&',
    _tokenUrl: 'https://oauth2.googleapis.com/token',
    _authorizationUrl: 'https://accounts.google.com/o/oauth2/auth',
    _redirectUrl: 'https://callback.athom.com/oauth2/callback',
    _scopes: [
      'https://www.googleapis.com/auth/userinfo.profile',
      'https://www.googleapis.com/auth/userinfo.email'
    ],
    _token: OAuth2Token {
      access_token: '???',
      refresh_token: null,
      token_type: 'Bearer',
      expires_in: 3599
    },
    _title: null,
    [Symbol(kCapture)]: false
  }
}

and throws this error:

in failed with error: googleCredentials.getRequestMetadata is not a function
    at callErrorFromStatus (/app/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
    at Object.onReceiveStatus (/app/node_modules/@grpc/grpc-js/build/src/client.js:420:73)
    at Object.onReceiveStatus (/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
    at /app/node_modules/@grpc/grpc-js/build/src/resolving-call.js:129:78
    at processTicksAndRejections (node:internal/process/task_queues:78:11)
for call at
    at ServiceClientImpl.makeBidiStreamRequest (/app/node_modules/@grpc/grpc-js/build/src/client.js:404:32)
    at ServiceClientImpl.converse (/app/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
    at new Conversation (/app/node_modules/google-assistant/components/conversation.js:136:34)
    at GoogleAssistant.start (/app/node_modules/google-assistant/index.js:49:26)
    at GoogleAssistant.<anonymous> (/app/drivers/google-assistant-driver/driver.js:99:42)
    at GoogleAssistant.emit (node:events:513:28)
    at assistantReady (/app/node_modules/google-assistant/index.js:22:12)
    at processTicksAndRejections (node:internal/process/task_queues:78:11) {

robertklep · April 17, 2024, 10:19am

Also: Release v2.0.0 · googleapis/google-auth-library-nodejs · GitHub

DanielvanBreda · April 17, 2024, 3:38pm

Ah so should i try to update the google-auth-library dependency?
I tried that this way, but the google-auth-library dependency within google-assistant remains ^7.0.4 instead of ^9.8.0

github.com

thadrax/nl.danielvanbreda.gaquery/blob/main/package.json

{
  "name": "nl.danielvanbreda.gaquery",
  "version": "1.0.0",
  "main": "app.ts",
  "scripts": {
    "preinstall": "npx npm-force-resolutions"
  },
  "devDependencies": {
    "@tsconfig/node16": "^16.1.3",
    "@types/homey": "npm:homey-apps-sdk-v3-types@^0.3.5",
    "@types/node": "^20.12.6",
    "eslint": "^7.32.0",
    "eslint-config-athom": "^3.1.3",
    "npm-force-resolutions": "^0.0.10",
    "typescript": "^5.4.4"
  },
  "dependencies": {
    "audit": "^0.0.6",
    "fix": "^0.0.3",
    "google-assistant": "^0.7.0",

This file has been truncated. show original

robertklep · April 17, 2024, 5:28pm

I think you should get rid of the override, since it clearly doesn’t work. If you can’t get google-assistant to work you may have to consider looking for an alternative, it’s been a couple of years since it was last updated so it may be deprecated itself.

Topic		Replies	Views
[APP][Pro] OpenAI ChatGPT Apps app , homey-pro	79	5623	February 17, 2024
Google assistant JWT token is invalid Questions & Help	0	463	April 5, 2019
Home Assistant App? Questions & Help	13	1020	June 29, 2021
Homey Bridge and google assistant Questions & Help flow , homey-bridge	8	139	October 19, 2023
Problems connecting Homey to Google Nest Hub Gen.2 Devices app	7	809	April 26, 2024

Related Topics