HTTP POST card with authentication and lot's of confusion

Questions & Help
1

I’m trying to create a way to block various services (YouTube / TikTok) some of my devices during a set of time. I’m using Adguard as ad blocker and this allows to communicatie through OpenAPI to this platform. If I add a device to Adguard and select the appropriate services, this is exactly what I would like (blocking internet all together results in switching to 4G for the specific devices and doesn’t help me).

Now I’m using the HTTP flow cards app (HTTP request flowkaarten App voor Homey | Homey) to try and achieve this. I just can’t seem to find out how to do the authentication. I need to do a ‘basic auth’.

Besides this, i’m running into some issues which are directly related to my knowledge.

  1. I can add ‘manually’ a new device to addguard. with the POST command:

URL: https://adguard.hoogenbosch.cloud/control/clients/update

{ “name”: “iphonevnMartijn”,“data”: { “name”:“iphonevnMartjin”, “blocked_services”:[“9gag”,“amazon”,“dailymotion”,“discord”,“disneyplus”,“ebay”,“epic_games”,“facebook”,“hulu”,“imgur”,“instagram”,“mail_ru”,“netflix”,“ok”,“origin”,“pinterest”,“qq”,“reddit”,“skype”,“snapchat”,“spotify”,“steam”,“telegram”,“tiktok”,“tinder”,“twitch”,“twitter”,“viber”,“vimeo”,“vk”,“wechat”,“weibo”,“whatsapp”,“youtube”], “ids”:[“192.168.0.125”], “tags”:[“user_child”], “upstreams”:, “filtering_enabled”:true, “parental_enabled”:false, “safebrowsing_enabled”:false, “safesearch_enabled”:false, “use_global_blocked_services”:false, “use_global_settings”:false} } }’

I am able to update this client. Unfortunately I can do this only once. The second time it says it cannot find the client. Btw; i’m doing this with the YARC client, because this can do authentication.

  1. I cannot understand how to ‘add’ a new client. I changed the url by replacing ‘update’ with ‘add’, but it asks for an ID, don’t know where to fill this information.

So bascially multiple questions.

I’m pulling my information from https://github.com/AdguardTeam/AdGuardHome/tree/master/openapi

2

hummmm… nobody to help me out :frowning:

Any pointers as where to look to get this done?

I can now add a client but don’t seem to get how to authenticate with the app in homey.

@robertklep you are a smart guy, by any chance know how to do this? So basically using the HTTP app to send:

{
  "method": "POST",
  "transformRequest": [
    null
  ],
  "transformResponse": [
    null
  ],
  "jsonpCallbackParam": "callback",
  "url": "https://adguard.hoogenbosch.cloud/control/clients/delete",
  "headers": {
    "Authorization": "Basic sdfsdfsdfsfsdOlsdfs2eewefwt",
    "Accept": "application/json, text/plain, */*",
    "Content-Type": "application/json;charset=utf-8"
  },
  "data": "{ \"name\": \"iphonevnMartijn\" }",
  "timeout": {}
}
3

For basic authentication, you could try adding the credentials in the URL:

https://USERNAME:PASSWORD@adguard.hoogenbosch.cloud/control/clients/delete

Make sure to URL-encode any special characters in both USERNAME and PASSWORD.

4

This indeed seems to work, at least in the YARC! client I’m using. From within Homey i’m getting Unexpected token F in JSON at position 0

image

When I change the URL to use the plain http and IP it has an unexpected token E instead of F, is there a location I can find out what these unexpected tokens mean?

5

The “F” is probably caused by the word “Forbidden” in the response (which the HTTP app is trying to parse as JSON, and failing). The “E” might be the word “Error” or something. It’s basically the first letter of the response.

No idea why Adguard is returning a Forbidden, apart from possibly username/password not being correct(ly encoded).

6

using the same information as used by YARC client, so username / passw should be correct. Maybe this route is impossible…

Any thoughts on doing the same in HA?

7

It could be that the server requires particular headers that aren’t sent along. It might be possible to get it working with Homeyscript. Here’s a start:

const auth     = Buffer.from(USERNAME + ":" + PASSWORD).toString('base64');
const response = await fetch('https://adguard.hoogenbosch.cloud/control/clients/delete', {
  method: 'POST',
  headers: {
    "Authorization": `Basic ${auth}`,
    "Accept":        "application/json, text/plain, */*",
    "Content-Type":  "application/json; charset=utf-8"
  },
  body: JSON.stringify({ name : 'iphonevnMartijn' }),
}).then(res => res.json()).catch(err => console.error(err));
1 Like
8 
FetchError: invalid json response body at https://adguard.hoogenbosch.cloud/control/clients/delete reason: Unexpected token F in JSON at position 0
    at /node_modules/node-fetch/lib/index.js:272:32
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async AddiphoneMartijn.js:1:18
    at async HomeyScriptApp.runScript (/app.js:313:22)
    at async Object.runScript (/api.js:24:18) {
  message: 'invalid json response body at https://adguard.hoogenbosch.cloud/control/clients/delete reason: Unexpected token F in JSON at position 0',
  type: 'invalid-json'
}

Wat ik ook doe, hoe ik ook quotes zet, hij blijft dit terug geven.

Hoe kan het dat die YARC client het wel gewoon doet? Doet die dan toch wat ‘bijzonders’ hiermee?

9

Heb je toevallig ergens curl geïnstalleerd?

curl --verbose -H'content-type: application/json' -d '{"name":"iphonevnMartijn"}' https://USERNAME:PASSWORD@adguard.hoogenbosch.cloud/control/clients/delete

En kijken wat precies de response is.

10 
* Connected to adguard.hoogenbosch.cloud (192.168.20.69) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=adguard.hoogenbosch.cloud
*  start date: Dec 22 19:06:11 2021 GMT
*  expire date: Mar 22 19:06:10 2022 GMT
*  subjectAltName: host "adguard.hoogenbosch.cloud" matched cert's "adguard.hoogenbosch.cloud"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5578655f90)
> POST /control/clients/delete HTTP/2
> Host: adguard.hoogenbosch.cloud
> user-agent: curl/7.74.0
> accept: */*
> content-type: application/json
> content-length: 26
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 403
< access-control-allow-origin: http://adguard.hoogenbosch.cloud
< vary: Origin
< content-type: text/plain; charset=utf-8
< content-length: 9
< date: Wed, 09 Feb 2022 12:53:55 GMT
<
* Connection #0 to host adguard.hoogenbosch.cloud left intact
11

Ik was vergeten username/password in de URL op te nemen. En --http1.1 toevoegen zodat er geen HTTP/2 gebruikt wordt.

12 
* upload completely sent off: 26 out of 26 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Access-Control-Allow-Origin: http://adguard.hoogenbosch.cloud
< Vary: Origin
< Date: Wed, 09 Feb 2022 13:02:44 GMT
< Content-Length: 9
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host adguard.hoogenbosch.cloud left intact

I guess there is the forbidden

why does it try to connect over http…

13

I don’t see authorization data being sent, perhaps curl won’t do that with HTTPS.

Never mind, you’re not showing the request?

14 
*   Trying 192.168.20.69:443...
* Connected to adguard.hoogenbosch.cloud (192.168.20.69) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=adguard.hoogenbosch.cloud
*  start date: Dec 22 19:06:11 2021 GMT
*  expire date: Mar 22 19:06:10 2022 GMT
*  subjectAltName: host "adguard.hoogenbosch.cloud" matched cert's "adguard.hoogenbosch.cloud"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Server auth using Basic with user 'mhoogenbosch'
> POST /control/clients/delete HTTP/1.1
> Host: adguard.hoogenbosch.cloud
> Authorization: Basic basdfasdfasdfszNDUt
> User-Agent: curl/7.74.0
> Accept: */*
> content-type: application/json
> Content-Length: 26
>
* upload completely sent off: 26 out of 26 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Access-Control-Allow-Origin: http://adguard.hoogenbosch.cloud
< Vary: Origin
< Date: Wed, 09 Feb 2022 13:09:57 GMT
< Content-Length: 9
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host adguard.hoogenbosch.cloud left intact
15

That all looks fine. Perhaps the server requires a very specific set of headers, so try adding the ones that the YARC client sends verbatim (using -H) to see if there’s a magic combination. And/or, if possible, check AdGuard log files for clues why it’s failing.

16

curl --verbose --http1.1 -H'Accept: application/json, text/plain, */*' 'content-type: application/json;charset=utf-8' -d '{"name":"iphonevnMartijn"}' https://user:pass@adguard.hoogenbosch.cloud/control/clients/delete

Results in:

curl: (3) URL using bad/illegal format or missing URL
*   Trying 192.168.20.69:443...
* Connected to adguard.hoogenbosch.cloud (192.168.20.69) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=adguard.hoogenbosch.cloud
*  start date: Dec 22 19:06:11 2021 GMT
*  expire date: Mar 22 19:06:10 2022 GMT
*  subjectAltName: host "adguard.hoogenbosch.cloud" matched cert's "adguard.hoogenbosch.cloud"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Server auth using Basic with user 'mhoogenbosch'
> POST /control/clients/delete HTTP/1.1
> Host: adguard.hoogenbosch.cloud
> Authorization: Basic sdfsdfsdf
> User-Agent: curl/7.74.0
> Accept: application/json, text/plain, */*
> Content-Length: 26
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 26 out of 26 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Access-Control-Allow-Origin: http://adguard.hoogenbosch.cloud
< Vary: Origin
< Date: Wed, 09 Feb 2022 13:21:19 GMT
< Content-Length: 9
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host adguard.hoogenbosch.cloud left intact

There are no more headers in YARC, this is what it sends:

{
  "method": "POST",
  "transformRequest": [
    null
  ],
  "transformResponse": [
    null
  ],
  "jsonpCallbackParam": "callback",
  "url": "https://adguard.hoogenbosch.cloud/control/clients/delete",
  "headers": {
    "Authorization": "Basic dfadfadfsdf",
    "Accept": "application/json, text/plain, */*",
    "Content-Type": "application/json;charset=utf-8"
  },
  "data": "{\"name\": \"iphonevnMartijn\"}",
  "timeout": {}
}
17

You’re not sending the Content-Type header. And make sure you add the headers verbatim, including the header name case (so Content-Type and not content-type). It shouldn’t matter (HTTP header names are case insensitive) but there are a lot of crappy HTTP servers around.

18 
* Connected to adguard.hoogenbosch.cloud (192.168.20.69) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_CHACHA20_POLY1305_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=adguard.hoogenbosch.cloud
*  start date: Dec 22 19:06:11 2021 GMT
*  expire date: Mar 22 19:06:10 2022 GMT
*  subjectAltName: host "adguard.hoogenbosch.cloud" matched cert's "adguard.hoogenbosch.cloud"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Server auth using Basic with user 'mhoogenbosch'
> POST /control/clients/delete HTTP/1.1
> Host: adguard.hoogenbosch.cloud
> Authorization: Basic sdfsfd
> User-Agent: curl/7.74.0
> Accept: application/json, text/plain, */*
> Content-Type: application/json;charset=utf-8
> Content-Length: 26
>
* upload completely sent off: 26 out of 26 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Access-Control-Allow-Origin: http://adguard.hoogenbosch.cloud
< Vary: Origin
< Date: Wed, 09 Feb 2022 13:42:30 GMT
< Content-Length: 9
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host adguard.hoogenbosch.cloud left intact

with

curl --verbose --http1.1 -H'Accept: application/json, text/plain, */*' -H'Content-Type: application/json;charset=utf-8' -d '{"name":"iphonevnMartijn"}' https://user:pass@adguard.hoogenbosch.cloud/control/clients/delete

I must say, it is very persistent.

19

Formally, I think there should be a space after the semicolon in the Content-Type header, but I doubt it’s the reason why it’s failing.

Given that you keep trying to delete a client, and it works with YARC, couldn’t the issue be that you can’t delete an already-deleted resource?

20

with space, as you suspected, no change. And indeed that could have been the case. With YARC if I do the removal i get a 200 response. I’m getting a 400 response when the client isn’t there anymore.

So yes you are right, if it’s not there, i’d expect a 400 message. But no 403 forbidden.

Can’t it be the formatting? I’m seeing / signs in the YARC code?