[APP][Pro] Archive Insights - export all your Insights data to a NAS, FTP or web folder

Has anyone got the webdav to work with onedrive?
I’m trying with the URL https://d.docs.live.net/“MYCUSTOMID” where mycustomid is a uniqe id for all users.
But when trying, i get an error saying 401 unauthorized.

I get lots of this type of error. What exactly does it mean? [log] 2022-05-14 10:49:23 Insights data is corrupt and will be truncated to the first 2925 records for Heating Controller price_SEK.

I can’t get the ftp server on my router to work. Zyxel online modem.
Could you also save that zipped file locally on my android phone ?
And what does Google Drive run on could you link there ?
Would like to save a record from insights of temperatures every month. Automatically via a flow

Edit: I found this https://www.multcloud.com
i look of this is works.

@Gruijter
is there really no way to do it locally ? Ore with a onther app ?
I don’t have a nas and webdav or ftp cloud storage for free is almost impossible to find.
You can download Paper Trails local on the internet.
Insights on crome I can not donwload.
Homey it says that you must click the three dots well then he hits only the website and not the graphic chart. What can I do ?

This is what i see

Have you looked hard enough for free online services? E.g. this one?

1 Like

@Gruijter Thank you so not hard enough. :wink:
I’m going to see what it is.
The insights joke is resolved, I have to turn my phone to landscape mode and then I can donwload it.

1 Like

The DriveHQ works great. And the export :smiley:

1 Like

I seem to have the same problem. When exporting day or week it all works fine, 30 export zip files are created. However, when I try to export month, only 23 files are created. It seems the larger one are skipped, In my case: Fibaro, Plugwise Smile and some others.

I would like to see the log that you mentioned in your post. Could you explain how you access it? Simple Log and Paper trail are not the ones for this I think?

Edit: found it, choose “configure app” and then there is a tab with the logs. Missed it. It has been a while since I configured the app…

Same here. Timeouts for 8 of 29 apps, using v 3.0.3s:

[log] 2022-05-29 08:36:39 APIErrorTimeout: Timeout after 5000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:475:23) at async App.zipAppLogs (/app.js:824:22) at async App._exportApp (/app.js:863:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }
[log] 2022-05-29 08:36:34 getting logs related to com.fibaro
[log] 2022-05-29 08:36:23 APIErrorTimeout: Timeout after 5000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:475:23) at async App.zipAppLogs (/app.js:824:22) at async App._exportApp (/app.js:863:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }
[log] 2022-05-29 08:36:18 getting logs related to com.amazon.ring
[log] 2022-05-29 08:36:01 APIErrorTimeout: Timeout after 5000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:475:23) at async App.zipAppLogs (/app.js:824:22) at async App._exportApp (/app.js:863:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }
[log] 2022-05-29 08:35:33 getting logs related to nl.klikaanklikuit
[log] 2022-05-29 08:35:15 APIErrorTimeout: Timeout after 5000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:475:23) at async App.zipAppLogs (/app.js:824:22) at async App._exportApp (/app.js:863:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }
[log] 2022-05-29 08:35:07 getting logs related to com.plugwise.smile
[log] 2022-05-29 08:34:49 APIErrorTimeout: Timeout after 5000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:475:23) at async App.zipAppLogs (/app.js:824:22) at async App._exportApp (/app.js:863:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }
[log] 2022-05-29 08:34:43 getting logs related to it.diederik.solar

Tried the insights online with my.homy.app (insights.homy.app redirects to that location too) as you suggested. That works fine, however, when using the insights, you can only export one item at a time. So I can’t export all of Plugwise Smile at once. At least, I think so, please correct me if I am wrong.

I did try to export only “Plugwise smile, last month.” That does work. So, it works when selected solely, but doesnt work if it is part of the bigger selection “All apps”

I have a Homey early 2016. It might take such an old Homey more time for the bigger job?

Also, I noticed that the last log line states: “Finished all exports”. So, the process is finished. However. The memory and CPU of the app remains high:

Any Ideas?

After asking about this I received a ‘secret’ option (meaning it is undocumented) from Athom. It allows me to set the timeout to a higher number. I need to find time to implement and test it though, so please be patient.

1 Like

version 3.1.0 was just released as test: Archive Insights | Homey

  • Increase homey-api timeout to 60s.
  • Reduced CPU load.
  • Packages update: homey-api@1.5.30 archiver@5.3.1 webdav@4.10.0.

This should get rid of the timeout errors. Also I introduced extra waiting times to reduce the overall CPU load. Downside is that an export takes longer now :slight_smile:

@Molstie @Ambush can you please test this version and let me know if it works for you?

1 Like

Great improvement, now only 2 apps are timed out:

[log] 2022-05-30 06:35:47 APIErrorTimeout: Timeout after 60000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:476:23) at async App.zipAppLogs (/app.js:827:22) at async App._exportApp (/app.js:867:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }
[log] 2022-05-30 06:27:23 APIErrorTimeout: Timeout after 60000ms at Function.timeout (/node_modules/homey-api/lib/Util.js:48:26) at HomeyAPIApp.call (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2.js:359:28) at async Manager.value (/node_modules/homey-api/lib/HomeyAPI/HomeyAPIV2/Manager.js:261:24) at async App.getLogEntries (/app.js:476:23) at async App.zipAppLogs (/app.js:827:22) at async App._exportApp (/app.js:867:21) at async App.runQueue (/app.js:224:4) { statusCode: 408 }

The rest is doing OK. So, it seems that my Homey indeed needs more time than average / other users.

Ah super!
I will increase the timeout even more, but 1 minute is already very long… Maybe you should get a faster Homey Pro or remove some apps?

What is the system load doing during an export? I could build in more relax-time for Homey, but I am not sure if that makes sense, even more because the export duration would drastically increase.

What is the system load doing during an export?

Just performed an export again, the uses 98,5, mb during export. When finished, it drops to 44.5 mb. Still a lot actually.

System overall:

Maybe you should get a faster Homey Pro or remove some apps?

Yeah, a faster Homy is probably not far away. I hoped to postpone for a while. I am really using al the current apps.

I just released app version 3.1.1:

  • Increase homey-api timeout to 120s.
  • Added low CPU load option during export

The Low CPU Load option can be switched on from the app settings page. When switched to on all exports will be performed very slowly and with reduced file compression. The resulting zip files will be 10-15% larger. The export time is drastically increased since 2 seconds ‘relax-time’ will be added for every (device)log. In my own tests this has resulted in very low CPU load and also reduced memory usage.

So if you have a slow or fully loaded Homey, but have all the time in the world to do an export, switching on this option makes sense.

@Molstie could you test and see if this keeps your system load within acceptable limits?

version 3.1.2 is released. It fixes a nasty bug I introduced in 3.1.1

  • Decrease homey-api timeout to 90s.
  • Fixed lowCPU option crash.
  • Catch FTP connection errors.

Hello, I have problem with FTP. Test is OK but after zipping, there is a error in log.

Thank you for your work!

[log] 2022-06-01 06:26:53 TypeError: this.FTPClient.on is not a function at App.getFTPClient (/app.js:729:19) at App.saveFTP (/app.js:750:15) at App._exportApp (/app.js:880:16) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async App.runQueue (/app.js:225:4)
[log] 2022-06-01 06:26:53 error: TypeError: this.FTPClient.on is not a function at App.getFTPClient (/app.js:729:19) at App.saveFTP (/app.js:750:15) at App._exportApp (/app.js:880:16) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async App.runQueue (/app.js:225:4)
[log] 2022-06-01 06:26:53 26 files zipped, 126202 total bytes

Yeah, I already saw crash reports coming in :face_exhaling:
Will try to post a fix tonight.

Sorry for late reply but tested now with version 3.1.2 but cannot perform any export - FTP fails with same error as horakamr listed above. SMB doesn’t even connect - only get following error:

[log] 2022-06-01 07:26:48 STATUS_ACCESS_DENIED (0xC0000022) : {Access Denied} A process has requested access to an object but has not been granted those access rights.
[log] 2022-06-01 07:26:48 testing SMB settings from frontend

Can you try again with version 3.1.3? That should fix the FTP error.