A different FTP server. E.g. this one: About DriveHQ - The leading Enterprise Cloud IT service provider
1 Like
The idea is that I want to move from my online storage (TransIP STACK via WEBDAV) to my own Raspberry Pi.
@Gruijter Thanks for the app at first!!
I noticed that the time in the exports is an hour behind.
I turn on the lights at 2:30 PM, in the export it says 1:30 PM.
I export at 11:59 PM to my nas, the folder has the text 10:59 PM
Is that something that can be fixed with an update?
Nothing needs to be fixed. Timestamp is in UTC, not local timezone.
ahh ok!
the dutch summer and wintertime 
Thanks
1 Like
SMB does not work for me. With FTP the app says itâs ok to save the settings so I assume the settings are correct than.
The log says all items are exported but I donât see the files in the folder where it should be.
Any idea why I donât see the files?
Thanks.
Edit, the exported files were places in the music folder. No matter what link I put in the FTP folder, it is always placed in the music folder.
How do I get it in the folder I want it to be?
Hi @Gruijter ,
Mary I ask you, where the Exports go, in case there is no target configured?
I just reinstalled your app after some time and forget to configure it again - surprisingly it starts exporting and âsuccessfullyâ finished⌠but I wonder where the reports are now 
on Homey /userdata. But after export that immediately gets deleted.
1 Like
Hey @Gruijter. Just checking: it seems Archive Insight is not working on the Homey pro early 2023?
- FTE connection is OK (âOK! sla de instellingen opâ)
- When selecting he tab âExportsâ, the dropdown âAPPâ only shows âalle appsâ, no other items.
- When selecting âExporteer nuâ, is says âAre you sure⌠etcâ and then âExport is begonnenâ
- The Log shows âExport que is flushedâ
- BTW: When I select âDeleteâ in the âLogsâ tab, the log apears to be deleted. However, when I leave that tap and return to it, the log is still there.
After that: noting happens anymore. Maybe itâs a know thing that I missed. I did not yet report this to Athom, first wanted to check.
ps: it does still work on the old Homey, it appears to have someting to do wirth the homey pro early 2023
Correct. Working on HP2023 compatibility as we speak. @Arie_J_Godschalk already did most of the rewrite. I just need to debug some more and finish it.
1 Like
My version is working, the issue had two part to it:
The way homey-api references the uri and id, whicch is changed with logs/insights/devices etc.
The other part was that i had wrongly converted logs from the migration.
Both is solved in my current version for hp2023, but i temporarly have no hp2019 to test (not gonna reset it till i know for sure it the hp2023 works correct after my real migration yesterday, altho its looking good!).
Also please note, i did extensive testing: All logs that can be, are send trough FTP to my synology.
There i have a script that extracts all zip files, all devices and apps, all capabilities etc. and places it into excel files for further processing.
And with my latest version, everything worked as before. Meaning, afaik, everything was as it used to be (which i also confirmed by hand).
One change tho: default logic variable filenames do not have the prefix variable-.
But in the headers of logic variables, i did make another change.
Before, only the logic IDâs were shown (in filenames and the headers).
Now, in the headers, the Logic/Variable NAME is shown
1 Like
This will take more time than I hoped. Arieâs work on HP2023 compatibiliy is much appreciated, but it also broke HP2016/HP2019 compatibility. So I need to dig into the code and find/fix all these issues.
Sorry m8, i tried to keep it compatible, but itâs difficult without testing.
Perhaps, if you havenât found the issue in a few days, i can recheck the code!
What is the issue btw?
No logs, no apps in the list, wrong names?
Perhaps i can find it in the code if you give me some more information!
no worries, thatâs all in the game. But the changes between HP2023 and older versions are significant for logs via Homey-api. Maybe that is why Athom urges devs to use the new v3 Homey-api. It might be less work to rewrite to the new homey-api, in stead of trying to make the current version working on all homey types. It will at least be more future proof.
1 Like
For users who export this data to a Synology and want to use it.
I use this Bash script that i run as task on the Synology every hour.
Every hour i export all insights from everything from the last hour, and about 20 minutes later, i run the next bash script on the synology.
Here is the Bash script for putting details into CSV files for energie and wind and weather info.
All files are also relocated after extracting.
Data="/volume1/Homey/Insights/Data"
Root="/volume1/Homey/Insights/Exported"
Home="$(pwd)"
Purged=""
#Dir = "$(dirname "$0")"
cd "$Root" # "$(dirname "$0")"
subdircount=$(find $Root -maxdepth 1 -type d | wc -l)
if [[ "$subdircount" -eq 1 ]]
then
echo "!!!Bestanden niet gevonden. Controleer Homey Insights!!!"
exit -1
fi
echo "Looping through directories"
# cd $Dir
for folder in *; do
echo "FOUND DIR $folder"
cd "$Root"
if [ -d "$folder" ]; then #if directory
# $f is a directory
echo "going into $folder"
YEAR=${folder:0:4}
# echo "$YEAR"
cd "$folder"
for file in *.zip; do #if file
if [ -f "$file" ]; then
# $f is a directory
echo "================================================================================================"
echo "found $file"
7z x -y "${file}"
if [ $? -eq 0 ]; then
echo "Successfully unpacked; purging rar files"
#rm -f "${file}"
else
echo "Unable to unpack;"
fi
fi
done
for deviceFolder in *; do
if [ -d "$deviceFolder" ]; then
echo "found $deviceFolder"
# Energiemeter
find "$deviceFolder" -name 'meter_power.csv' -type f |sort | while read File; do
meterFolder="$Data/Energiemeter/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/meter_power.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/meter_power.csv"
#rm -f "$File"
# sed 1d "$File" | while read d; do
# echo $d
# done
# Dir="$(dirname "$File")"
# echo "" #$File"
done
# Weer
find "$deviceFolder" -name 'measure_gust_angle.csv' -type f |sort | while read File; do
meterFolder="$Data/Weer/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/measure_gust_angle.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/measure_gust_angle.csv"
done
find "$deviceFolder" -name 'measure_gust_strength.csv' -type f |sort | while read File; do
meterFolder="$Data/Weer/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/measure_gust_strength.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/measure_gust_strength.csv"
done
find "$deviceFolder" -name 'measure_wind_angle.csv' -type f |sort | while read File; do
meterFolder="$Data/Weer/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/measure_wind_angle.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/measure_wind_angle.csv"
done
find "$deviceFolder" -name 'measure_wind_strength.csv' -type f |sort | while read File; do
meterFolder="$Data/Weer/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/measure_wind_strength.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/measure_wind_strength.csv"
done
# Temperatuur
find "$deviceFolder" -name 'measure_temperature.csv' -type f |sort | while read File; do
meterFolder="$Data/Temperatuur/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/measure_temperature.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/measure_temperature.csv"
done
find "$deviceFolder" -name 'target_temperature.csv' -type f |sort | while read File; do
meterFolder="$Data/Temperatuur/$YEAR/$deviceFolder/"
mkdir -p "$meterFolder"
>> "$meterFolder/target_temperature.csv"
sed -i '1d' "$File"
cat "$File" >> "$meterFolder/target_temperature.csv"
done
rm -r "$deviceFolder"
# #cd "$deviceFolder"
# for file in $(dirname $Root)*; do #if file
# echo "asdasdsa$file"
# if [ -f "$file" ]; then
# # $f is a directory
# echo "================================================================================================"
# fi
# done
fi
done
movedfilecount=$(find $Data/Alle/$folder -maxdepth 1 -type f | wc -l)
if [[ "$movedfilecount" -gt 0 ]]
then
echo "!!!Bestanden al verplaatst!!!"
rm -r "$Root/$folder/"
else
mv -f "$Root/$folder/" "$Data/Alle"
fi
#COPY folder to DATA!
# find ${Root} '.zip' |sort | while read File; do
# Dir="$(dirname "$File")"
# # if [ "${Prevdir}" != "${Dir}" ]; then
# cd "${Dir}"
# echo "${File}"
# echo "================================================================================================"
# # Process the first rar file in this directory thanks to |sort
# 7z x -y "${File}"
# if [ $? -eq 0 ]; then
# echo "Successfully unpacked; purging rar files"
# # rm -f "${File}"
# else
# echo "Unable to unpack;"
# fi
# # fi
# # Prevdir=$Dir
# done
# cd $folder
# for folder in "$(dirname "$0")"/*; do
# if [ -d "$folder" ]; then
# # $f is a directory
# echo "going into $folder"
# cd $folder
# fi
# done
fi
done
exit
# zip
echo "Looking for zip files to unpack..."
# Prevdir=""
find ${Root} '*/*.zip' |sort | while read File; do
Dir="$(dirname "$File")"
# if [ "${Prevdir}" != "${Dir}" ]; then
cd "${Dir}"
echo ""
echo "= $(pwd)"
echo "${File}"
echo "================================================================================================"
# Process the first rar file in this directory thanks to |sort
7z x "${File}"
if [ $? -eq 0 ]; then
echo "Successfully unpacked; purging rar files"
rm -f "${File}"
# rm -f *.r[0-9][0-9]
# rm -f *.s[0-9][0-9]
# rm -f *.t[0-9][0-9]
else
echo "Unable to unpack; purging entire directory"
# Purged="${Purged}${Dir}\n"
# cd ..
# rm -rf "${Dir}"
fi
# fi
# Prevdir=$Dir
done
cd "${Home}"
BTW,
i found that when you set the Export last hour, i still get values older thatn the last hour.
Also, i hope there can come a âprevious hourâ option, which would, when run at for instance 14:05, give really all instights from 13:00 to 14:00.
Becuase right now, if the export takes a while, the last exported zip can have a starting point 5 minutes later then the first .zip file.
For analysys, this is not very handy 
I am gonna take a look at the new api right now, try to fix this issues.
@Gruijter
Changes i made:
-
Using the new homey-api 3.0.2. now
-
It works for me on the HP2023 and on the HP2019!
But, the homey-api doesnât handle the correct solution on HP2019!, so this cannot be rolled out yet!
Already created a message on slack! -
Before, for every app or exported entity, because of how the loops where, you keep reloading all apps, all devices, all logs, etc.
Now, it retrieves all information just once, then starts looping through the to be exported entities.
Saves a lott of resources and speeds it all up! -
You wait 10 seconds between each entity, to reduce overloading.
But with 45 apps, that 7.5 minutes waiting for nothing!
I have created a settings for it, by default still 10 seconds, but you can change that now.
When set to 0, the timeout will still be 1 millisecond, so the process will not be killing homey.
And for me, 0 seconds works fine and there is no crashing issue or anything.
So, perhaps the default should be 0, so people can increase it themself? -
There were some reloading of logs right after each other, i have changed that for performance.
-
Because it now goes way faster (if wanted), the resolution of one hour is less out of sync between entities.
-
Booleans (Yes/No) donât use the resolution. This creates a lott of double information, because all Booleans that are still saved in the device insights, are included, even if it falls outside the resolution.
I have changed this by manually ignore log-entries that fall outside the resolution given.
Thus: Booleans now also respect the given resolution.
Please note tho: resolution on HP2019 is based on local tike, but on the hp2023, its based on utc.
I have informed Athom about this. -
Saving the zip files is now await for, thus no timeout needed!
Goes way faster this way!
Exporting all stuff on my HP2019 took about 11-12 minutes before.
Now it does it in about 2-3 minutes!
But again: Before the homey-api is fixed for the HP2019 (and older i guess), this cannot be rolled out, because resolution is not handled at all!
But, the sourcecode seems fine and works correct with both HP2019 and HP2023!
2 Likes
