[APP][PRO] Audit

I have created a new application which helps you create an audit trail from the actions which occur with in homey.

Tracks

Currently it tracks changes to

  1. User alarms, (create, delete, update, enable, disable, time)
  2. Application, (install, uninstall, restart, crash, update available, auto updating, enable, ready)
  3. Capability (sensors, alarms, doors, lights, everything)
  4. Device (create, delete, update, name, zone, available, ready)
    5.Flows, (create, delete, update, name, broken, enable, move)
  5. Logic (create, delete, update)
  6. User (create, delete, update, name, enable, present, sleep, verified, favourite devices, favourite flows)
  7. Weather (update, city, temperature, description, humidity, pressure)
  8. Zone (create, delete, update, active, child active, moved, number of active devices in it)
  9. More coming soon.

Output

These changes are tracked both as metadata as well as a human readable string, in can also audit the full current homey object as well as the previous one.

Bathroom Hue Sensor Motion alarm turned on

{
    "action": "update",
    "code": "alarm_motion",
    "context": {
        "runtime": {
            "column": 28,
            "file": "lib/watcher/capability.js",
            "function": "trigger",
            "line": 58,
            "method": "trigger",
            "type": "Capability"
        },
        "system": {
            "main_file": "",
            "pid": 36501
        }
    },
    "device": "Bathroom Hue Sensor",
    "dt": "2024-01-18T12:08:05.249Z",
    "entity": "capability",
    "fields": [
        "alarm_motion"
    ],
    "id": "708aee96-3957-4bc8-b3c1-351f0f411bfd",
    "level": "info",
    "message": "Bathroom Hue Sensor Motion alarm turned on",
    "name": "Motion alarm",
    "publish": true,
    "reason": "alarm_motion",
    "type": "alarm",
    "value": "true",
    "zone": "Bathroom"
}

Bathroom has became active

{
    "action": "update",
    "active": true,
    "dt": "2024-01-18T12:08:05.298Z",
    "entity": "zone",
    "fields": [
        "active",
        "activeOrigins"
    ],
    "id": "8a6928ea-26b2-4922-9ead-d20ee6695dd3",
    "level": "info",
    "message": "Bathroom has became active",
    "name": "Bathroom",
    "publish": true,
    "reason": "active",
    "zone": "bathroom"
}

The number of active devices Inside has increased to 2

{
    "action": "update",
    "active": true,
    "dt": "2024-01-18T12:08:05.437Z",
    "entity": "zone",
    "fields": [
        "activeOrigins"
    ],
    "id": "166e6d5f-7e1d-4387-ae02-ca188f031403",
    "level": "debug",
    "message": "The number of active devices Inside has increased to 2",
    "name": "Inside",
    "publish": true,
    "reason": "activeOrigins",
    "zone": "inside"
}

Transport

The audit information is provided back in multiple ways.

  1. The app creates flows for each of the audits.
  2. Simple format of data stored and accessible directly from the homey (zipped daily and kept for 14 days)
  3. Detailed information is stored and accessible directly from home (zipped daily and kept for 3 days)
  4. BetterStack (https://betterstack.com/) a free (or paid) service.
  5. SimpleLog app
  6. Email

Flows

This method is only useful as a proof of concept, as many flows will be disabled as they are triggered too often in a decent sized house.

Simple format files

Each file contains a time string the log level and message, each day the log file is zipped up and saved and any log files over 14 days deleted.

[2024-01-18 12:10:09.827] info: 	Lounge Front Multisensor Motion alarm turned off
[2024-01-18 12:10:09.839] debug: 	The number of active devices Lounge has decreased to 2
[2024-01-18 12:10:21.620] info: 	Bathroom Lightstrip Hue decreased from 100 to 13
[2024-01-18 12:10:21.641] info: 	Bathroom Lightstrip Color saturation decreased from 100 to 55
[2024-01-18 12:10:21.657] info: 	Bathroom Lightstrip Color temperature decreased from 0.62 to 0.61
[2024-01-18 12:11:56.751] info: 	Heater Humidity increased from 62% to 63%
[2024-01-18 12:12:35.070] info: 	Bathroom Mirror Downlight 1 Dim level decreased from 30% to 0%
[2024-01-18 12:12:35.082] info: 	Bathroom Mirror Downlight 2 Dim level decreased from 30% to 0%
[2024-01-18 12:12:35.114] info: 	Bathroom Lightstrip Dim level decreased from 30% to 0%
[2024-01-18 12:12:35.139] debug: 	đź“Ž Bathroom Mirror Downlights Dim level decreased from 30% to 15%
[2024-01-18 12:12:35.161] debug: 	đź“Ž Bathroom Mirror Downlights Dim level decreased from 15% to 0%
[2024-01-18 12:12:35.183] debug: 	đź”— Bathroom Mirror Light Dim level decreased from 30% to 15%
[2024-01-18 12:12:35.216] debug: 	đź”— Bathroom Mirror Light Dim level decreased from 15% to 0%
[2024-01-18 12:13:06.164] info: 	Bathroom Mirror Downlight 1 Turned off
[2024-01-18 12:13:06.182] info: 	Bathroom Mirror Downlight 2 Turned off
[2024-01-18 12:13:06.204] info: 	Bathroom Lightstrip Turned off
[2024-01-18 12:13:06.228] debug: 	đź“Ž Bathroom Mirror Downlights Turned off
[2024-01-18 12:13:06.255] debug: 	đź“Ž Bathroom Lights Turned off
[2024-01-18 12:13:10.344] error: 	Bedroom Bed Plug is no longer available [Rate Limited]
[2024-01-18 12:13:12.551] error: 	Bedroom Leah’s Blanket is no longer available [Rate Limited]
[2024-01-18 12:13:17.504] debug: 	Hue Plug Power increased from 1.73W to 1.85W
[2024-01-18 12:14:59.390] info: 	Heater Humidity decreased from 63% to 62%
[2024-01-18 12:15:33.845] info: 	Kitchen Bench Plant Temperature decreased from 21.8°C to 21.7°C
[2024-01-18 12:15:33.936] info: 	Kitchen Bench Plant Luminance increased from 87lx to 403lx
[2024-01-18 12:15:33.978] info: 	Kitchen Bench Plant Nutrition increased from 153µS/cm to 156µS/cm
[2024-01-18 12:15:34.005] info: 	Kitchen Bench Plant Moisture decreased from 17% to 15%
[2024-01-18 12:15:34.753] info: 	Kitchen Bench Plant has been updated
[2024-01-18 12:18:07.241] info: 	Heater Temperature increased from 21°C to 21.5°C

Detailed format files

Again each day the log is zipped up and stored (in plain text during the day off), and after 3 days the log file is deleted.

2024-01-18 13:00:00.120] info: {
  "id": "7d9e5b06-0584-49bf-a5d7-887ecb42bd66",
  "name": "state",
  "action": "update",
  "entity": "logic",
  "fields": [
    "value"
  ],
  "level": "info",
  "message": "state has been updated from \"night\" to \"late\"",
  "publish": true,
  "reason": "value",
  "type": "string",
  "value": "late",
  "label": "audit"
}
[2024-01-18 13:00:00.128] crit: {
  "message": "Log file /userdata/simple-2024-01-18-13.log created",
  "level": "crit",
  "label": "audit"
}
[2024-01-18 13:00:00.131] crit: {
  "message": "Log file rotating from /userdata/simple-2024-01-18-12.log to /userdata/simple-2024-01-18-13.log",
  "level": "crit",
  "label": "audit"
}
[2024-01-18 13:00:00.134] crit: {
  "message": "Log file /userdata/detailed-2024-01-18-13.log created",
  "level": "crit",
  "label": "audit"
}
[2024-01-18 13:00:00.137] crit: {
  "message": "Log file rotating from /userdata/detailed-2024-01-18-12.log to /userdata/detailed-2024-01-18-13.log",
  "level": "crit",
  "label": "audit"
}
[2024-01-18 13:00:00.170] debug: {
  "id": "c7c2fdac-98ef-4640-af00-f209a33536bf",
  "name": "Turned on",
  "action": "update",
  "entity": "capability",
  "fields": [
    "onoff"
  ],
  "level": "debug",
  "message": "đź”— Garden Lights Turned off",
  "publish": true,
  "reason": "onoff",
  "zone": "Garden",
  "code": "onoff",
  "type": "onoff",
  "value": "false",
  "device": "đź”— Garden Lights",
  "label": "audit"
}
[2024-01-18 13:00:00.188] crit: {
  "message": "Log file archived to /userdata/simple-2024-01-18-12.log.gz",
  "level": "crit",
  "label": "audit"
}
[2024-01-18 13:00:00.193] crit: {
  "message": "Log file archived to /userdata/detailed-2024-01-18-12.log.gz",
  "level": "crit",
  "label": "audit"
}
[2024-01-18 13:00:01.395] info: {
  "id": "0dfe8929-5ce7-43bc-a3fe-6963d43759d1",
  "name": "Dim level",
  "action": "update",
  "entity": "capability",
  "fields": [
    "dim"
  ],
  "level": "info",
  "message": "Kitchen Colour Downlight Dim level decreased from 56% to 10%",
  "publish": true,
  "reason": "dim",
  "zone": "Kitchen",
  "code": "dim",
  "type": "dim",
  "value": "10%",
  "device": "Kitchen Colour Downlight",
  "label": "audit"
}

Betterstack

By far the best method! Go to betterstack and create a new account, then just add the “id” to the app.
All your audit logs will be available directly with in betterstack,. The free account will keep the data for 3 days.

Betterstack is incredibly powerful, you can choose which fields to show :

You can can also use a query languge to limit what to show.

Each audit can be expanded out to show all the meta data

As well as the full details on the homey object

SimpleLog app

Working with simpleLog app to allow the audit to talk directly to it and have your audit information added directly to the app. This is currently blocked, but hopefully will be available in the near future.

Email

I may add this feature, no promises though

Levels

The app follows the standard simple log levels (to allow easy integration into the simpleLog homey app.

  1. Emergency (emerg): indicates that the system is unusable and requires immediate attention.
  2. Alert (alert): indicates that immediate action is necessary to resolve a critical issue.
  3. Critical (crit): signifies critical conditions in the program that demand intervention to prevent system failure.
  4. Error (error): indicates error conditions that impair some operation but are less severe than critical situations.
  5. Warning (warn): signifies potential issues that may lead to errors or unexpected behavior in the future if not addressed.
  6. Notice (notice): applies to normal but significant conditions that may require monitoring.
  7. Informational (info): includes messages that provide a record of the normal operation of the system.
  8. Debug (debug): intended for logging detailed information about the system for debugging purposes.

Fields

Every audit contains atleast the following information

    protected object: {
        id: string                      // The unique identifier of the entity
        name: string                    // The name of the entity, ie. 'My kitchen light', 'Weekend Alarm'
        action: string                  // The action occuring, create, update or delete
        entity: string,                 // The enity which has chnaged, ie. device, alarm, object 
        fields: string[]                // Am array of all fields we have been altered since the last action
        reason: string,                 // An OPINIONATED reason as to why the event is being published, which field changed.
        message: string                 // Human readable message of what has occured
        level: string                   // the level of the audit log (debug, info, notice, warning, error, critical, alert, emergency)
        publish: boolean                // Whether or not we should ignore the objects alteration
    }

Notes

This is all a work in progress about to go to athom for approval, early days.

17 Likes

@Jamie this is cool and really a very basic feature athom itself is lacking on a easy to pull audit log on what happened when.
On most devices currently i can only pull on/off when/who/what flow

Exactly what I was thinking!
Very interested to try & test @Jamie
Good plan to integrate with Simple Log, I hope it will be possible.

Betterstack looks cool as well.

Sorry just need to create a settings page and then ill submit.

3 Likes

Hi Jamie,looking forward to it, looks very interesting project :slight_smile:

Cool stuff!!!
I extected that something like this would be available as a basic function.

Hi,

This app is looking quite advanced!

I would like for a starter to get notified in the timeline/push notifications when something happens to Homey.

-Disconnects from Wifi (time + date)
-Network request error (time + date)
-Low on free memory (time + date)

Is this possible with your app?

Any chance for us to test it (please:)?

Do you have any indication as to when this will be available to use?

I’m sorry, the app is up and running and doing a good job. But I simply have not gotten around to adding the settings page for the application so that others can use it.

For context, while adding a “device” setting is very straight forward - in order to add aN “application setting” Homey doesnt have a template and you have to manually create it (in html, css and javascript) then manually save each of your settings. Its actually a pain and is why all the app settings look different.

@Tirrazo some of that information is available via other apps, I was actually using one of them - but a bug in the app/homey sent through multiple updates per second - burning through my free quota of better stack. Ill chase up and see if it has been fixed.

1 Like

No response from the app creator (sysInternals) or Athom sorry …

To give context this bug in sysInternalsArhom, caused my data use to go up from 10mb a day to 600mb …. Per day in log messages.

Was pretty nasty, so I’ve disabled the app. If you want this feature reach out to the app creator. In the mean time I’m going to have to write some code to ignore that specific app.

1 Like

Jamie do you plan or do you have source code available somewhere or given portion of code, which is responsible ? That could help to understand what changed/happened in this version - they implemented mostly cache for rrd graphs and one more optimization, I wish, I would remember - so I don’t see how your app could be influenced.
Also sysinternals app hasn’t been updated for some time, so doesn’t seems to be related.

On the others side, you have have discovered a bug, which might be worth to address.