[APP][PRO] Audit

[Jamie]

I have created a new application which helps you create an audit trail from the actions which occur with in homey.

This application is now available on the app store using this link : Audit App for Homey | Homey

Tracks

Currently it tracks changes to

1. User alarms, (create, delete, update, enable, disable, time)
2. Application, (install, uninstall, restart, crash, update available, auto updating, enable, ready)
3. Capability (sensors, alarms, doors, lights, everything)
4. Device (create, delete, update, name, zone, available, ready)
   5.Flows, (create, delete, update, name, broken, enable, move)
5. Logic (create, delete, update)
6. User (create, delete, update, name, enable, present, sleep, verified, favourite devices, favourite flows)
7. Weather (update, city, temperature, description, humidity, pressure)
8. Zone (create, delete, update, active, child active, moved, number of active devices in it)
9. More coming soon.

Output

These changes are tracked both as metadata as well as a human readable string, in can also audit the full current homey object as well as the previous one.

Bathroom Hue Sensor Motion alarm turned on

{
    "action": "update",
    "code": "alarm_motion",
    "context": {
        "runtime": {
            "column": 28,
            "file": "lib/watcher/capability.js",
            "function": "trigger",
            "line": 58,
            "method": "trigger",
            "type": "Capability"
        },
        "system": {
            "main_file": "",
            "pid": 36501
        }
    },
    "device": "Bathroom Hue Sensor",
    "dt": "2024-01-18T12:08:05.249Z",
    "entity": "capability",
    "fields": [
        "alarm_motion"
    ],
    "id": "708aee96-3957-4bc8-b3c1-351f0f411bfd",
    "level": "info",
    "message": "Bathroom Hue Sensor Motion alarm turned on",
    "name": "Motion alarm",
    "publish": true,
    "reason": "alarm_motion",
    "type": "alarm",
    "value": "true",
    "zone": "Bathroom"
}

Bathroom has became active

notice: {
  "message": "Bedroom has became active",
  "level": "notice",
  "publish": true,
  "detail": {
    "zone": "bedroom",
    "active": true
  },
  "meta": {
    "id": "462416a2-d1ae-459a-8be9-81bbb682919e",
    "name": "Bedroom",
    "action": "update",
    "entity": "zone",
    "fields": [
      "active",
      "activeOrigins"
    ],
    "reason": "active",
    "timestamp": 1720138507949
  },
  "label": "audit"
}

The number of active devices Inside has increased to 2

debug: {
  "message": "The number of active devices in Upstairs has increased to 2",
  "level": "debug",
  "publish": true,
  "detail": {
    "zone": "upstairs",
    "active": true
  },
  "meta": {
    "id": "e02531f0-a951-45f4-8078-538f206a00ff",
    "name": "Upstairs",
    "action": "update",
    "entity": "zone",
    "fields": [
      "activeOrigins"
    ],
    "reason": "activeOrigins",
    "timestamp": 1720138507959
  },
  "label": "audit"
}

Transport

The audit information is provided back in multiple ways.

1. The app creates flows for each of the audits.
2. Simple format of data stored and accessible directly from the homey (zipped daily at 12 midnight UTC and kept for 14 days)
3. Detailed information is stored and accessible directly from home (zipped daily and kept for 3 days)
4. BetterStack (https://betterstack.com/) a free (or paid) service.
5. SimpleLog app (coming soon)
6. Email (coming soon)

Flows

This method is only useful as a proof of concept, as many flows will be disabled as they are triggered too often in a decent sized house.

Simple format files

Each file contains a time string the log level and message, each day the log file is zipped up and saved and any log files over 14 days deleted. You can view all of the files on homey from with in the application settings page. Or direction by going to :

Locally

http://{HOMEY_IP}/app/com.audit/userdata/yyyymmdd.message.log

http://192.168.0.100/app/com.audit/userdata/202331231.message.log

Globally

http://{HOMEY-IP-Dash}.homey.homeylocal.com/app/com.audit/userdata/yyyymmdd.message.log

http://192-168-0-100.homey.homeylocal.com/app/com.audit/userdata/202331231.message.log

[2024-01-18 12:10:09.839]notice: 	state has been updated from "day" to "afternoon"
[2024-01-18 12:10:21.620]crit: 	Log file rotating from /userdata/message.20240704.log to /userdata/message.20240705.log
[2024-01-18 12:10:21.641]crit: 	Log file /userdata/detail.20240705.log created
[2024-01-18 12:10:21.657]crit: 	Log file rotating from /userdata/detail.20240704.log to /userdata/detail.20240705.log
[2024-01-18 12:10:09.839] notice: 	Weekday Alarm! has been enabled
[2024-01-18 12:10:21.620] warning: 	Dimdown has been renamed to When Sunset Flow
[2024-01-18 12:10:21.641] debug: 	Lounge Thermostat Last seen seconds ago increased from 58 to 68
[2024-01-18 12:10:21.657] debug: 	Study Thermostat Last seen seconds ago increased from 5 to 15
[2024-01-18 12:11:56.751] crit: 	When Sunset Flow has been deleted
[2024-01-18 12:12:35.070] crit: 	Fireworks has been created
[2024-01-18 12:13:06.164] info: 	Bathroom Mirror Downlight 1 Turned off
[2024-01-18 12:13:06.182] info: 	Bathroom Mirror Downlight 2 Turned off
[2024-01-18 12:13:06.204] info: 	Bathroom Lightstrip Turned off
[2024-01-18 12:13:06.228] debug: 	📎 Bathroom Mirror Downlights Turned off
[2024-01-18 12:13:06.255] debug: 	📎 Bathroom Lights Turned off
[2024-01-18 12:13:10.344] error: 	Bedroom Bed Plug is no longer available [Rate Limited]
[2024-01-18 12:13:12.551] error: 	Bedroom Leah’s Blanket is no longer available [Rate Limited]
[2024-01-18 12:13:17.504] debug: 	Hue Plug Power increased from 1.73W to 1.85W
[2024-01-18 12:14:59.390] info: 	Heater Humidity decreased from 63% to 62%
[2024-01-18 12:15:33.845] info: 	Kitchen Bench Plant Temperature decreased from 21.8°C to 21.7°C
[2024-01-18 12:15:33.936] info: 	Kitchen Bench Plant Luminance increased from 87lx to 403lx
[2024-01-18 12:15:33.978] info: 	Kitchen Bench Plant Nutrition increased from 153µS/cm to 156µS/cm
[2024-01-18 12:15:34.005] info: 	Kitchen Bench Plant Moisture decreased from 17% to 15%
[2024-01-18 12:15:34.753] info: 	Kitchen Bench Plant has been updated
[2024-01-18 12:18:07.241] info: 	Heater Temperature increased from 21°C to 21.5°C

Detailed format files

Again each day the log is zipped up and stored (in plain text during the day off), and after 3 days the log file is deleted.

Locally

http://{HOMEY_IP}/app/com.audit/userdata/yyyymmdd.detailed.log

http://192.168.0.100/app/com.audit/userdata/202331231.detailed.log

Globally

http://{HOMEY-IP-Dash}.homey.homeylocal.com/app/com.audit/userdata/yyyymmdd.detailed.log

http://192-168-0-100.homey.homeylocal.com/app/com.audit/userdata/202331231.detailed.log

notice: {
  "message": "state has been updated from \"day\" to \"afternoon\"",
  "level": "notice",
  "publish": true,
  "detail": {
    "value": "afternoon",
    "type": "string"
  },
  "meta": {
    "id": "7d9e5b06-0584-49bf-a5d7-887ecb42bd66",
    "name": "state",
    "action": "update",
    "entity": "logic",
    "fields": [
      "value"
    ],
    "reason": "value",
    "timestamp": 1720155600016
  },
  "label": "audit"
}
 crit: {
  "message": "Log file rotating from /userdata/message.20240704.log to /userdata/message.20240705.log",
  "level": "crit",
  "label": "audit"
}
crit: {
  "message": "Log file /userdata/detail.20240705.log created",
  "level": "crit",
  "label": "audit"
}
crit: {
  "message": "Log file rotating from /userdata/detail.20240704.log to /userdata/detail.20240705.log",
  "level": "crit",
  "label": "audit"
}
notice: {
  "message": "Weekday Alarm! has been enabled",
  "level": "notice",
  "publish": true,
  "detail": {
    "time": "06:46",
    "enabled": false
  },
  "meta": {
    "id": "e246ccf3-1c94-4007-9536-13f387077980",
    "name": "Weekday Alarm!",
    "action": "update",
    "entity": "alarm",
    "fields": [
      "enabled"
    ],
    "reason": "enabled",
    "timestamp": 1720161944340
  },
  "label": "audit"
}

warning: {
  "message": "Dimdown has been renamed to When Sunset Flow",
  "level": "warning",
  "publish": true,
  "detail": {
    "enabled": true,
    "broken": false
  },
  "meta": {
    "id": "f3eb075b-7a8e-4dad-adae-57866c032cbc",
    "name": "When Sunset Flow",
    "action": "update",
    "entity": "flow",
    "fields": [
      "name"
    ],
    "reason": "name",
    "timestamp": 1720162023577
  },
  "label": "audit"
}
debug: {
  "message": "Lounge Thermostat Last seen seconds ago increased from 58 to 68",
  "level": "debug",
  "detail": {
    "code": "se_last_seen_seconds",
    "device": "Lounge Thermostat",
    "type": "se",
    "value": "68",
    "zone": "Lounge",
    "owner": "com.sensibo",
    "driver": "sensibo"
  },
  "meta": {
    "id": "3fe8fe9c-6bd1-46d2-b1ac-a062f97d54c7",
    "name": "Last seen seconds ago",
    "action": "update",
    "entity": "capability",
    "fields": [
      "se_last_seen_seconds"
    ],
    "reason": "se_last_seen_seconds",
    "timestamp": 1720162027496
  },
  "label": "audit"
}
debug: {
  "message": "Study Thermostat Last seen seconds ago increased from 5 to 15",
  "level": "debug",
  "detail": {
    "code": "se_last_seen_seconds",
    "device": "Study Thermostat",
    "type": "se",
    "value": "15",
    "zone": "Study",
    "owner": "com.sensibo",
    "driver": "sensibo"
  },
  "meta": {
    "id": "fdd956a2-3469-4af8-9e9a-550d6cd98969",
    "name": "Last seen seconds ago",
    "action": "update",
    "entity": "capability",
    "fields": [
      "se_last_seen_seconds"
    ],
    "reason": "se_last_seen_seconds",
    "timestamp": 1720162027521
  },
  "label": "audit"
}
crit: {
  "message": "When Sunset Flow has been deleted",
  "level": "crit",
  "publish": true,
  "detail": {},
  "meta": {
    "id": "f3eb075b-7a8e-4dad-adae-57866c032cbc",
    "name": "When Sunset Flow",
    "action": "delete",
    "entity": "flow",
    "fields": [
      "actions",
      "broken",
      "conditions",
      "enabled",
      "folder",
      "name",
      "trigger",
      "triggerable",
      "uri"
    ],
    "reason": "unknown",
    "timestamp": 1720162029068
  },
  "label": "audit"
}
crit: {
  "message": "Fireworks has been created",
  "level": "crit",
  "publish": true,
  "detail": {},
  "meta": {
    "id": "e186c673-ceb6-4deb-8c2e-e72f2584354c",
    "name": "Fireworks",
    "action": "create",
    "entity": "mood",
    "fields": [],
    "reason": "none",
    "timestamp": 1720162107081
  },
  "label": "audit"
}

Betterstack

By far the best method! Go to betterstack and create a new account, then just add the “id” to the app.
All your audit logs will be available directly with in betterstack,. The free account will keep the data for 3 days.

Betterstack is incredibly powerful, you can choose which fields to show :

You can can also use a query languge to limit what to show.

Each audit can be expanded out to show all the meta data

As well as the full details on the homey object

SimpleLog app

Working with simpleLog app to allow the audit to talk directly to it and have your audit information added directly to the app. This is currently blocked, but hopefully will be available in the near future.

Email

I may add this feature, no promises though

Levels

The app follows the standard simple log levels (to allow easy integration into the simpleLog homey app.

1. Emergency (emerg): indicates that the system is unusable and requires immediate attention.
2. Alert (alert): indicates that immediate action is necessary to resolve a critical issue.
3. Critical (crit): signifies critical conditions in the program that demand intervention to prevent system failure.
4. Error (error): indicates error conditions that impair some operation but are less severe than critical situations.
5. Warning (warn): signifies potential issues that may lead to errors or unexpected behavior in the future if not addressed.
6. Notice (notice): applies to normal but significant conditions that may require monitoring.
7. Informational (info): includes messages that provide a record of the normal operation of the system.
8. Debug (debug): intended for logging detailed information about the system for debugging purposes.

As an example this is applied to the device capability updates in the following way :

Level	Description	Example
Debug	Debug, Statistical, Metering and Trace	Any metering of data, information about last updated or last seen, duration, any running tally (such as daily, weekly or monthly)
Info	Information on Measurements	temperature, humidity, ultraviolet, pressure, water, pm2.5, co, co2, battery, power, voltage, current, etc
Notice	Notice of user actions	on/off, dim, light hue, light temperature, light mode, window-covering state, window-covering position, volume, speaker volume, vacuum cleaner state, etc.
Warning	Warning of observations	contact alarm, motion alarm, presence alarm, battery alarm, generic alarm
Error
Critical	Critical changes to the system	locked, lock mode, garage, home alarm state, thermostat mode
Alert
Emergency	Emergency	smoke alarm, fire alarm, water alarm, heat alarm, co alarm, co2 alarm, pm25 alarm, tamper alarm.

Fields

Every audit contains at least the following information

    protected object: {
        id: string                      // The unique identifier of the entity
        name: string                    // The name of the entity, ie. 'My kitchen light', 'Weekend Alarm'
        action: string                  // The action occuring, create, update or delete
        entity: string,                 // The enity which has chnaged, ie. device, alarm, object 
        fields: string[]                // Am array of all fields we have been altered since the last action
        reason: string,                 // An OPINIONATED reason as to why the event is being published, which field changed.
        message: string                 // Human readable message of what has occured
        level: string                   // the level of the audit log (debug, info, notice, warning, error, critical, alert, emergency)
        publish: boolean                // Whether or not we should ignore the objects alteration
    }

Notes

There are a couple of known issues within the app, I am working through.

• Getting the right level of logging for each item.
• Docs around BetterStack have gone missing.
• Hard Drive Management, I am not happy with the risk this has introduced and need to think about how to manage this.
• There is a bug where all of the weather is currently sent rather then just the part which changed.

[Jepke]

@Jamie this is cool and really a very basic feature athom itself is lacking on a easy to pull audit log on what happened when.
On most devices currently i can only pull on/off when/who/what flow

[Peter_Kawa]

Exactly what I was thinking!
Very interested to try & test @Jamie
Good plan to integrate with Simple Log, I hope it will be possible.

Betterstack looks cool as well.

[Jamie]

Sorry just need to create a settings page and then ill submit.

[Sharkys]

Hi Jamie,looking forward to it, looks very interesting project

[Warmduscher]

Cool stuff!!!
I extected that something like this would be available as a basic function.

[Tirrazo]

Hi,

This app is looking quite advanced!

I would like for a starter to get notified in the timeline/push notifications when something happens to Homey.

-Disconnects from Wifi (time + date)
-Network request error (time + date)
-Low on free memory (time + date)

Is this possible with your app?

[SingKT]

Any chance for us to test it (please:)?

[NotBernard65]

Do you have any indication as to when this will be available to use?

[Jamie]

I’m sorry, the app is up and running and doing a good job. But I simply have not gotten around to adding the settings page for the application so that others can use it.

For context, while adding a “device” setting is very straight forward - in order to add aN “application setting” Homey doesnt have a template and you have to manually create it (in html, css and javascript) then manually save each of your settings. Its actually a pain and is why all the app settings look different.

@Tirrazo some of that information is available via other apps, I was actually using one of them - but a bug in the app/homey sent through multiple updates per second - burning through my free quota of better stack. Ill chase up and see if it has been fixed.

[Jamie]

No response from the app creator (sysInternals) or Athom sorry …

To give context this bug in sysInternalsArhom, caused my data use to go up from 10mb a day to 600mb …. Per day in log messages.

Was pretty nasty, so I’ve disabled the app. If you want this feature reach out to the app creator. In the mean time I’m going to have to write some code to ignore that specific app.

[Sharkys]

Jamie do you plan or do you have source code available somewhere or given portion of code, which is responsible ? That could help to understand what changed/happened in this version - they implemented mostly cache for rrd graphs and one more optimization, I wish, I would remember - so I don’t see how your app could be influenced.
Also sysinternals app hasn’t been updated for some time, so doesn’t seems to be related.

On the others side, you have have discovered a bug, which might be worth to address.

[SingKT]

How are things? Any chance of a test version one of these days?

[Jamie]

Sorry, kinda forgot about it once it was up an running in my house. Ill try and find some time to put an image and icon on it and submit it to Athom.

[Jamie]

I tried, but it really isnt my thing - this was the closest I got and well it sucks.

If your interested in the application and able to create a 16:9 ratio image - reach out.

[Sharkys]

Not sure if this will help at least temporarily ? Also not sure about Adobe Firefly logo…

update - another one

[Jamie]

Thanks, ill add it and submit.

When I opened it up today I actually had an idea of how I could add a whole new level of control. Which im going to start working now, but I wont delay getting this out any longer - just be aware it will be an ‘early access’ and improvements are coming.

[Sharkys]

Maybe another attempt…

[Jamie]

Sorry for the delay, I decided to run some tests against matter devices and found an issue. Will fix it asap.

[Jamie]

All fixed, settings page completed and thanks we also have an image

Just need to double check the flows and make sure they trigger when ever something is updated. They used to hit a flood protection issue so I stopped using them. Will likely release with out some flows if it becomes annoying.