How I built a Homey app without an IDE — with Claude Code, Superpowers Skill, and an agentic development loop

Developers
1

I recently published a Homey app — Violet + BADU Blue Pool Control (app thread) — for a pool controller. That thread is about the app. This one is about how it was built, because the process was interesting in itself: the app was written without using a traditional IDE at all. Everything happened in Claude Code in the terminal, with GitHub for version control. Over time, the setup also grew into an automated, self‑checking development loop.

Here is how it went.

How it began — a few talks that changed my mind

I have worked in software development for a few decades, and I had clear opinions about how it should be done. A few talks changed that, and they are the reason I tried this at all.

The one that stayed with me was Boris Cherny (Creator and Head of Claude Code) saying that, once Claude Code existed, he basically stopped opening an IDE and did not go back to it. For someone who spent his whole career working inside an editor, that was surprising.

If you want to see what changed my mind, these are the talks:

So I decided to test the idea on a real and unfamiliar project. I am a solid C#/.NET developer, and JavaScript is close enough to read, but I had never written a Homey app, never used this controller’s local API, and the pool‑chemistry maths I wanted (a live Langelier Saturation Index safety net to protect my heat exchanger from corrosion) was new to me. That made it a good test: if IDE‑free, agent‑driven development only works on small toy problems, a project like this would show it.

The traditional path would have been several weeks of learning the SDK, working out the controller’s API, and then writing and debugging everything by hand. Instead, I stayed in the role of engineer and domain expert, and let the agent do the typing.

Superpowers: the basis for working without an IDE

What made this real engineering, and not just “chatting with an AI until something works”, was a set of workflow skills called Superpowers (by Jesse Vincent / obra). It gives the agent a fixed process and does not let it skip steps:

  • brainstorming a design before writing any code,
  • writing plans from that design,
  • test‑driven development — tests first, then the code,
  • systematic debugging instead of trial and error,
  • a code review and a verification step at the end.

Two community skills handle the Homey‑specific parts: a homey‑app skill (everything Claude Code needs to now abount Homey SDK, conventions etc in one place) and a homey‑cli skill (using the Homey CLI, inspecting the live device, building flows from the terminal, way more functionality than the Homey MCP Server). This let the agent check its work against my real Homey while it worked.

The project was built milestone by milestone. Each milestone was turned into a spec, then a plan, then implemented test‑first, then validated and released, with:

  • a committed progress dashboard for every milestone,
  • git hooks that block broken manifest/changelog JSON and prevent secrets from being committed,
  • automated security reviews (a dedicated review subagent and a written threat model) for anything that writes to the controller,
  • everything in two languages (EN/DE), including the tests.

The result: an app with about 40 device capabilities, the LSI engine, optional write control, and a full unit‑test suite — from the first milestone to a Store test release in roughly two weeks of part‑time evenings (around 30–40 hours of my own time). If I had built it the traditional way, alone, while learning the SDK, working out the controller’s API, checking the chemistry, and doing the write path safely in two languages, I would estimate several months of spare‑time work (150–250 hours). The AI did not replace the engineering decisions; it mainly saved time on the parts that would have cost the most: learning the platform and the domain.

The next step: making the loop run on its own

Once the app was ready for the Store, I became more interested in the development loop than in the app itself. A short self‑assessment put my setup at level 3 of 4: the inner loop was verified (hooks, TDD, subagent checks), but there was no automated outer loop, no CI, and the test results did not actually block anything. This produced a list of 12 concrete improvements.

So I pointed the agent at its own toolchain. Using the /claude‑automation‑recommender skill (which analyses a repository and suggests hooks, subagents, CI and skills) together with the Fable 5 model, I worked through all 12. The main ones:

  • Blocking gates — failing tests or a failing type‑check now block the commit; a missing changelog or version bump blocks a publish.
  • CI and a heartbeat — GitHub Actions runs the tests and homey app validate on every push, plus a nightly run.
  • A mock harness — the glue code in the device driver (where the one real crash had happened) is now tested every time.
  • A read‑only live‑smoke test against the real controller.
  • A stop‑verify hook — a work session cannot end with failing tests or a failing validate.
  • A nightly triage routine that runs the suite and writes the findings into an inbox file that every new session reads first.
  • Hook telemetry feeding a workflow “retro” — a small meta‑loop that learns from repeated friction.
  • Condition‑driven milestone loops (/goal) with a separate, cheaper model as the checker.
  • Model tiering — mechanical work runs on a small, fast model, and only the checking and judging steps use the strongest model, so the checker is never the weak point.

This moved the setup to level 4: an automated outer loop — nightly CI and triage, test feedback that really blocks, and telemetry as a basis for improvement.

Making it reusable

The last step was to move the whole setup out of the pool project and into a portable skill/plugin — skill‑agentic‑loop‑framework — so the same setup can be used on the next project from day one, instead of being rebuilt each time. GitHub - tnsturm/skill-agentic-loop-framework · GitHub

How well it works today

  • The inner loop is verified by the hooks, TDD, subagent checks and the retro meta‑loop. The outer loop runs on its own: nightly CI stays green, and the triage inbox gets filled.
  • The human still makes the decisions. Push and publish stay manual steps — the agent never ships on its own. My real work is the domain knowledge, testing on the real hardware, and the decisions.
  • A good example: I evaluated moving the trickiest file to TypeScript and, after a proper test, decided not to do it — it would have added risk and kept two diverging build artefacts for no real type‑safety gain (the strict JavaScript check already covered that). The loop is good at doing the work; deciding whether to do it is still my job.
  • Known bugs are not ignored — they are frozen as todo tests that describe the correct behaviour, so every run lists them without turning green until they are properly fixed.
  • Model tiering keeps the cost reasonable — you do not pay for the strongest model on mechanical work, and you never save money on the checker.

In short: I shipped an app well outside my previous experience, and I got a reusable, self‑checking development loop as a side effect.

Tools, if you want to try this

The bottleneck moves

Boris Cherny describes the whole change with one idea: the bottleneck moves. First the hard part is writing the code. Once that is automated, the hard part becomes verification. Once that is automated, it becomes the decision to merge and ship. After that, he expects it to be planning. His rule: “Every time there’s a new bottleneck, you have to automate that bottleneck.”

Back to the start

This brings me back to where I began: Boris Cherny not opening an IDE anymore. A few weeks ago that surprised me. After building and shipping a real app this way, and watching the bottleneck move step by step, it no longer looks like a bold claim to me. It looks like the direction things are going.

Happy to answer questions about any part of the workflow. The app itself is here: [Violet + BADU Blue — Pool control for Homey - Apps - Homey Community Forum] (and a German write‑up on poolsteuerung.de: [Violet + BADU Blue in Homey Pro – neue App mit LSI Berechnung - PoolDigital Forum]). The source is on GitHub: GitHub - tnsturm/violet-homey-app · GitHub

2

So you never even check the code? So if Claude somehow recommends a malware/non existent package on npm (happens more than you think) you just install it without even checking? That is very dangerous, you should always check the code or command before running it.

3

Many Homey apps nowadays are being generated by LLM’s without any scrutiny, from what I’m seeing. Also because the people using LLM’s tend to not be developers so don’t know where to look.

4

Hi @Torsten_Sturm,

Great to read this learning path from an experienced developer! Very inspiring! I’m no developer myself but I do work a lot with AI both professionally and for personal (help in developing my apps) use.

The most important part in your post for me is this:

That’s crucial for people to realize, AI is just another tool, not the solution.

5

Well, I know that not everybody who uses Claude is a developer, but you can at least check through the code to see if there are any strange URLs or obfuscated code.

I build a few Android (React Native) apps with Claude for personal use, but I often see that it just hallucinates a package that doesn’t exist (for example for Bluetooth advertising). There are many hackers finding these nonexistent packages from Claude and publishing their malware using that package name on npm. Then, when someone installs it (or Claude installs it for them), they’ll be infected with malware.

The first thing to do when Claude recommends a package is to go to npmjs.com and looking it up there. Go to the Versions tab, when was the first version published? If it was published recently, you should be more skeptical. Go to the Dependencies tab, does it have any dependencies that it doesn’t need? Go to the Code tab, are there any pre-install scripts (very often abused by malware)? Are there any strange binaries or obfuscated code? And when the package is fully safe, you should always install the latest version. Claude often recommends older package versions with many security vulnerabilities, so just running npm install [package]@latest instead of the version that Claude recommends is best practice.

6

Hi Sven,
in the beginning, I did check everything by myself. But then again you or your team become a new bottelneck.
But exactly this is the point… Claude Code or Codex etc are Development Automation Tools.
Think of Claude Code as a team of extremly fast and efficient human developers.
It is not possible then that you as a teamleader check every single developers decision about package usage. You will create (pre-)guidelines and checks and verifications within CI.
Same is with Claude Code. You have to tell it what to do. But because it is much faster than humans developers, you can tell him: do 100% TDD, write a test for 100% of functionality, do full end to end testing every time, and if you find something solve it until all verification gates are green.

And yes, there is still halluzination, but with proper guidelines and the latest Models (Fable 5), this got way better. But think of the humnan development team… not all of them will be the worlds leading employees in this domain… you will have to take precautions to automatically recognize and fix common mistakes. Same with Claude Code…

As an example, most Clauders now you use the context7 plugin… it always ensures, that Claude Code works with the latest SDK docs of the project… automatic and within seconds.

7

It’s not about untested code, but more about the risk of getting infected with malware. It often happens that Claude comes up with an npm package that doesn’t exist. But sometimes, hackers will publish an npm package with that name, so that whenever Claude installs the previously non-existent package, it will then infect you with malware. I would recommend reading this article for more information about the hallucinated packages:

Yes, but human developers don’t simply come up with packages that don’t exist​:wink:

8

Claude seems to be missing some fundamental Homey specifics though.

For instance, using a random UUID as device ID is not a good idea, and Homey stores aren’t encrypted.

9

very good point… I will check what Claude suggests how to handle it. Thanks for that special feedback.

10

Yes indeed, the whole point of that ID is to make sure that the same device can only be added once. Should be a unique identifier (e.g. ID from the cloud server of the manufacturer or the MAC address).

Yes. However, stores can’t be fetched from the API. Settings can be fetched from the API though. AFAIK there is no way to encrypt anything on Homey.

11

I see…
Some other Homey Apps are using the IP address (com.luxtronik.heatpump/drivers/luxtronik-heatpump/driver.js ), but this is also not recommended by Pairing | Homey Apps SDK.

But how do I get the MAC address at this point?

12

And that’s another vibe coded (fully AI-generated) app, so the same applies here with the AI not being able to understand Homey’s architecture.

How does your device connect? Most devices connect using a cloud service, usually those store an ID for the device that you can use. If it connects via a LAN connection it can be trickier.

13

Ask Claude :wink: