[APP][Pro] OpenAI ChatGPT

Apps
,
21

Ha, I asked chatgpt to create an adv. flow, but it came up with a bullet list of how I should do that myself :stuck_out_tongue_winking_eye: instead of code.
But, this tool creates adv. flow by running a script in the api playground!

1 Like
22

Ah, yeah, that is a nice option, but that’s not exactly the same as creating the flow, it’s just creating a flow exchange definition that can be imported as a new flow, isn’t it?

Anyway, if you (or someone else come up with a ChatGPT query that outputs such a flow definition I can probably make a shortcut for it in the app.

I imagine such a flow as: “Tell ChatGPT to analyze the command xxxxxxxx and perform actions based on the result”

This flow will classify the query in a few built in actions in the app:

Whereas the built-in actions could be

  1. Turn on/off device nnnn
  2. Set the temperature of device nnn
  3. Create a flow exchange definition and show it in the setup screen (where you would be able to click it to manually import it, because I don’t think I can actually create the flow automatically)
  4. Give generic feedback from a device (temperature, battery-level, etc…) and trigger a flow with this text string as an argument. (or return it if it’s an advanced flow)
  5. …other built in actions…

This would be a bit into the future though as I don’t have time for this right now, but if others make the actual ChatGPT queries work then I can just throw them in here with little effort so this could speed up the process…

23

My first attempt was this question to ChatGPT:
I want to create an advanced flow in Homey Pro, where I can record the exact energy in kWh consumed by my dishwasher when washing dishes at 65° Celsius. This should also show the water consumption in m³. Please explain to me how I create such an advanced flow.

The answer I got back from ChatGPT was insufficient. So I rephrased my question to:
A smart plug registers that the power of the dishwasher has changed. If the power is greater than 100, the dishwasher is switched on. There is no need to include a delay for the dishwasher to heat up, so that “delay” can be omitted. Nor can it determine what temperature the dishwasher is set to. The smart plug only recognises the fluctuation of power. Based on this information, give me a better explanation.

Again, I was not satisfied with the answer ChatGPT returned. So I rephrased my question to:
The “when” trigger is the changed power of the smart plug. If this power is greater than 100, the dishwasher is on. If this power is less than 10, a 4-minute hourglass starts. If, within these 4 minutes, the power changes again and is greater than 5, the hourglass stops, because that means the dishwasher is still running. But if the hourglass has ended, that is the sign that the dishwasher is finished. Based on this information, please give me a better explanation.

The answer of ChatGPT:
"Based on the new information, you can follow the following steps to create the advanced flow in Homey Pro to record the dishwasher’s energy consumption:

  1. Create a “When” trigger: Go to “Automations” in the Homey Dashboard and select “Add flow”. Then choose “When this flow starts” and select “A device changes status”. Here, choose the smart plug linked to your dishwasher and select “On” as the status.
  2. Create an “And” card: Next, add an “And” card to check if the dishwasher’s energy consumption exceeds 100 watts. Select “Device” and choose the smart plug as the device. Then select “Power” and choose “is higher than” and enter 100 watts.
  3. Add a “Then” card: Add a “Then” card and select “Logic”. Select “Measure energy” and select the energy meter linked to your smart plug.
  4. Create an “Else” branch: Add an “Else” card and select “Timer”. Then select “Start a timer”. Set the timer to 4 minutes and name the timer.
  5. Create a “When” trigger for the timer: Create a second “When this flow starts” trigger and select “A timer has expired”. Select the timer you just created.
  6. Add a “Then” card: Add a “Then” card and select “Notification”. Add a notification indicating that the dishwasher is ready.
  7. Add an “And” card: Add an “And” card and select “Device”. Choose the smart plug as the device and select “Power” and select “is lower than” and enter 10 watts.
  8. Create an “And” card within the “Else” branch: Add an “And” card within the “Else” branch and select “Device”. Choose the smart plug as the device and select “Power” and select “is higher than” and enter 5 watts.
  9. Add a “Then” card within the “And” card: add a “Then” card and select “Timer”. Choose “Stop a timer” and select the timer you just started.
  10. Add notifications: Add notifications so you are notified when the flow is activated and when it is completed.

With these steps, you should have created a flow that can record your dishwasher’s energy consumption based on the smart plug and give you a notification when the dishwasher is done."

Does this help? :grinning:

24

Well, it’s a step on the way, but I guess for this to be automated I need the output from chatGPT in a parseable form.

So we define the following:

  • Input_text = Whatever the user told homey to do
  • ChatGPT_wrapper = “Whatever text to wrap around the Input_text in order to produce the Output we need”
  • Output_actions = an array of instructions of the following form [Action 0, Action 1, …, Action n]
    • where Action is a json-structure {id: number, parameter1: text/number, parameter2: text/number, …, parameter n }
    • id is a number that have the following meanings:
      1. Create a “when” card that will trigger when the device with name=parameter1 changed temperature, the output of this when card will be the temperature itself which should be stored in a variable with the name stored in parameter2"
      2. Create an “and” card for logic comparison that compares the variable with name stored in parameter1 with the value stored in parameter2 using the compare-function stored in parameter3. A compare function must be a number which has the following meaning [0: not equal, 1: equal, 2: less than, 3: less than or equal, 4: greater than, 5: greater than or equal].
      3. Create a “then” card that will create a timeline notification with message = parameter1
      4. Create a “then” card that will turn on the device with id=parameter1
      5. Create a “then” card that will turn off the device with id=parameter1
      6. Create a “then” card that will change the temperature of the device with id=parameter1 to be parameter2 degrees celcius
      7. …fill in the list with other definitions of your liking…

Where only the set of actions and id’s we define will be allowed, otherwise it must report that the required set of actions are not supported by the app yet and give feedback what action is not supported so we can add it to the supported list.

25

Hey guys, this thread is about the Homey app, not about ChatGPT itself.
So please keep on topic or start a new thread in the Off-Topic category.

26

It was actually on-topic @RonnyW (allthough @RonnO was talking a bit around the porridge), as the suggestion bottoms out in finding out how to extend the app to do automations based on the output from ChatGPT, which requires us to find out how to change the Input to ChatGPT in a format that gives a parseable output that can be understood by the app.

This will let us add on automations to the app that will act on the output from ChatGPT.

You may continue the discussion in this ticket to avoid cluttering this forum:

27

I think even GPT4 is only thought until september 2021. So it probably doesn’t even know what an Advanced Flow is since it wasn’t released yet at that point.

1 Like
28

New version 1.4.0 is available for testing

Ok folks, as requested, I added support for GPT-4 (and the faster, much cheaper gpt-3.5-turbo)
If the test-version doesn’t have any problems it will be pushed to stable in a few days.
Have fun!

1 Like
29

@tokreutz was so kind to add support for multiple chat sessions in the form of chatbot devices, I merged the changes today and sent out a new test version. Since I’m leaving for easter holiday I will not have any possibility to do any changes to the test version or follow up before after easter, so I’ll leave the new version in testing for some time to keep the stable version safe while I’m on holiday.

Thanks a lot for the contribution @tokreutz ! :heart:

30

I had some experiments with “gpt-3.5-turbo” today and had great success.
I will add an action to the app this week that goes something like this:

  • "Analyze user query xxxxx, perform relevant actions and return a text string of what happened.

Actions will include:

  • Turn on/off devices
  • Change temperature of devices
  • request information from devices

Anything else you would like to add?
(creating flows as discussed above will come later)

31

Cool stuff. One feedback could potentially be to support a very generic “UpdateCapability” action that could update any settable capability of any device. That would be less limiting, and would require fewer actions to implement and will support a wider range of devices. I think the models are capable of identifying this if they get enough context in the system prompts. :slight_smile:

On a side note, I have experimented with similar things myself, and I felt my experiments went a little outside of the “OpenAI scope”, and I didn’t want to confine myself to OpenAI only, so I have started on an app of my own for this. Mostly as a learning project. Might be a little but overlap with your experiments, but I don’t think there is any conflict.

32

Good idea, I tried to experiment with this a little but I had some trouble getting the AI to be sufficiently deterministic. I will have to postpone my update until I improve this some more.

One of the problems is that I cannot give the AI too much information about the devices at once because if you have many devices it quickly exceeds the maximum length of the input… as well as making the requests very expensive. So I need to first reduce the device list to only include relevant devices and then figure out relevant capabilities in subsequent requests. Also, trying to get it to work with gpt-3.5-turbo to avoid too high bills. Unfortunately, my free trial has run out so I have to pay to do all the experiments before I can make the new features available to you all. OpenAI’s policy isn’t very friendly toward Open source developers. The devs provide them with users so they earn money but that doesn’t stop them from milking the developers too.

Did you try this one?

It’s certainly a bit tempting to switch over if you can get it for free yes.

33

Haven’t looked at that, no. But there are free (but less accurate) APIs out there, like Huggingface for example. I am not familiar with them though.

I was more referring to additional functionality outside of just exposing OpenAI APIs as an app.

34

Maybe an idea to use the app Tasker to use the microphone of your tablet or phone to record the speech and then let Tasker send that input with a we hook or http request to Homey. You can then use that to send to OpenAI?

35

Every question I ask I get an error Request failed with status code429 response".

Only created the API key just now, but I get this response with the first question.

36

429 means “Too Many Requests”

It means that the OpenAI servers are overloaded and that they block users from accessing the service, starting with users having free test accounts.

I am not sure if there is much you can do except try at a time where the server have less traffic. Or potentially you may be able to get around it if you upgrade your account to paid?

I just tested and I got a normal response, though, it’s not exactly peak hours :slight_smile:

37

Hi,

I didn’t know traffic could be blocked during peak hours. Learned something new today.

Thanks!

38

Hey - looking to find a way of taking the text respose, and turn it into a text string.
I am asking the app to look up an 80s song, I would then like it to play that song on sonos.

Any ideas how to do that?
Thanks,
Nick

39

Sorry about the Norwegian flow, don’t know how to make it english without rebooting my homey:

image
image746×326 38.6 KB

So the thing is that you need to ask questions in a specific form with examples, otherwise the answer might not be presented in a way you like:

Question for ChatGPT:

Please list artist and song in the following fashion without any other words:
Artist - Song
Example:
Question:
List a great song from the 90’s.
Answer:
Nirvana - Smells Like Teen Spirit

Now it’s your turn.
Question:
What is the best ABBA song
Answer:

40

I was just fooling around a bit with this today and tried to make it a part of what happens when my child press her action button.

Trying to get a morning/goodnight story created by AI and then read out loud through either Sonos or Chromecast.

But i get the same issues both places with not finding a good way to discover when to call for the next partial answer … the delay for 12 seconds is the closest i could come since each partial answer usually takes about 10-11 seconds to read out loud.

Anyone have any input? :slight_smile:

image
image1880×1155 291 KB