One developer had the great idea of using the OpenAI bot as a home automation assistant. The result is amazing, and should serve as a lesson for Apple, Amazon or Google assistants …
It’s quite a DIY, however, and it might just be a first look at the future of personal assistants. Mate Marshalko, a developer who may have grown tired of the answers Siri gave him, had a great idea: He tested ChatGPT to control his connected home by voice.
In a video posted to Reddit and YouTube, he showed the — utterly amazing — result of his experiment, which has got Google Assistants, Apple, and Amazon back on the ropes.
If you regularly use Siri, the Google Assistant, or Alexa to control your lights, radiators, and other connected things, you must have realized that you have to adapt to it. In other words, perform very basic commands: “turn on the light in the living room,” “turn up the heat,” “turn off the TV.” It is impossible to make them understand more complex requests, so they quickly lose control.
Marschalko’s tool, built in an hour by combining the power of ChatGPT and Apple’s “Shortcuts” application, is a game-changer. And he correctly interprets the most complex requests of all GAFA assistants. Example ? This request that would have clearly put traditional voice assistants in a cabbage: My wife is coming in 15 minutes. Turn on the lights outside for her when she goes to the park. The program understands it without the slightest concern, and the assistant proudly answers: The lights should be on the moment your guest arrives.” Or this one that also benefits from ChatGPT knowledge: ” Set the thermostat in the bedroom to the temperature that you think will help me sleep better And the helper to reply: The bedroom thermostat was set at 19 degrees. Enjoy your sleep! An absolute refinement, the wizard responds differently each time, as if brought to life.
Procedure… Not that complicated
To connect his dozens of lights, thermostats, ventilation system and cameras to ChatGPT, he proceeded in two stages. First, we had to “teach” ChatGPT. A long query (see below) looks like a small computer program…written in natural language.
And so Marschalko demanded that ChatGPT respond to every request in the form of JSON, a popular data-structuring format that’s easy to understand through the Shortcuts app for iOS. Then he described the types of requests needed, specified the JSON structure that ChatGPT should generate, and finally gave an accurate description of his home and the connected objects that were in each room. That’s almost all. Ah yes, he asked too bot to impersonate” The mind of the house, an intelligent artificial intelligence, without revealing his true identity. »
Then it created a new shortcut on iOS, which looks like a supercharged version of that We described it to you a few days ago. Rather long and complex, this chain of commands in short makes it possible to create the interface, thanks to data in JSON format, with its connected objects, via the Apple HomeKit platform.
It is not (yet) at the moment
Deft and effective, Marshalko’s hack is not without flaws. First, there is the price. This solution really requires access to ChatGPT through its own API, thus… it pays, even if OpenAI provides a (small) free credit. But it is quickly exhausted here, because the order is long. It takes about $1 for every 70 requests, according to the programmer. And then there is… the slowness. As such, the video can be overwhelming, as ChatGPT seems to respond quickly. However, it set in: there is an incompressible delay of a few seconds between the voice command and the bot’s response.
However, we can hope that Apple, Google or Amazon will also consider leveraging generative AI to improve the relevance of their voice assistants. Who came, with ChatGPT, to take an old smack.
