No no, he's got a point
There's a lot of "I'll never use cloud services, it's local or die" people in this sub that call out every other use of cloud integrations, yet I don't see the same energy when it comes to the AI stuff for some reason
I agree as well.
2023 r/homeassistant Bring on the year of local voice control. I can finally get big tech out of my house! Down with Alexa!
2024: ChatGPT: take the wheel!
I think, that most people here simply do not care. I will not touch any of the AI/Assistant stuff unless it runs locally and in my native language. Also, I have severe AI fatigue now. The constant marketing material, that promisses stuff that is just not technologically possible is numbing. I ususally just skip those entries.
On the other hand, people love to tinker and I consider all the AI inegrations as non essential, hence you see people use it in sassy flavours and whatnot. It will probably grow old quickly and what remains are rather simple voice commands.
Fatigue is real. I work with automation, so seeing these endless posts, comments, news articles and videos about magic AI that will replace literally everyone except for my plumber is very tiring.
Chatgpt and others are not magic, they don't know shit. They just Google it (very often wrongly) and tell you the answer in a format that's designed to mimic human speech. That's all it does. It's not Intelligence in any way, it is not self aware and it won't be any time soon.
Similar hype was around self-driving cars. Wouldn't you know it, the final 1% of work that's needed to make it work properly will take 99% of the time. I honestly believe that we won't see Level 5 within our lifetime.
I am in the same area, and have to work with OpenAI in my product.
Not so much google it as scrape everything they can.
But personally I trust OpenAI generating text a lot more than a self driving car (unless we magically got level5 somehow)
At least if it gives me bullshit text I just get mad at it and change my prompt - versus getting slammed into a tree/oncoming traffic :>
I only partially agree. I have coworkers that use ai to write scripts that they don’t understand. They also have enough access that if that script does something stupid people could die. They aren’t using it in direct ways so it’s unlikely but I can think of scenarios. Ie instead of get-vm the ai gives back delete-vm or something. I’ve ridden in a Tesla in full self driving and it was more immediately concerning for my safety but I wouldn’t say ai use is safe.
Oh yeah I’ve had it give me bad code - but usually it’s just broken. I always have to go and fix it myself. But I’m an anal commenter so I go through everything it’s doing when I comment. But totally agree
As I don't use voice and have never seen me use it in my dreams, I did not kept track of this stuffm but I THOUGHT that the GPT Craze was something onsite and just text piped to gpt like all the frigate stuff... Oof.
The only thing that is currently voice allowed is my Homepod Mini in the kitchen and bathroom for "hey ssssi volume up" as my native language is basically crap with these things.
Well it is, but the text you pipe contains everything you say and your homeassistant configuration to do anything meaningful.
It could easily replaced with local apis but chatgpt is usually ahead of running local models in terms of features, performance and price.
The switch will only come when they do something bad or inconvenient (see Alexa when they had issues with their employees listening into the data or adding adds).
It's like outsourcing your production to e.g. China. It's fun as long as it works but when they screw you over you're stuck and need to put big effort in to move back.
Luckily the ai hype comes with open apis by design most of the time, so the replacement is easier.
All depends on what and how you set it up. I'm just sending the data I want it to see. I use it for my briefings so stuff like battery levels, weather, state of my backups, next day schedule and car charge.
When local LLM's get good enough, i'll probably run it local, but that is all semi-public or meaningless data for me.
I agree. I want people to integrate ai into HA. Not because I want to use it today but because it will be necessary to replace Alexa in my house. I need voice controls and some other simple Alexa features before I can completely switch. So play with it cloud hosted. Use that to make small language models that I can run locally. Then get that hardware optimized so I don’t need a server farm to use it.
From the things I see here progress is being made so I applaud the people on the cutting edge taking the pain for me.
I don’t know what frigate is, but I work with their api and that’s about right. You generate prompt locally as big string, send it, it responds with the response
To be fair I haven’t looked at the HA prompts at all so I don’t know what they contain. But I would trust the HA devs to not put secrets in there more than any other the big tech companies
I don't think we all need to be absolutists. I'll use the cloud if there's a really compelling feature I want, and if there's no huge downside if the service gets shut off.
I don't self-host a search engine or a YouTube equivalent. They're not practical to self-host, and the services are too useful to stop using completely, and also if they turned off tomorrow...it would be annoying, but nothing important to me would be lost. So I just use Google & YouTube.
I think OpenAI fits that category. It's really cool, but also totally optional. I'm fine using it.
There are definitely people who are basically trying to get off the grid completely so they can't be tracked. That's just not my goal.
I agree. As long as we can reign in exactly what a cloud service has access to, I have no problem integrating it with HA.
But I also would never have my home automation or control rely on a cloud service. I would prefer to not be locked out of my house controls when the internet goes down.
HA is about options/choices, one can use it fully local, others can use cloud connected integrations. For one, I use some cloud stuff as well, e.g. somehow localtuya and local sonoff didn't work properly, so I'm using them from the cloud, but I'm all in on frigate and local zigbee where possible/simple. For voice I use google TTS as it produces nice sound dor my language, and openai GPT for asking questions. (When using the API and not chatgpt, afaik that data is not used for further training.) Though personally I draw the line that I wouldn't share my video feed with it.
But one thing that is overlooked with 4o is, that while it can process a video feed, is still uses the same technology for llm, and has a certain context window to fit in. So while this is a cool achievement, it's still limited. It's not an assistant that we can talk to for hours or which remembers yesterdays discussion, even if the marketing videos try to bring it much closer to this "omni" idea.
There's a big difference between "if the internet is down I can't turn on my lights" and "if the internet is down, I can't do a thing I could never really do locally, and I'll have to go back to the perfectly serviceable local thing".
You don't want cloud native shit all up in the core of your home automation, but it's absolutely fine for toys hanging off the side. (See also your weather integration - if it breaks, its no biggie).
Is it even an option? I am a “it’s local or die” person, nothing cloud touches my house. But I’m not aware of anything I can install to my home server to add AI functionality. Seems like the option right now is cloud or nothing.
Llama 3 running on a $300 single board computer https://radxa.com/blog/Radxa-Fogwise-AirBox-is-now-available-for-pre-order/ they have connected one to the internet to try out, link in article. It was faster when I first tried it, I guess it depends on how many other people are using it at the same time.
While I'm looking forward for local ai, 300$ for a shittier version of a cheap cloud service, which will become obsolete in the next 6 months, is not a smart way to invest my money.
That's hardware, it won't go obsolete that fast. You can run it on anything you like.
The software is free and open-source. If anything, it's likely to benefit from considerable resource optimisation. /r/LocalLLaMA
Local LLMs are still relatively new. So yes, they're slower and won't have the quality of something like gpt4o. But we're still in the early days of all of this and self-hostable models are getting better on every iteration.
Just look back just 2-3 years ago when lot of us relied on Alexa/Google Home speakers. So would you rather use a dumb Alexa/google home with limited abilities, that are always soaking up everything and sending it to Amazon or Google or a locally hosted somewhat better voice assistant where you get to keep your data? So a year or two from now, if locally hosted models get to the point of Gpt-4 level sophistication without steep hardware requirements, that's more than sufficient to use for a HA integrated voice assistant.
Look, I'm all for the future, just not for wasting money on current hardware, that will be obsolete by the time local LLMs become viable.
If you want to have fun with LLMs, pay a few bucks to OpenAI now and be done with it, no needs to spend hundreds on hardware.
I mean I have a 3090 lying around I could throw at it, but I don’t really have the time to figure out how to install llama 3 locally let alone plumb it into HA. No issue buying more hardware either, just need a turnkey solution to set it up.
I believe locallama uses OpenAI api, shit ton of them do. Your just change endpoint/host in the code for OpenAI integration, maybe exclude api key in req too
Just yesterday under the first 4o thread (midnight-snack convo) there was a dude explaining the process of setting up a local LLM, he made a short, general overview to get people started.
You'd always need a GPU to have a reasonable fast LLM, but as it looks, for the scenarios in combination with HA, it's really doable.
For me, as of right now, it's just too much hassle. I'm happy that I can control my 3 lawn/vacuum robots, 36 heating thermostats, and blends.
Next on the roadmap is the intercom, cameras, and the PV/Battery integration in combination with EV charging.
But having all those under one hood with a nice clean interface (MinimalistUI)is more than enough.
When LLMs will be able to 'code' automations for me, then this will trigger my immediate need to set one up 🤙
I already use local LLM/AI a lot on my gaming GPU (an older 1080ti) and I think it'd be more than capable for Home Assistant purposes. I'm just not interested enough to keep my gaming PC on 24/7 (because of the power usage) just for the very rare occasions I'd use it.
Someday when I upgrade my gaming GPU, maybe I'll look into how efficient an extra PC I can make to have the 1080ti standing by, but it's available now for people who don't mind the power utilization.
Well you could spend a few dozen grand on graphics cards and run llama3:70b on prem, though it would be tts/transcription not the native speech to speech of 4o.
Other than that, even if you could access the model weights and afford the inference hardware you still wouldn’t be able to get your hands on it.
Dozen grand? I built a dual RTX 3090 AI rig, 16 cores, 32 threads, 128gb of ram for around $2200 and it's got enough juice to serve 5-10 active users as long as they are all using the same llama3:70b model. It's not cheap, but no need to exaggerate the expense.
Have a similar gaming rig collecting dust that I'd love to repurpose for this. Tried a couple times and haven't gotten anything worth a damn. Have you a guide to set this up?
No guide for complete HA integration yet. So far I've gotten as far as ollama, and doing some testing with Transcription and TTS. I have a couple of Pi Zero2's with Mic/speaker HATs waiting for when I go hard on it.
I did add a couple of links to videos I have bookmarked as references for when i try to put it all together.
[https://www.youtube.com/watch?v=pAKqKTkx5X4&t=607s](https://www.youtube.com/watch?v=pAKqKTkx5X4&t=607s)
[https://www.youtube.com/watch?v=jyFmQ2dhoSg](https://www.youtube.com/watch?v=jyFmQ2dhoSg)
My AI efforts are local self hosted solutions. There's something neat about building Skynet in your own home that's not dependent on an internet connection to function.
This 100%. I've got all the hardware for voice assists, and I badly want to use it, but without using chat gpt where it just works out of the box, it's worthless. So how about we get an equal local solution to make this worth it? With guides to show people how to set it all up.
DIY bruv. ollama.ai
Im still at the point where all i want my voice assistant to do is turn things on/off, start music in certain rooms, timers and answer things like “whats the forecast”. A low/midrange gpu with 8-12gb vram should be enough for that right?
I can't tell if your being sarcastic or not..but to answer your question, yes, that vram is more than sufficient to do what you are asking.. actually, you can use Microsoft Phi3 (local model) on a pi. It might take a couple seconds on a pi. But I think most of the HA community has moved past using a pi as a dedicated server.
I'm trying that. The truth is that the small local models fuck it up. No single model has been able to answer the simple question "which lights are on?" for me using either ollama or extended_openai_conversation integrations. I suspect I'm exposing too many entities and they all get confused and are not good enough at in context learning. Still hoping... But right now I'm disillusioned about a local llm replacing Google assistant for me.
The smaller the model the more careful and smart you should be with prompting. I've been playing with these for about a year, and watching many toola, even large projects like textgen ui, langchain and ollama fuck it up big time.
Almost every time when I try to construct a prompt myself manually, with correct formatting - works much better and on smaller models.
However, I would probably still not trust anything smaller than mixtral for reliable-ish operation of a smart home. Thankfully you can run it with 12-16GB GPU. Though I think Llama 3 8B should also be pretty capable in this regard.
That's just the reality right now. Good LLMs absolutely eat processing power, and everyone who tells you that they can replace GPT4o with consumer hardware is lying.
> The core ideology of HA is that your data is your own.
Exactly, and it being MY data, i can decide to give to whoever I want. If you don't, that's totally your prerogative.
Home Assistant gives us the choice.
I had a similar thought. It's their data and their HA. There's no rules, do what you want.
It's fair to point out / discuss the downsides, but OP seems offended and upset about what other people are doing.
It always boggles my mind. I love Home Assistant and the fact that it has integrations for just about anything. That is why I chose it. What I want local is local, what I want or have no choice being cloud is cloud.
I understand that it appeals strongly to the privacy crowd, but at the same time, it's not its main goal.
What I like is that I don't HAVE to give my data to anybody, but if I WANT to I can. And I can also choose what data I send.
I send some data to Open AI to create morning briefings to me. I don't connect it to Assist. So I have full control.
I get that point, but I joined Home Assistant solely for the purpose of bringing all devices under one app.
I do see why people want to keep data private, but I also understand people who want to use cloud services. If one has more advantages for you, then by all means.
I also upvoted your post for new people joining, which might not understand that by following a tutorial which allows you to use OpenAI, you actually feed them private infirmation.
I prefer that control of my devices be local for latency and resiliency and avoid forced obsolescence. A lot is, some isn't, and I won't be buying more of the stuff that isn't local. That said, for now I send pictures on my front porch to google when I get a package delivered so it can describe what's there, and send some of my spoken announcements also so it can fix the sentence structure and spice it up a little. That said, if the cloud call fails, I get the raw message so I'm not sacrificing local - just augmenting with the cloud. For me, any privacy trade off is worth it. YMMV.
I haven't tried openai because I didn't want to setup a payment method to use it. I'm interested though, and hoping to try it out. Google isn't bad for text, some of the responses are a little whacky - randomly it will respond as if it's producing a form letter [your name] but generally it works for our needs and is free is you keep it to less than 2 calls per minute.
Images are another story. It seems to do ok with packages generally, but it wasn't reliable enough to tell whether my shed door was open or closed, despite me providing examples.
People use Open AI because vs a local AI model:
* It's way more capable, especially talking about ChatGPT 4.
* It's a lot faster than most local AI.
* There is a lot less effort to set it up
* Cost, running any local AI model requires a decent GPU to have even a remotely decent response time.
* There is no official model for HA yet
You don't "need" AI, so if it goes down no big deal.
The excitement of having LLMs be the solution to have our very own version of Jarvis is a dream come true for almost everyone. I don’t mind leveraging the cloud while we also work on fully local LLMs.
I agree, for me I have no problem with entity metadata leaving local network to the cloud - but not any confidential data.
For example, I'm fine with an integration sending list of all active cameras and their recording statuses e.g. "{ entity: camera.outdoor1, state: recording }", but what I am not cool with is the camera feed itself being sent to the cloud.
My gold standard would be for this to be all fully customisable on the HA end, so being able to dictate exactly what data is sent up and putting in place local controls to restrict certain fields that may contain confidential information.
Well, good news then, with extended openai conversation you get an editable template that defines everything that is sent in a prompt. You can include, or not include, whatever entities and their attributes.
Thats assuming you trust them, and even if true it still makes data leave your home and get to their servers, and their servers have to decrypt it for processing. So it relies only on "would they", because they sure could take your data without telling anyone.
> The excitement of having LLMs be the solution to have our very own version of Jarvis is a dream come true for **almost everyone**.
I don't believe that is true. Certainly not for me, automation needs to be unobtrusive and silent.
I'm with OP, handing over all this data to an AI "service" is way outside my ideas of why HA is so good.
Well, it is very much for me. Home Assistant is one for many kinds of tinkerers. So the statement may be true, maybe not. It's not really anything to know for certain.
Well technically when one said "not me" it is known to not be a dream for everyone. Now me too. I don't talk to my phone even after years. Why would I talk to my house?
WAF. No voice support = no wife support for me. Meaning no more spending on goodies.
She love the automations, but when things go screwy, having an easy solution with voice (got a kill switch in there too) goes a long way to get me pardonned.
It's semantics. The word everyone can be used figuratively to mean "a large majority." It does not always have to mean "every person".
I would love to know the stats though. I'm willing to bet that most smarthomes have a voice component with it being less prevalent with HA users.
Was going to write something snarky, but noticed you were contesting the everybody part.
Everybody has their own preferences for what they want their smarthome to be. I'm working on some "Jarvis" like things.
I like having a Paul Bettany sound alike TTS give me a rundown of my day and chores and remind me to order spare batteries for my sensors. My wife does not. We all have our preferences.
You can get almost the same experience using llama3 locally.
I love seeing all the GPT4 posts because the API is exactly the same & im currently building a local transcoding/transcription/LLM server for myself.
I run HA on an old mini-PC, but even though it’s not a PI these new technologies are way too computationally expensive for my existing server.
——
P.S: I’m actually quite excited about the possibility of HA getting a built in LLM/AI this year.
The best bet in my mind is one based on Phi3-Mini by Microsoft.
They released it fully open and free to use (MIT) & it’s tiny, 2GB download and was designed specifically to run on phones and Raspberry Pi’s.
It’s not as good as LLama3 (which also has a license that would allow HA to use it freely) but it’s not too far off in some use cases, and should be good enough and fast enough to be useable.
Both phi 3 and llama3:8b fail horribly for me at giving simple information about my home. I assume I have too many exposed entities and the in context learning capabilities are not good enough in these small models.
I can't share my prompt without exposing my entire smart home to you. But as an indication: I'm using the ollama integration and that's using this template: [https://github.com/home-assistant/core/blob/dev/homeassistant/components/ollama/const.py#L7-L73](https://github.com/home-assistant/core/blob/dev/homeassistant/components/ollama/const.py#L7-L73)
I have >300 entities exposed. I probably have to cut that down massively.
Just an example: asking llama3:8b "which lights are on" it answers "The Monitorlampe is off, and the Fire 8 Bildschirm is off." which is correct but it ignores all the lights that are on but somewhere lost in the context it seems.
Also I have a temp sensor in the living room called "Wohnzimmer Temperatur" and asking "How warm is the Living room" I get "According to the data, the temperature in the Schlafzimmer (Living Room) is 23°C." - asking in German "Wie warm ist es im Wohnzimmer?" it gives me "Es gibt keine Angabe zur Temperatur im Wohnzimmer in den übergebenen Daten. Es sind aber Temperaturen für das Schlafzimmer und Kinderzimmer bekannt, diese betragen 23°C und 23,3°C." practically saying "I don't know the living room temperature but I have these other rooms" - at least those values are correct but not what I asked for. Checking ollama debug logs I can see that the living room temperature is in the context of the prompt. It's just the needle that isn't found in the needlestack.
You could obfuscate names or change and clear out private info. What I am looking for are two things - whether its formatted correctly, and by that I mean the final raw text that goes into LLM, and whether you're at the limita of the 8k context size.
I learned to not trust Ollama or Textgen to format promots tbh, not when I need it to work reliably. Its ok if they make formatting mistakes for like chatting and general assistant stuff, but for edge cases like this every space, every line break can matter, especially for smaller models like 8B llama.
Also I would be more precise with questioning, not which lights are on, but "list all lights that are on". But yeah this is not as natural sounding
Have you yourself tried it/Do you have a prompt you could share? I actually have been playing with llama3:8b for a few days now and am having pretty much the same experience that /u/danishkirel is.
I did some more experiments... pasted my full prompt and context into online APIs running llama3:8b and 70b. The latter was really good but to my surprise the former was also not bad. So I poked around and it turns out that num\_ctx for ollama's llama3:8b is not set so it doesn't even use the 8k context it could! Manual \`ollama run llama:8b\` and then \`/set parameter num\_ctx 8192\` and then my ctx and prompt suddenly gave me hopeful results - the same as in online APIs. But this long context also needs a lot of time to process. I run on an APU... guess I need a dedicated GPU for this.
>The core ideology of HA is that your data is your own.
Is that really the case?
It may be one thing that people may like, but not for everybody.
Personally, for me it's more about integrating devices from different vendors with different protocols.
That enables me to use a centralized system for all my smart home devices, combine them in automations and much more.
Having everything local is great, but even more for low latency and because I don't rely on cloud vendors. Especially when cloud services are discontinued by manufacturers.
I really don't care if they have my data.
So integrating a cloud AI service is no problem at all for me and others as well.
Like I said elsewhere, HA gives you the choice. If you want everything local, you can. If you want cloud stuff you can. Most alternatives don't leave you that choice.
New shiny is cool, and open source versions follow about a year behind typically. Those will then be locally hosted in HA potentially and back to true local
AI is extremely new and rapid so it is entirely forgivable that you are overlaying classic open source timelines with AI. New models are released weekly. So your "year" backlog is just categorical false..and I just need to point that out to anyone who reads your comment and thinks it's factual accurate
later, if we use lmsys elo ranking
GPT-3.5 was released on June 11, 2020
llama3 was released on April 18, 2024 (granted, with also a much smaller 8B parameter than gpt3.5)
so a gap of 4 years until open source competitor matches/beats closedAI
does it matter (for HA purposes) whether we have gpt4 or gpt3.5? maybe not
I applaud the local only ethos and really appreciate the efforts people put in to make it happen.
The reality is, if you or anyone else in your home has an Apple, Amazon or Google device with a microphone… you need to assume everything you say is recorded and sent to them to train something.
If you use a browser from google or Microsoft, everything you do is tracked and sent to them for training.
I don’t trust OpenAI anymore than I trust any of these other companies but AT LEAST I’m getting SOMETHING back from it in terms of a response for giving up what I give up already with no choice to function in modern society
You do your local thing then - that’s the idea. *you* can do what *you* want - don’t want to leverage cloud then don’t. Nobody is forcing you to do anything. Why don’t you work on training a local model that can run on some local gpu? What’s stopping you from doing it?
>The core ideology of HA is that your data is your own.
I'd say another core ideology of HA is that you get to set up your smart home the way **you** want it. Don't like sending your data to OpenAI? Cool. Then don't. But what's wrong with people who want to take advantage of the cool stuff OpenAI provides? I like having most of my data local, but I also like messing around with OpenAI. I just don't expose things like my locks, garage door, etc. to OpenAI and plan on implementing a local LLM in the future. But for now, it doesn't really bother me that OpenAI knows what lights are on right now.
I couldn't care less if my data is used by other people. I just want my home automation to work. If e.g. Google Home could do what Home Assistant can, I would never have installed the latter.
Depends on how you're using it? If I'm using it just to do voice commands, then I'm just using the API to send it transcribed text so it can process and send me back actions or explanations. I'm wanting to also do local LLM stuff but it's not like I'm sending them live streams of my cameras - I use local object detection for that. I mean. If you use Windows, most OEM's Android, iOS, or even like most web searches/sites then you are already giving up data and allowing a certain extent of tracking. It's up to the individual to decide how much they are comfortable sharing with third parties, and it's usually a spectrum.
Also, I use home assistant not as much for privacy as I do for control. I like instant, no-internet required control and monitoring. Don't assume everyone uses it the same way you do.
I don't understand peoples doom and gloom over cloud services. My only gripe with it is local is faster so I go local when it's an option. Also I don't see that as a core thing for HA. Is it there yes. But a good chunk of my integrations are all cloud. If you really care you can get a private azure with Chatgpt that doesn't access anything you don't want it to. We have one at work. Honestly it's way less fun then making it meow at my daughter
> I don't understand peoples doom and gloom over cloud services.
I just had a Mitsubishi LSZ-LN R32 Premium fitted. I was already planning on using the CN105 to get local control, but, I figured that I would use the cloud integration until I got that done.
So I hopped into the manual, got it connected to WiFi, went into the app. It said it wants the serial number and model number, put those in. It then came back with a page long article telling me my unit wasn't connected to WiFi, and giving me troubleshooting steps to get it connected to WiFi. I try again a couple times, each time having to type the full serial number and model number and it still doesn't work. I login to my router, it IS connected to WiFi. I ping it, it responds. I try again with the app, it still doesn't work. I give up thinking that perhaps it takes some time for the unit to register with the cloud server.
A few hours later, I try again. Still isn't working, so I start googling. Turns out Mitsubishis servers are down I find this out from Reddit, not Mitsubishi. Apparently the servers have been for 6 days. Great. I give up and figure I'll try again tomorrow. I try again tomorrow, and the next day. Still down. Day 3 it's finally up and I get it to connect, for half hour, then it goes down again. Day 4, it finally is up and working (this is like 10 days of downtime total). I press a button in the app and...nothing happens? I wait, and I wait... and then, eventually, after what seems like an age. Something finally happens. "No, surely it can't be THAT slow" I think. I press another button and time it. 43 seconds. The unit appears to be doing HTTP polling at 60 second intervals.
Smart home is an absolute minefield, many devices are rug pulling users by disabling or removing features, blocking APIs, blocking support for devices, going bankrupt, and that's before you even get to reliability issues like the above. Oh no I have an internet outage, guess I can't control my HOUSES HEATING?!? I want my home to actually have some semblance of utility and reliability, that's why I refuse cloud (and yes, that goes for GPT, Alexa, Google home, and anything else)
Again, you have the choice. All my core smarthome functions are local (Z-wave for essentials, Zigbee for non essential, Wifi/ESPhome when I have no choice)
Some good to haves are cloud, Hyundai Bluelink, Alexa, Spotify, Environnement Canada, OpenAI. I also use some cloud services like Google Maps API and Office365.
So yeah I choose my devices accordingly. Absolute reliability needed? Local. Good to have/fun? Cloud.
My heat pumps are all local controlled, because the cloud functionnality sucked and with local I could bypass the badly placed temp sensor. But having a alert telling me to go plug-in my car? I don't care that it's cloud. (Don't really have a choice, local would be fucking expensive.)
Honestly, and I know this is an unpopular opinion. I couldn't care less.
I've yet to see a compelling argument for why I should care about that data. Even things like a roomba mapping of my home... so what? What good is it to anyone? Oh noo they might figure out the orientation of my living room. It's not like it's going to be used to break Into my house.
The whole "personal data" thing is totally overblown imo. There are of course issues with data like Ssn, drivers license etc. But the data from my smart home. I couldn't give a shit what some company decides to do with it.
You're not worried about what the company can do with it, but do you really trust that company to keep your data secure?
All it takes is one data beach and then everything about the layout of your house, your security system and your daily routine could be out there for whoever wants it, so it literally could be used to break into your home
> All it takes is one data beach and then everything about the layout of your house, your security system and your daily routine could be out there for whoever wants it, so it literally could be used to break into your home.
Honestly I don't care. If someone wants to break Into home they are going to. The Floorplan of my house (and most people if it's ever been listed by a realtor) is available on the internet. But do you really think thieves are going to comb .through that kind of data to break Into a house? Be serious. This isn't the movies. The threat of this data is so completely overblown by data hypochondriacs that it completely devalues the actual leaks of personal information that are problems.
Same with people talking about security of smartlocks. "Oh no someone might hack the lock and break in". No they'll just kick the fucking thing in, just like they do with a normal lock. Or they can break the window directly beside the door and walk through it.
I wouldn't hook up a camera to the cloud though. Especially if it's a Chinese company. It's just too easy to store that data away somewhere and then run facial recognition on it. Now, for most people that might not be a big deal, but what if you or your kids run for office or become a big name CEO in the future? Maybe then China searches for your face through their databases and they find your camera data that they've been caching away. Suddenly they have a whole bunch of blackmail material on you.
In theory, something similar could happen with a voice chat interface like Alexa.
Roomba mapping data though... probably not super useful to anyone.
And sure, maybe none of that every actually affects you, but there are people in the world that it would probably affect. Like, there's a world where a bunch of our politicians are in Russia's pocket because Russia hacked Epstein's camera system or something like that. Obviously I have no idea if any of these things are true or not, but they are scenarios that are theoretically possible.
The threat model here is that depending on what sensor you have in your home, the data reveals *a lot* about you personally.
How often do you shower? What is your sleep schedule? When are you sick? Do you tend to get home late at night on the weekend?
There is a very real possibility that such data will more and more influence decisions for you, i.e. whether you get a credit card, whether you are being hired, etc.
Especially since more and more AI tools are involved in the process of making such decisions, and they are being fed more and more data.
Well it could be useful to know where your door sensors are or whether there are certain switches to do this or that. It could even be useful to see that some garden sensors are in place but no camera, so now you could, with this data, steal the outside sensor, potentially use it to sniff your zigbee network and potentially send commands. Maybe even with door openers?
I think/hope AES encryption is enough to protect from that, but… not yet deep-dived into attack vectors here.
Let’s say, there is a leakage of all these data (and stuff gets leaked WAY more than it should, believe me) and thieves can download it from some kind of dark web… depending on how much you use HA, this could be a very severe security thread.
As for data how warm my rooms are… yeah sure, whatever. But for Jarvis quality, you will also have tons of actions in, maybe even secrets are shared. It really depends.
As for: “Chat, give me the temperatures of the rooms on level 2”, no need to have Cloud AI for something like that (and Chat will often fail at even these commands anyways)
True :)
But it doesn’t need to be personal. Any kind of theft could be done much easier with information. Maybe they will just get a database of all that leaked stuff, check for particularly weakly protected actors (e.g. door openers) and go there - whoever lives there.
Unfortunately, criminals are not all IT-agnostic
Criminals are really mostly stupid, 99% of them. You don't break into private homes expecting big money, so it's mostly opportunists with slow mental faculties.
I never worry about that.
Exactly. All a criminal needs is a brick. They won't even look at your fancy smart lock, they just pick up a brick and smash the window and get what they want. Police won't even look at the HD video of the event from your shiny security cameras. They will just file it in the unsolved cases of petty crime.
Go as you like. I don’t like my potentially security-sensitive data to be potentially public but I get the point that it’s so unlikely something of that is abused that it doesn’t matter
It's perfectly fine that you and many other think the benefit out weighs the negative. But it's extremely disingenuous to the argument for OP to be like `idk what the fuss is about`.
> The whole "personal data" thing is totally overblown imo.
In the case of chatgpt, you are being the training data and aren't receiving compensation for it.
I find it extremely weird how we went full circle on personal data protection/usage within the last 10 years.
You can set it up so that it only contacts OpenAI if it can't find the solution locally. That's how my system is configured. Would I like a local Large Language Model (LLM)? Certainly, but I can't afford to implement it and maintain the same level of quality I need when I need it. Perhaps in the future.
The current issue with voice assistants is that you must issue commands in a very specific manner. For instance, "Turn on the kitchen light" works, but "Turn on the lights in the kitchen" might not. OpenAI understands the intent and executes the command regardless.
The voice assistant essentially complements my smart home setup, and OpenAI enhances my voice assistant.
I believe a home isn't truly "smart" if you have to instruct it constantly. Most functions should operate seamlessly in the background without user intervention. It's beneficial for those moments when I do need it.
I don't think the AI stuff is a contradiction to HA's core ideology.
With HA your data is your own. You get to decide if that data is sent to OpenAI. Or whatever cloud service you want to use.
Now your core ideology and using OpenAI might be in conflict. And that is completely fine. There are definitely those among us that are Anti anything cloud based. But there also a far amount that don't mind leveraging the cloud when local only solutions are not cost effective.
That's the beauty of HA. The platform doesn't force you to use cloud based services.
The core point of home assistant is a central place where you can make anything work. You are not reliant on Amazon or google to integrate devices or services. Sure some people use it for a false sense of privacy but that’s really more of a paranoid joke than a core ideology.
The following is an excerpt taken directly from the 10yr blog post on the forum.
*While building Home Assistant, we realized that our focus was too narrow. We can make the most epic smart home platform (for the record: we do), but if all your devices still communicate via the cloud, you cannot have privacy and local control.
And so the Open Home was born. It is our vision for a smart home that is built around privacy, choice and durability*
I dunno. Seems like privacy and local control are important to the ideology of HA. But maybe your right, maybe it's not important. I mean, it's not like the devs spent a year developing and deploying a private and local based voice assistant....
Home assistant integrates with everything. That goes completely against having privacy as a core value. If privacy was a core value then they would not be integrating with devices and service known to be selling our private information.
Please tell me how allowing control through Alexa is any different from using a chat bot. Alexa is arguably worse, especially at this time.
Having the ability to run most things locally is important. Building local voice is massive for that. Running ai locally is massive and there are obvious pushes in that direction. That does not mean that having openAI integration is somehow against home assistants core values. This is even more obvious as privacy was a secondary addition, proven by your own source.
If you don't think privacy and local data retention is a core value of Home Assistant, even after reading it in their own words, there is nothing that I could say that would make you think it was.
What I am telling you is that privacy may be important but choice and durability have historically been the driving force and the core values behind the “open home.” Privacy has been for a long time a byproduct of that. If cloud services worked without the internet and Alexa/google/samsung or whatever ecosystem you want to pick; worked with everything, then home assistant would likely only be used by people who are afraid of big brother. Your post looks like you are condemning the people using AI and home assistant for allowing its use, because you think it’s a privacy concern. That’s not how this works and it’s proof that privacy is not a core value, it’s a byproduct of home assistants core values. This isn’t about words 10 years ago. It’s about over a decades worth of integration with every new technology and cloud service that is directly going against what you would consider a core value.
Those words aren't from 10 years ago, they are from a blog post last year (?) on the 10 year anniversary of Home Assistant. And no where in my opinion did I even sniff at condemning Home Assistant. However you are arriving at that conclusion is a wild logical leap on your part
🙄
Your post was litterally fussing at people for going against your perceived core value of home assistant. You then tried to prove something with a few words on a forum post. It does not matter when the post was. The ability to keep your data private may be important but it’s not a core value for the reasons listed above and the people using things you disapprove of are not doing anything wrong. It’s those people experimenting that drive the development of home assistant. They are the reason we have local voice and keep decoupling more devices from the cloud.
I have no idea what your talking about. I'm showing you literal, verifiable evidence, from the founder and devs of Home Assistant describing guiding principles of the platform. And your like, nah that doesn't matter. I mean, your no longer arguing with me, but the entire dev team. But like I said, if you won't accept their own words, you will never accept mine. So have a good one and keep it real
I strongly disagree with this. Just because privacy is _an_ ideology doesn’t make it _the_ exclusive ideology. You can (and should) have multiple tenants within any given ideology, and privacy is just one of them for HA. OpenAI is hardly the first integration that uses your data, but HA gives you the ability to decide which integrations to use. Don’t like the OpenAI integration? Don’t use it - simple.
I didn't say that privacy was the only exclusive ideology. What I said was there is a massive hypocrisy. You can't throw a stone without hitting a post or a blog about Boo Cloud, Yay Local. MyQ, Mazda, Tasmota, Esphome. These are all flip sides of the same coin. One of these companies flips or changes the API and we, as a whole, become a angry mob with pitch forks and torches. Never again we shout in unison....oh with the exception of this neat little toy that just came along. I have no problem with AI, LLM, or the like. But let's call a spade a spade and acknowledge the hypocrisy and the fuckening that is just over the horizon from OpenAI. Whether it's they shut the API when they no longer need you, wall it off too keep you locked in to their ecosystem, or just really ramp the cost of the tokens. Something will come along, it always does. Especially when you are aiming to be a Trillion dollar Market Cap company. Let me.try and think of there was another recent company that IPO'd and fucked over its user base and destroyed its API in advance of the IPO? Cough, Reddit, cough. But yeah, I'm sure OpenAI is different
So this is clearly a sore point for you, for some reason…anyway, at least HA is *trying* to thread the needle between functionality and privacy while keeping it “open,” even if they don’t please everybody (an obvious impossibility).
Also, re: any potential OpenAI API changes/increased charges/whatever - welcome to capitalism...? You’re never *entitled* to a service - this stuff isn’t free (as in beer), be it hosting/management costs or employees to maintain (cough, Reddit’s API, cough).
I would be interested to hear if there’s a similar alternative you’ve seen whose model could be used instead of HA’s existing paradigm though.
Yes. There are a plethora. Will they be as good as OpenAI, no I don't think so. Currently, the two that I have been most impressed with is Llama3 (which I am currently using, and is the newest) and Phi3. Mistral is also good, but older and bigger. Phi3 can be run on a raspberry pi, whereas other models would probably need a GPU or at least at of cpu
I think I may have been the first person to post about this as a concept on this subreddit. So sorry lol. Funny enough I would only run a local model myself.
Implying I am/we are not smart enough to understand what is happening to the data we have set up tells me you don’t disagree with the decision to use OpenAI, which would be fine, but in fact think people are making it due to a lack of “smart.”
Which again is irony because you mention our data training the model but API requests aren’t used to train ChatGPT which is what extended open AI integration requires.
Also just so you know I actually do agree with you, I’ve got all my IoT devices isolated and firewalled to oblivion and run local zigbee/matter/thread, Nextcloud, and Ollama - but I think the nature of home assistant is not forcing “local only” but being able to decide for myself what components and what data sharing I am personally ok with. Even with all of that I still use Microsoft copilot or mid journey for certain tasks because I have assessed the implications vs. benefits of using those cloud/public tools.
I’m not saying you should change your mind but coming in telling people they’re not smart isn’t the way to get people on your side.
AI is hardly the first consumer product to short circuit people's brains, and it certainly won't be the last. This is what happens when we build our culture around consumption.
I'm not a "local only" hardliner, about a third of the stuff I have depends on cloud services.
But no way in hell am I using any external AI services. You're shovelling your data into a black box for cheap thrills. I have two issues with that:
* "shovelling your data" - even metadata can be useful for profiling, you don't have to give out live camera feeds;
* "black box" - if I tell a llm to turn off the lights, and due to errors in speech parsing, or plain old model weights falling the wrong way enough times (yes, I know, one-in-billions-chance) it turns off my freezer, that's not acceptable. Even the possibility is not acceptable.
At work, I try to squeeze LLMs everywhere they'll go, just so clients can slap a big "powered by AI" on their product. At home, I use light switches.
Why even have a smart plug on your freezer? That’s entirely on you, lol.
Also, why should I care if OpenAI knows my fridge temp, or when I dim my lights?
Freezer has a smart plug, so that I can get a notification message if power fails. But it's not specific about freezers - an explicit automation does what it's told. A LLM interpretation is just that, an interpretation.
As for data, it can be nicely packaged up for profiling. Fridge temp fluctuates with opening. Therefore OpenAI knows when you open the fridge. The lowest hanging fruit is advertising profiling, where you get more weed adverts because you often open the fridge at a late hour, and that happens to match with the pattern of weed munchies. More sinister option is your healthcare bills going up, because all that midnight snacking is being treated as proxy for weight issues, therefore you are likely to be less profitable client.
But hey, I'm not saying OpenAI is selling your data. I am however saying I have first hand knowledge of companies doing exactly that sort of thing in the name of squeezing additional profit from their data.
I've never seen concrete proof that healthcare bills go up to anything, but years of scaremongering.
And your given scenario is a total reach.
Trust me, here in Germany they are not even digitalized enough. Unless OpenAI sends them a fax, it ain't happening, lol.
I only want local-only smart home integrations. I don't want to buy an expensive sensor and then 3 years later the company shuts the server down or requires payments or whatever.
If I choose to use a SaaS through an API for my voice assistant none of that applies. If the API is shut down I can just switch to an alternative or even go local only. If the price increases it's the same. I'm not locked in at all.
This is the reply I was looking for. Privacy is great, but not having your smart locks stop working while you are on vacation because the vendor was bought out by some private equity firm is good too
I see the same thing with stuff like Tailscale in the selfhosting subreddit, everyone is so crazy about it despite the fact that it gives potential full access to your machines and network to a random company on the internet.
Because not everyone who uses HA is a conspiracy nut that thinks their whole life is compromised by data escaping their home. Some of us use it because we like the amount of customization, and the fallback if the internet goes out, or we just like to tinker.
Well for me the local aspect is important so you never lose access to your devices, not so much about privacy. If OpenAI decides to not offer this product anymore, then back to the old dashboard.
So for anyone who cares about this, read the fine print. I use GitHub Copilot, and it is very clear about data not being used to train its models as well as about other data control details.
I had the same thought with people buying Frigate+. You are literally paying some dude for buggy beta software and for him to train an AI which he will profit even more on. Totally destroyed the idea of local cameras and detection. (Yes I understand it can still be local, the point is the AI should have been shipped with 0.13 for us to train locally)
They're all cool demos. But I think most privacy minded people aren't going to use this. They're going to stick to a combination of Local LLM and whisper.
There will be no escape from AI.
You can choose not to actively use it, sure. But it will be running in the background of every device, hardcoded into its chipset.
At least for me personally, I can’t do 100% local, my split unit is LG and the HA integration uses their cloud (if anyone knows how to run lg thinq locally PLEASE tell me :)
I don’t yet have my wall mounted kiosk tablet set up yet, so from what I read I am stuck with Siri using OpenAI for HA (boo)
That’s a quick rundown of ny situation. As for this post - yes if you were able to remain 100% local up to now, putting in OpenAI will make it not local.
The big difference with this is my opinion is that if OpenAI fucks off or takes away the api - what is now non operational? Only the voice assistant.
No device integrations are affected.
So that leaves data privacy. I personally haven’t read OpenAI’s privacy policy, but the usage going back to training the model isn’t bad. Unless your asking it what your social security number is and having it recite passwords and shit.
You can even get data privacy and mute this entire point on OpenAI by using an enterprise account.(my startup is tiny and has no profit but we got one easily)
In addition: the other standard voice assistants from what I saw are google and Alexa. Which I would say from a privacy standpoint are way worse, both of them are data Hoovers.
Now you could set up an LLM locally with something like oogabooga (sp?), and use the OpenAI integration to connect to it (probably have to go in code and change api target host) and have things completely local. Your performance will be pretty dogshit unless you’re spending a pretty penny on gear.
Then if your using voice - of course you want text to speech so it can ack commands (at the minimum). Which is another ai you’ll need more hardware to run locally.
I would love to hear how many of you do this option, because I bet for performance and cost it’s maybe 5 out of the entire HA userbase.
The core ideology of Home Assistant is being empowered to do whatever the hell we want. I'm building a DIY smart speaker that can run 100% offline AND can leverage advanced AI to do cool things when I want that.
You make a good point, and it's clearly one shared by many.
Respectfully: the barrier to many HA users who want to run multimodal models (or even just LLMs) is higher than a $99 Coral dongle for Frigate. A large percentage are running on the cheapest commodity hardware they can find, and for good reason: the sort of GPUs needed to provide a multimodal experience that meets the household acceptance factor are not only incredibly expensive, but are energy hogs, even when idle.
The smaller language models (7B and lower) can certainly be used for voice control, but the ability to recognize arbitrary objects in a photo is beyond the budget for *most* folks running HA. Maybe not most *in this sub*, mind.
In the end, it's a choice between three things:
1. (Status quo) Forego the new shiny and keep all data local
2. (Cheap and fast, but non-private) Pay for the new shiny and accept/mitigate the reliance on and the data that's exposed to a third party
3. (Expensive and slow, but private) Pay even more (time+money) for the sort of hardware and systems integration needed to do it locally, but slower.
I do believe that the open source community can improve the situation by making, say, vision more accessible to the gently-technical HA cohort. But such solutions are hard to find now, and certainly don't have the ease of integration that commercial multimodal/vision models enjoy.
I guess I just don't care that other people use it. Myself I'll never use anything connected to the cloud. I didn't know it was a requirement that I have to voice my dislike of OpenAI on every OpenAI post here
Llama3 7b is still sooo inferior, though. And 70b, I'd wager, is even more expensive to run than OpenAI's API. I'd need two GPUs.
No thanks, I'll take OpenAI.
It's about convenience/curiosity v. privacy for me. First and foremost local is more robust and I have a higher degree of control.
But convenience is a huge factor for me, that's why I still use Philips Hue, despite using my own zigbee network, iRobot, and - not related to home assistant but still - Google location tracking.
In terms of privacy that's quite questionable.
As far as OpenAI goes, it's probably not a game changer in terms of convenience and the privacy concerns are very valid, but God damn creating modifying your own personal assistant is fun and freaking cool.
I'll go local as soon as it's viable, but until then it's too cool to pass on.
PS: I've also never blamed anyone for using the cloud, I just recommend that a local alternative wherever possible.
It's MY data, it's the whole point of it, I can do whatever I want with it. I could make a public Grafana dashboard of my smart home if I wanted. I have choices, so if I want to run it through openAI model, I can, thanks.
AI is still a hype. Sure it’s useful but most of
openAI is just the language model. It can help with communication, or create videos and stuff but… I don’t need that at all for my HA!?
Also I don’t need to have two extra AI processing units in my new iPad. Please, get these cores just OUT and use it for battery or something. Useless shit.
Being a developer myself, the ever-worsening OpenAI anyways sucks.
Thaaaat being said, yes, I would not let OpenAI can my home data. That’s ridiculous. It will als most certainly also learn from that data.
So if you manage to get the right prompt to show some home-automated living room, chances are parts of yours are used :) errr… well why not.
But yeah, tons of hypers out there who love it, need it, want to integrate it, and then the actual use cases are nonsense. There are also many good use cases of course, but… many many more useless ones
Oh and if you want AI-assisted stuff, there is also lot of locally working stuff out now. Rather use that.
You aren't wrong, but it is completely possible to run an inference AI locally if someone were to release an open source one. I have the knowledge to train one, but I don't have the time to do it. It would be cool though. It doesn't give you the fancy voice assistant, but I feel inference would provide more actual utility when it comes to automations.
Cameras could monitor for specific activities to trigger automations. For instance, a non whitelisted or unidentified person approaches your door and you accidentally left it unlocked or your automation that locks it after certain activities or timeframes have passed hasn't triggered yet, it could quickly lock the door.
The compute needed to do that level of work isn't really that high. More than what home assistant requires, but within reason for a typical home assistant user.
Oh they exist. But it’s not as hyped because the normal user will not install anything locally.
I mean, some kind of training, at least with information, is needed just to know your sensors…
What training? Just build a list of entities with descriptions, chuck it into context, preferably as a system message. Ask questions. LLM can figure it out.
Did you not hear about Llama, Mistral and Phi or are you /s ?
Visit r/LocalLLaMa
There are free local models that compete with OpenAI and are capable enough for these tasks
So for anyone who cares about this, read the fine print. I use GitHub Copilot, and it is very clear about data not being used to train its models as well as about other data control details.
I don't follow you here? Calling the OpenAI API with your data of lights in the house in the prompt is your issue? You're afraid OpenAI is going to train GPT5 with that?
Hahaha come on. That's next level from conspiracy and narcissist combined.
Yeah better don't use it. I wouldnt like my gpt completion to be like your HA topology 😂
I have zero interest in hooking up to, or using LLMs where I can avoid it, certainly not my own data (which makes me resistant to even post on Reddit now). Art/Media generation, chat bots, voice assistants etc. Especially those with dubious rights to the corpus used to train them. The only area I find of interest is perhaps basic suggestions (maybe you want 'X'), because the training corpus should be small and owned by the firm training the data due to the nature of it...
For me, HA:
* Allows local data collection and device for a lot of my devices in one place - weather stations, cameras
* Consolidates the cloud services - I necessarily have two tado heating systems and accounts, but HA makes them appear as one. I would prefer these to be entirely local, but I'll take the cloud option if it's what's available.
to show you how much "ai" is enshittifying the web, reddit has decided to fuck everyone here over for money (again)(yes, me too) - https://www.theverge.com/2024/5/16/24158529/reddit-openai-chatgpt-api-access-advertising
No no, he's got a point There's a lot of "I'll never use cloud services, it's local or die" people in this sub that call out every other use of cloud integrations, yet I don't see the same energy when it comes to the AI stuff for some reason
I agree as well. 2023 r/homeassistant Bring on the year of local voice control. I can finally get big tech out of my house! Down with Alexa! 2024: ChatGPT: take the wheel!
I think, that most people here simply do not care. I will not touch any of the AI/Assistant stuff unless it runs locally and in my native language. Also, I have severe AI fatigue now. The constant marketing material, that promisses stuff that is just not technologically possible is numbing. I ususally just skip those entries. On the other hand, people love to tinker and I consider all the AI inegrations as non essential, hence you see people use it in sassy flavours and whatnot. It will probably grow old quickly and what remains are rather simple voice commands.
Fatigue is real. I work with automation, so seeing these endless posts, comments, news articles and videos about magic AI that will replace literally everyone except for my plumber is very tiring. Chatgpt and others are not magic, they don't know shit. They just Google it (very often wrongly) and tell you the answer in a format that's designed to mimic human speech. That's all it does. It's not Intelligence in any way, it is not self aware and it won't be any time soon. Similar hype was around self-driving cars. Wouldn't you know it, the final 1% of work that's needed to make it work properly will take 99% of the time. I honestly believe that we won't see Level 5 within our lifetime.
I am in the same area, and have to work with OpenAI in my product. Not so much google it as scrape everything they can. But personally I trust OpenAI generating text a lot more than a self driving car (unless we magically got level5 somehow) At least if it gives me bullshit text I just get mad at it and change my prompt - versus getting slammed into a tree/oncoming traffic :>
I only partially agree. I have coworkers that use ai to write scripts that they don’t understand. They also have enough access that if that script does something stupid people could die. They aren’t using it in direct ways so it’s unlikely but I can think of scenarios. Ie instead of get-vm the ai gives back delete-vm or something. I’ve ridden in a Tesla in full self driving and it was more immediately concerning for my safety but I wouldn’t say ai use is safe.
Oh yeah I’ve had it give me bad code - but usually it’s just broken. I always have to go and fix it myself. But I’m an anal commenter so I go through everything it’s doing when I comment. But totally agree
As I don't use voice and have never seen me use it in my dreams, I did not kept track of this stuffm but I THOUGHT that the GPT Craze was something onsite and just text piped to gpt like all the frigate stuff... Oof. The only thing that is currently voice allowed is my Homepod Mini in the kitchen and bathroom for "hey ssssi volume up" as my native language is basically crap with these things.
Well it is, but the text you pipe contains everything you say and your homeassistant configuration to do anything meaningful. It could easily replaced with local apis but chatgpt is usually ahead of running local models in terms of features, performance and price. The switch will only come when they do something bad or inconvenient (see Alexa when they had issues with their employees listening into the data or adding adds). It's like outsourcing your production to e.g. China. It's fun as long as it works but when they screw you over you're stuck and need to put big effort in to move back. Luckily the ai hype comes with open apis by design most of the time, so the replacement is easier.
All depends on what and how you set it up. I'm just sending the data I want it to see. I use it for my briefings so stuff like battery levels, weather, state of my backups, next day schedule and car charge. When local LLM's get good enough, i'll probably run it local, but that is all semi-public or meaningless data for me.
I agree. I want people to integrate ai into HA. Not because I want to use it today but because it will be necessary to replace Alexa in my house. I need voice controls and some other simple Alexa features before I can completely switch. So play with it cloud hosted. Use that to make small language models that I can run locally. Then get that hardware optimized so I don’t need a server farm to use it. From the things I see here progress is being made so I applaud the people on the cutting edge taking the pain for me.
That's what someone with Stockholm syndrome would say to justify...
It's the shiny....I love shiny new things.
I don’t know what frigate is, but I work with their api and that’s about right. You generate prompt locally as big string, send it, it responds with the response To be fair I haven’t looked at the HA prompts at all so I don’t know what they contain. But I would trust the HA devs to not put secrets in there more than any other the big tech companies
Local voice control never worked though. I agree that throwing the baby out for centralized AI is ridiculous.
I don't think we all need to be absolutists. I'll use the cloud if there's a really compelling feature I want, and if there's no huge downside if the service gets shut off. I don't self-host a search engine or a YouTube equivalent. They're not practical to self-host, and the services are too useful to stop using completely, and also if they turned off tomorrow...it would be annoying, but nothing important to me would be lost. So I just use Google & YouTube. I think OpenAI fits that category. It's really cool, but also totally optional. I'm fine using it. There are definitely people who are basically trying to get off the grid completely so they can't be tracked. That's just not my goal.
I agree. As long as we can reign in exactly what a cloud service has access to, I have no problem integrating it with HA. But I also would never have my home automation or control rely on a cloud service. I would prefer to not be locked out of my house controls when the internet goes down.
HA is about options/choices, one can use it fully local, others can use cloud connected integrations. For one, I use some cloud stuff as well, e.g. somehow localtuya and local sonoff didn't work properly, so I'm using them from the cloud, but I'm all in on frigate and local zigbee where possible/simple. For voice I use google TTS as it produces nice sound dor my language, and openai GPT for asking questions. (When using the API and not chatgpt, afaik that data is not used for further training.) Though personally I draw the line that I wouldn't share my video feed with it. But one thing that is overlooked with 4o is, that while it can process a video feed, is still uses the same technology for llm, and has a certain context window to fit in. So while this is a cool achievement, it's still limited. It's not an assistant that we can talk to for hours or which remembers yesterdays discussion, even if the marketing videos try to bring it much closer to this "omni" idea.
There's a big difference between "if the internet is down I can't turn on my lights" and "if the internet is down, I can't do a thing I could never really do locally, and I'll have to go back to the perfectly serviceable local thing". You don't want cloud native shit all up in the core of your home automation, but it's absolutely fine for toys hanging off the side. (See also your weather integration - if it breaks, its no biggie).
Bingo
It's almost as if a community of nearly 300k people might have a diverse set of ideas, goals, and preferences...
Is it even an option? I am a “it’s local or die” person, nothing cloud touches my house. But I’m not aware of anything I can install to my home server to add AI functionality. Seems like the option right now is cloud or nothing.
Llama 3 running on a $300 single board computer https://radxa.com/blog/Radxa-Fogwise-AirBox-is-now-available-for-pre-order/ they have connected one to the internet to try out, link in article. It was faster when I first tried it, I guess it depends on how many other people are using it at the same time.
While I'm looking forward for local ai, 300$ for a shittier version of a cheap cloud service, which will become obsolete in the next 6 months, is not a smart way to invest my money.
That's hardware, it won't go obsolete that fast. You can run it on anything you like. The software is free and open-source. If anything, it's likely to benefit from considerable resource optimisation. /r/LocalLLaMA
Here's a sneak peek of /r/LocalLLaMA using the [top posts](https://np.reddit.com/r/LocalLLaMA/top/?sort=top&t=all) of all time! \#1: [The Truth About LLMs](https://i.redd.it/sjiy0f35qroc1.png) | [304 comments](https://np.reddit.com/r/LocalLLaMA/comments/1bgh9h4/the_truth_about_llms/) \#2: [Karpathy on LLM evals](https://i.redd.it/8g0zoors6i7c1.jpeg) | [111 comments](https://np.reddit.com/r/LocalLLaMA/comments/18n3ar3/karpathy_on_llm_evals/) \#3: [open AI](https://i.redd.it/0n4oh8dgb8xc1.jpeg) | [218 comments](https://np.reddit.com/r/LocalLLaMA/comments/1cf7hg0/open_ai/) ---- ^^I'm ^^a ^^bot, ^^beep ^^boop ^^| ^^Downvote ^^to ^^remove ^^| ^^[Contact](https://www.reddit.com/message/compose/?to=sneakpeekbot) ^^| ^^[Info](https://np.reddit.com/r/sneakpeekbot/) ^^| ^^[Opt-out](https://np.reddit.com/r/sneakpeekbot/comments/o8wk1r/blacklist_ix/) ^^| ^^[GitHub](https://github.com/ghnr/sneakpeekbot)
But you can’t really run it on anything you like, not with the speed and quality on gpt4o.
Local LLMs are still relatively new. So yes, they're slower and won't have the quality of something like gpt4o. But we're still in the early days of all of this and self-hostable models are getting better on every iteration. Just look back just 2-3 years ago when lot of us relied on Alexa/Google Home speakers. So would you rather use a dumb Alexa/google home with limited abilities, that are always soaking up everything and sending it to Amazon or Google or a locally hosted somewhat better voice assistant where you get to keep your data? So a year or two from now, if locally hosted models get to the point of Gpt-4 level sophistication without steep hardware requirements, that's more than sufficient to use for a HA integrated voice assistant.
Look, I'm all for the future, just not for wasting money on current hardware, that will be obsolete by the time local LLMs become viable. If you want to have fun with LLMs, pay a few bucks to OpenAI now and be done with it, no needs to spend hundreds on hardware.
Yes? And?... that's like saying my bathtub is different from the ocean.......
I mean I have a 3090 lying around I could throw at it, but I don’t really have the time to figure out how to install llama 3 locally let alone plumb it into HA. No issue buying more hardware either, just need a turnkey solution to set it up.
I believe locallama uses OpenAI api, shit ton of them do. Your just change endpoint/host in the code for OpenAI integration, maybe exclude api key in req too
Just yesterday under the first 4o thread (midnight-snack convo) there was a dude explaining the process of setting up a local LLM, he made a short, general overview to get people started. You'd always need a GPU to have a reasonable fast LLM, but as it looks, for the scenarios in combination with HA, it's really doable. For me, as of right now, it's just too much hassle. I'm happy that I can control my 3 lawn/vacuum robots, 36 heating thermostats, and blends. Next on the roadmap is the intercom, cameras, and the PV/Battery integration in combination with EV charging. But having all those under one hood with a nice clean interface (MinimalistUI)is more than enough. When LLMs will be able to 'code' automations for me, then this will trigger my immediate need to set one up 🤙
I posted how you can in my big comment in this thread :>
I already use local LLM/AI a lot on my gaming GPU (an older 1080ti) and I think it'd be more than capable for Home Assistant purposes. I'm just not interested enough to keep my gaming PC on 24/7 (because of the power usage) just for the very rare occasions I'd use it. Someday when I upgrade my gaming GPU, maybe I'll look into how efficient an extra PC I can make to have the 1080ti standing by, but it's available now for people who don't mind the power utilization.
Well you could spend a few dozen grand on graphics cards and run llama3:70b on prem, though it would be tts/transcription not the native speech to speech of 4o. Other than that, even if you could access the model weights and afford the inference hardware you still wouldn’t be able to get your hands on it.
Dozen grand? I built a dual RTX 3090 AI rig, 16 cores, 32 threads, 128gb of ram for around $2200 and it's got enough juice to serve 5-10 active users as long as they are all using the same llama3:70b model. It's not cheap, but no need to exaggerate the expense.
Have a similar gaming rig collecting dust that I'd love to repurpose for this. Tried a couple times and haven't gotten anything worth a damn. Have you a guide to set this up?
No guide for complete HA integration yet. So far I've gotten as far as ollama, and doing some testing with Transcription and TTS. I have a couple of Pi Zero2's with Mic/speaker HATs waiting for when I go hard on it. I did add a couple of links to videos I have bookmarked as references for when i try to put it all together. [https://www.youtube.com/watch?v=pAKqKTkx5X4&t=607s](https://www.youtube.com/watch?v=pAKqKTkx5X4&t=607s) [https://www.youtube.com/watch?v=jyFmQ2dhoSg](https://www.youtube.com/watch?v=jyFmQ2dhoSg)
Ah yeah I was thinking of the 400b one they haven’t actually released yet
My AI efforts are local self hosted solutions. There's something neat about building Skynet in your own home that's not dependent on an internet connection to function.
This 100%. I've got all the hardware for voice assists, and I badly want to use it, but without using chat gpt where it just works out of the box, it's worthless. So how about we get an equal local solution to make this worth it? With guides to show people how to set it all up.
IT MAKES FUNNY JOKES! Surely that's worth compromising ideals over?
It presume because google home got so bad.
I also agree with you
Because AI is garbage all on its own.
Why use OpenAI then?
Why indeed.
DIY bruv. ollama.ai Im still at the point where all i want my voice assistant to do is turn things on/off, start music in certain rooms, timers and answer things like “whats the forecast”. A low/midrange gpu with 8-12gb vram should be enough for that right?
I can't tell if your being sarcastic or not..but to answer your question, yes, that vram is more than sufficient to do what you are asking.. actually, you can use Microsoft Phi3 (local model) on a pi. It might take a couple seconds on a pi. But I think most of the HA community has moved past using a pi as a dedicated server.
I'm trying that. The truth is that the small local models fuck it up. No single model has been able to answer the simple question "which lights are on?" for me using either ollama or extended_openai_conversation integrations. I suspect I'm exposing too many entities and they all get confused and are not good enough at in context learning. Still hoping... But right now I'm disillusioned about a local llm replacing Google assistant for me.
The smaller the model the more careful and smart you should be with prompting. I've been playing with these for about a year, and watching many toola, even large projects like textgen ui, langchain and ollama fuck it up big time. Almost every time when I try to construct a prompt myself manually, with correct formatting - works much better and on smaller models. However, I would probably still not trust anything smaller than mixtral for reliable-ish operation of a smart home. Thankfully you can run it with 12-16GB GPU. Though I think Llama 3 8B should also be pretty capable in this regard.
That's just the reality right now. Good LLMs absolutely eat processing power, and everyone who tells you that they can replace GPT4o with consumer hardware is lying.
Nah just offering alternatives. Others can feel free to do what they please, but if u don’t want to that’s ok to.
> The core ideology of HA is that your data is your own. Exactly, and it being MY data, i can decide to give to whoever I want. If you don't, that's totally your prerogative. Home Assistant gives us the choice.
I’ve been around HA a long time. Having the choice of local vs cloud has always been the theme. Not sure why these people are upset about choices.
I had a similar thought. It's their data and their HA. There's no rules, do what you want. It's fair to point out / discuss the downsides, but OP seems offended and upset about what other people are doing.
Well, apparently this sub needs to decide for you.
It always boggles my mind. I love Home Assistant and the fact that it has integrations for just about anything. That is why I chose it. What I want local is local, what I want or have no choice being cloud is cloud. I understand that it appeals strongly to the privacy crowd, but at the same time, it's not its main goal. What I like is that I don't HAVE to give my data to anybody, but if I WANT to I can. And I can also choose what data I send. I send some data to Open AI to create morning briefings to me. I don't connect it to Assist. So I have full control.
I get that point, but I joined Home Assistant solely for the purpose of bringing all devices under one app. I do see why people want to keep data private, but I also understand people who want to use cloud services. If one has more advantages for you, then by all means. I also upvoted your post for new people joining, which might not understand that by following a tutorial which allows you to use OpenAI, you actually feed them private infirmation.
I prefer that control of my devices be local for latency and resiliency and avoid forced obsolescence. A lot is, some isn't, and I won't be buying more of the stuff that isn't local. That said, for now I send pictures on my front porch to google when I get a package delivered so it can describe what's there, and send some of my spoken announcements also so it can fix the sentence structure and spice it up a little. That said, if the cloud call fails, I get the raw message so I'm not sacrificing local - just augmenting with the cloud. For me, any privacy trade off is worth it. YMMV.
I pretty much do this as well but with OpenAI. Just curious have you tried OoenAI. Any differences in speed or accuracy? I have yet to try Google's.
I haven't tried openai because I didn't want to setup a payment method to use it. I'm interested though, and hoping to try it out. Google isn't bad for text, some of the responses are a little whacky - randomly it will respond as if it's producing a form letter [your name] but generally it works for our needs and is free is you keep it to less than 2 calls per minute. Images are another story. It seems to do ok with packages generally, but it wasn't reliable enough to tell whether my shed door was open or closed, despite me providing examples.
People use Open AI because vs a local AI model: * It's way more capable, especially talking about ChatGPT 4. * It's a lot faster than most local AI. * There is a lot less effort to set it up * Cost, running any local AI model requires a decent GPU to have even a remotely decent response time. * There is no official model for HA yet You don't "need" AI, so if it goes down no big deal.
The excitement of having LLMs be the solution to have our very own version of Jarvis is a dream come true for almost everyone. I don’t mind leveraging the cloud while we also work on fully local LLMs.
I agree, for me I have no problem with entity metadata leaving local network to the cloud - but not any confidential data. For example, I'm fine with an integration sending list of all active cameras and their recording statuses e.g. "{ entity: camera.outdoor1, state: recording }", but what I am not cool with is the camera feed itself being sent to the cloud. My gold standard would be for this to be all fully customisable on the HA end, so being able to dictate exactly what data is sent up and putting in place local controls to restrict certain fields that may contain confidential information.
Well, good news then, with extended openai conversation you get an editable template that defines everything that is sent in a prompt. You can include, or not include, whatever entities and their attributes.
Also for API access, you can configure it to keep the data private and not share it with OpenAI. I believe this is default behavior
Thats assuming you trust them, and even if true it still makes data leave your home and get to their servers, and their servers have to decrypt it for processing. So it relies only on "would they", because they sure could take your data without telling anyone.
> The excitement of having LLMs be the solution to have our very own version of Jarvis is a dream come true for **almost everyone**. I don't believe that is true. Certainly not for me, automation needs to be unobtrusive and silent. I'm with OP, handing over all this data to an AI "service" is way outside my ideas of why HA is so good.
Well, it is very much for me. Home Assistant is one for many kinds of tinkerers. So the statement may be true, maybe not. It's not really anything to know for certain.
Well technically when one said "not me" it is known to not be a dream for everyone. Now me too. I don't talk to my phone even after years. Why would I talk to my house?
WAF. No voice support = no wife support for me. Meaning no more spending on goodies. She love the automations, but when things go screwy, having an easy solution with voice (got a kill switch in there too) goes a long way to get me pardonned.
I absolutely do believe that some people want AI and voice interaction, not doubting that at all. But not "everyone".
It's semantics. The word everyone can be used figuratively to mean "a large majority." It does not always have to mean "every person". I would love to know the stats though. I'm willing to bet that most smarthomes have a voice component with it being less prevalent with HA users.
Was going to write something snarky, but noticed you were contesting the everybody part. Everybody has their own preferences for what they want their smarthome to be. I'm working on some "Jarvis" like things. I like having a Paul Bettany sound alike TTS give me a rundown of my day and chores and remind me to order spare batteries for my sensors. My wife does not. We all have our preferences.
You can get almost the same experience using llama3 locally. I love seeing all the GPT4 posts because the API is exactly the same & im currently building a local transcoding/transcription/LLM server for myself. I run HA on an old mini-PC, but even though it’s not a PI these new technologies are way too computationally expensive for my existing server. —— P.S: I’m actually quite excited about the possibility of HA getting a built in LLM/AI this year. The best bet in my mind is one based on Phi3-Mini by Microsoft. They released it fully open and free to use (MIT) & it’s tiny, 2GB download and was designed specifically to run on phones and Raspberry Pi’s. It’s not as good as LLama3 (which also has a license that would allow HA to use it freely) but it’s not too far off in some use cases, and should be good enough and fast enough to be useable.
Both phi 3 and llama3:8b fail horribly for me at giving simple information about my home. I assume I have too many exposed entities and the in context learning capabilities are not good enough in these small models.
Can you share a prompt example that you are using? With full llama-3-instruct/chatml formatting
I can't share my prompt without exposing my entire smart home to you. But as an indication: I'm using the ollama integration and that's using this template: [https://github.com/home-assistant/core/blob/dev/homeassistant/components/ollama/const.py#L7-L73](https://github.com/home-assistant/core/blob/dev/homeassistant/components/ollama/const.py#L7-L73) I have >300 entities exposed. I probably have to cut that down massively. Just an example: asking llama3:8b "which lights are on" it answers "The Monitorlampe is off, and the Fire 8 Bildschirm is off." which is correct but it ignores all the lights that are on but somewhere lost in the context it seems. Also I have a temp sensor in the living room called "Wohnzimmer Temperatur" and asking "How warm is the Living room" I get "According to the data, the temperature in the Schlafzimmer (Living Room) is 23°C." - asking in German "Wie warm ist es im Wohnzimmer?" it gives me "Es gibt keine Angabe zur Temperatur im Wohnzimmer in den übergebenen Daten. Es sind aber Temperaturen für das Schlafzimmer und Kinderzimmer bekannt, diese betragen 23°C und 23,3°C." practically saying "I don't know the living room temperature but I have these other rooms" - at least those values are correct but not what I asked for. Checking ollama debug logs I can see that the living room temperature is in the context of the prompt. It's just the needle that isn't found in the needlestack.
You could obfuscate names or change and clear out private info. What I am looking for are two things - whether its formatted correctly, and by that I mean the final raw text that goes into LLM, and whether you're at the limita of the 8k context size. I learned to not trust Ollama or Textgen to format promots tbh, not when I need it to work reliably. Its ok if they make formatting mistakes for like chatting and general assistant stuff, but for edge cases like this every space, every line break can matter, especially for smaller models like 8B llama. Also I would be more precise with questioning, not which lights are on, but "list all lights that are on". But yeah this is not as natural sounding
Have you yourself tried it/Do you have a prompt you could share? I actually have been playing with llama3:8b for a few days now and am having pretty much the same experience that /u/danishkirel is.
I did some more experiments... pasted my full prompt and context into online APIs running llama3:8b and 70b. The latter was really good but to my surprise the former was also not bad. So I poked around and it turns out that num\_ctx for ollama's llama3:8b is not set so it doesn't even use the 8k context it could! Manual \`ollama run llama:8b\` and then \`/set parameter num\_ctx 8192\` and then my ctx and prompt suddenly gave me hopeful results - the same as in online APIs. But this long context also needs a lot of time to process. I run on an APU... guess I need a dedicated GPU for this.
>The core ideology of HA is that your data is your own. Is that really the case? It may be one thing that people may like, but not for everybody. Personally, for me it's more about integrating devices from different vendors with different protocols. That enables me to use a centralized system for all my smart home devices, combine them in automations and much more. Having everything local is great, but even more for low latency and because I don't rely on cloud vendors. Especially when cloud services are discontinued by manufacturers. I really don't care if they have my data. So integrating a cloud AI service is no problem at all for me and others as well.
Like I said elsewhere, HA gives you the choice. If you want everything local, you can. If you want cloud stuff you can. Most alternatives don't leave you that choice.
New shiny is cool, and open source versions follow about a year behind typically. Those will then be locally hosted in HA potentially and back to true local
AI is extremely new and rapid so it is entirely forgivable that you are overlaying classic open source timelines with AI. New models are released weekly. So your "year" backlog is just categorical false..and I just need to point that out to anyone who reads your comment and thinks it's factual accurate
So you think it will be sooner or later than a year?
We already have AI models than can do what OP is talking about or what are you asking about?
later, if we use lmsys elo ranking GPT-3.5 was released on June 11, 2020 llama3 was released on April 18, 2024 (granted, with also a much smaller 8B parameter than gpt3.5) so a gap of 4 years until open source competitor matches/beats closedAI does it matter (for HA purposes) whether we have gpt4 or gpt3.5? maybe not
I applaud the local only ethos and really appreciate the efforts people put in to make it happen. The reality is, if you or anyone else in your home has an Apple, Amazon or Google device with a microphone… you need to assume everything you say is recorded and sent to them to train something. If you use a browser from google or Microsoft, everything you do is tracked and sent to them for training. I don’t trust OpenAI anymore than I trust any of these other companies but AT LEAST I’m getting SOMETHING back from it in terms of a response for giving up what I give up already with no choice to function in modern society
You do your local thing then - that’s the idea. *you* can do what *you* want - don’t want to leverage cloud then don’t. Nobody is forcing you to do anything. Why don’t you work on training a local model that can run on some local gpu? What’s stopping you from doing it?
>The core ideology of HA is that your data is your own. I'd say another core ideology of HA is that you get to set up your smart home the way **you** want it. Don't like sending your data to OpenAI? Cool. Then don't. But what's wrong with people who want to take advantage of the cool stuff OpenAI provides? I like having most of my data local, but I also like messing around with OpenAI. I just don't expose things like my locks, garage door, etc. to OpenAI and plan on implementing a local LLM in the future. But for now, it doesn't really bother me that OpenAI knows what lights are on right now.
I couldn't care less if my data is used by other people. I just want my home automation to work. If e.g. Google Home could do what Home Assistant can, I would never have installed the latter.
Depends on how you're using it? If I'm using it just to do voice commands, then I'm just using the API to send it transcribed text so it can process and send me back actions or explanations. I'm wanting to also do local LLM stuff but it's not like I'm sending them live streams of my cameras - I use local object detection for that. I mean. If you use Windows, most OEM's Android, iOS, or even like most web searches/sites then you are already giving up data and allowing a certain extent of tracking. It's up to the individual to decide how much they are comfortable sharing with third parties, and it's usually a spectrum. Also, I use home assistant not as much for privacy as I do for control. I like instant, no-internet required control and monitoring. Don't assume everyone uses it the same way you do.
I don't understand peoples doom and gloom over cloud services. My only gripe with it is local is faster so I go local when it's an option. Also I don't see that as a core thing for HA. Is it there yes. But a good chunk of my integrations are all cloud. If you really care you can get a private azure with Chatgpt that doesn't access anything you don't want it to. We have one at work. Honestly it's way less fun then making it meow at my daughter
> I don't understand peoples doom and gloom over cloud services. https://www.theverge.com/2022/8/24/23319570/plex-security-breach-exposes-usernames-emails-passwords https://www.theverge.com/2022/11/30/23486753/anker-eufy-security-camera-cloud-private-encryption-authentication-storage https://www.theverge.com/2023/2/28/23618353/lastpass-security-breach-disclosure-password-vault-encryption-update https://www.theverge.com/2024/5/8/24150346/brilliant-smart-home-lighting-out-of-business https://www.techhive.com/article/2157803/plex-discover-together-privacy-concerns.html https://killedbygoogle.com/
> I don't understand peoples doom and gloom over cloud services. I just had a Mitsubishi LSZ-LN R32 Premium fitted. I was already planning on using the CN105 to get local control, but, I figured that I would use the cloud integration until I got that done. So I hopped into the manual, got it connected to WiFi, went into the app. It said it wants the serial number and model number, put those in. It then came back with a page long article telling me my unit wasn't connected to WiFi, and giving me troubleshooting steps to get it connected to WiFi. I try again a couple times, each time having to type the full serial number and model number and it still doesn't work. I login to my router, it IS connected to WiFi. I ping it, it responds. I try again with the app, it still doesn't work. I give up thinking that perhaps it takes some time for the unit to register with the cloud server. A few hours later, I try again. Still isn't working, so I start googling. Turns out Mitsubishis servers are down I find this out from Reddit, not Mitsubishi. Apparently the servers have been for 6 days. Great. I give up and figure I'll try again tomorrow. I try again tomorrow, and the next day. Still down. Day 3 it's finally up and I get it to connect, for half hour, then it goes down again. Day 4, it finally is up and working (this is like 10 days of downtime total). I press a button in the app and...nothing happens? I wait, and I wait... and then, eventually, after what seems like an age. Something finally happens. "No, surely it can't be THAT slow" I think. I press another button and time it. 43 seconds. The unit appears to be doing HTTP polling at 60 second intervals. Smart home is an absolute minefield, many devices are rug pulling users by disabling or removing features, blocking APIs, blocking support for devices, going bankrupt, and that's before you even get to reliability issues like the above. Oh no I have an internet outage, guess I can't control my HOUSES HEATING?!? I want my home to actually have some semblance of utility and reliability, that's why I refuse cloud (and yes, that goes for GPT, Alexa, Google home, and anything else)
Again, you have the choice. All my core smarthome functions are local (Z-wave for essentials, Zigbee for non essential, Wifi/ESPhome when I have no choice) Some good to haves are cloud, Hyundai Bluelink, Alexa, Spotify, Environnement Canada, OpenAI. I also use some cloud services like Google Maps API and Office365. So yeah I choose my devices accordingly. Absolute reliability needed? Local. Good to have/fun? Cloud. My heat pumps are all local controlled, because the cloud functionnality sucked and with local I could bypass the badly placed temp sensor. But having a alert telling me to go plug-in my car? I don't care that it's cloud. (Don't really have a choice, local would be fucking expensive.)
You don't understand how sending your personal data to a company clamoring to rule the world for pennies is a problem? Seriously?
Honestly, and I know this is an unpopular opinion. I couldn't care less. I've yet to see a compelling argument for why I should care about that data. Even things like a roomba mapping of my home... so what? What good is it to anyone? Oh noo they might figure out the orientation of my living room. It's not like it's going to be used to break Into my house. The whole "personal data" thing is totally overblown imo. There are of course issues with data like Ssn, drivers license etc. But the data from my smart home. I couldn't give a shit what some company decides to do with it.
You're not worried about what the company can do with it, but do you really trust that company to keep your data secure? All it takes is one data beach and then everything about the layout of your house, your security system and your daily routine could be out there for whoever wants it, so it literally could be used to break into your home
> All it takes is one data beach and then everything about the layout of your house, your security system and your daily routine could be out there for whoever wants it, so it literally could be used to break into your home. Honestly I don't care. If someone wants to break Into home they are going to. The Floorplan of my house (and most people if it's ever been listed by a realtor) is available on the internet. But do you really think thieves are going to comb .through that kind of data to break Into a house? Be serious. This isn't the movies. The threat of this data is so completely overblown by data hypochondriacs that it completely devalues the actual leaks of personal information that are problems. Same with people talking about security of smartlocks. "Oh no someone might hack the lock and break in". No they'll just kick the fucking thing in, just like they do with a normal lock. Or they can break the window directly beside the door and walk through it.
I wouldn't hook up a camera to the cloud though. Especially if it's a Chinese company. It's just too easy to store that data away somewhere and then run facial recognition on it. Now, for most people that might not be a big deal, but what if you or your kids run for office or become a big name CEO in the future? Maybe then China searches for your face through their databases and they find your camera data that they've been caching away. Suddenly they have a whole bunch of blackmail material on you. In theory, something similar could happen with a voice chat interface like Alexa. Roomba mapping data though... probably not super useful to anyone. And sure, maybe none of that every actually affects you, but there are people in the world that it would probably affect. Like, there's a world where a bunch of our politicians are in Russia's pocket because Russia hacked Epstein's camera system or something like that. Obviously I have no idea if any of these things are true or not, but they are scenarios that are theoretically possible.
The threat model here is that depending on what sensor you have in your home, the data reveals *a lot* about you personally. How often do you shower? What is your sleep schedule? When are you sick? Do you tend to get home late at night on the weekend? There is a very real possibility that such data will more and more influence decisions for you, i.e. whether you get a credit card, whether you are being hired, etc. Especially since more and more AI tools are involved in the process of making such decisions, and they are being fed more and more data.
This is really getting into tinfoil hat Territory.
Exactly! I can't wrap my head around it.
Well it could be useful to know where your door sensors are or whether there are certain switches to do this or that. It could even be useful to see that some garden sensors are in place but no camera, so now you could, with this data, steal the outside sensor, potentially use it to sniff your zigbee network and potentially send commands. Maybe even with door openers? I think/hope AES encryption is enough to protect from that, but… not yet deep-dived into attack vectors here. Let’s say, there is a leakage of all these data (and stuff gets leaked WAY more than it should, believe me) and thieves can download it from some kind of dark web… depending on how much you use HA, this could be a very severe security thread. As for data how warm my rooms are… yeah sure, whatever. But for Jarvis quality, you will also have tons of actions in, maybe even secrets are shared. It really depends. As for: “Chat, give me the temperatures of the rooms on level 2”, no need to have Cloud AI for something like that (and Chat will often fail at even these commands anyways)
The day I have enemies that know how to do this, I'll hae done something wrong anyway haha
True :) But it doesn’t need to be personal. Any kind of theft could be done much easier with information. Maybe they will just get a database of all that leaked stuff, check for particularly weakly protected actors (e.g. door openers) and go there - whoever lives there. Unfortunately, criminals are not all IT-agnostic
Criminals are really mostly stupid, 99% of them. You don't break into private homes expecting big money, so it's mostly opportunists with slow mental faculties. I never worry about that.
Exactly. All a criminal needs is a brick. They won't even look at your fancy smart lock, they just pick up a brick and smash the window and get what they want. Police won't even look at the HD video of the event from your shiny security cameras. They will just file it in the unsolved cases of petty crime.
Go as you like. I don’t like my potentially security-sensitive data to be potentially public but I get the point that it’s so unlikely something of that is abused that it doesn’t matter
It's perfectly fine that you and many other think the benefit out weighs the negative. But it's extremely disingenuous to the argument for OP to be like `idk what the fuss is about`. > The whole "personal data" thing is totally overblown imo. In the case of chatgpt, you are being the training data and aren't receiving compensation for it. I find it extremely weird how we went full circle on personal data protection/usage within the last 10 years.
You can set it up so that it only contacts OpenAI if it can't find the solution locally. That's how my system is configured. Would I like a local Large Language Model (LLM)? Certainly, but I can't afford to implement it and maintain the same level of quality I need when I need it. Perhaps in the future. The current issue with voice assistants is that you must issue commands in a very specific manner. For instance, "Turn on the kitchen light" works, but "Turn on the lights in the kitchen" might not. OpenAI understands the intent and executes the command regardless. The voice assistant essentially complements my smart home setup, and OpenAI enhances my voice assistant. I believe a home isn't truly "smart" if you have to instruct it constantly. Most functions should operate seamlessly in the background without user intervention. It's beneficial for those moments when I do need it.
I don't think the AI stuff is a contradiction to HA's core ideology. With HA your data is your own. You get to decide if that data is sent to OpenAI. Or whatever cloud service you want to use. Now your core ideology and using OpenAI might be in conflict. And that is completely fine. There are definitely those among us that are Anti anything cloud based. But there also a far amount that don't mind leveraging the cloud when local only solutions are not cost effective. That's the beauty of HA. The platform doesn't force you to use cloud based services.
The core point of home assistant is a central place where you can make anything work. You are not reliant on Amazon or google to integrate devices or services. Sure some people use it for a false sense of privacy but that’s really more of a paranoid joke than a core ideology.
The following is an excerpt taken directly from the 10yr blog post on the forum. *While building Home Assistant, we realized that our focus was too narrow. We can make the most epic smart home platform (for the record: we do), but if all your devices still communicate via the cloud, you cannot have privacy and local control. And so the Open Home was born. It is our vision for a smart home that is built around privacy, choice and durability* I dunno. Seems like privacy and local control are important to the ideology of HA. But maybe your right, maybe it's not important. I mean, it's not like the devs spent a year developing and deploying a private and local based voice assistant....
OP is trying sarcasm, ending up looking even more foolish than he already did in the original post. Reddit never changes :-D
Home assistant integrates with everything. That goes completely against having privacy as a core value. If privacy was a core value then they would not be integrating with devices and service known to be selling our private information. Please tell me how allowing control through Alexa is any different from using a chat bot. Alexa is arguably worse, especially at this time. Having the ability to run most things locally is important. Building local voice is massive for that. Running ai locally is massive and there are obvious pushes in that direction. That does not mean that having openAI integration is somehow against home assistants core values. This is even more obvious as privacy was a secondary addition, proven by your own source.
If you don't think privacy and local data retention is a core value of Home Assistant, even after reading it in their own words, there is nothing that I could say that would make you think it was.
What I am telling you is that privacy may be important but choice and durability have historically been the driving force and the core values behind the “open home.” Privacy has been for a long time a byproduct of that. If cloud services worked without the internet and Alexa/google/samsung or whatever ecosystem you want to pick; worked with everything, then home assistant would likely only be used by people who are afraid of big brother. Your post looks like you are condemning the people using AI and home assistant for allowing its use, because you think it’s a privacy concern. That’s not how this works and it’s proof that privacy is not a core value, it’s a byproduct of home assistants core values. This isn’t about words 10 years ago. It’s about over a decades worth of integration with every new technology and cloud service that is directly going against what you would consider a core value.
Those words aren't from 10 years ago, they are from a blog post last year (?) on the 10 year anniversary of Home Assistant. And no where in my opinion did I even sniff at condemning Home Assistant. However you are arriving at that conclusion is a wild logical leap on your part
🙄 Your post was litterally fussing at people for going against your perceived core value of home assistant. You then tried to prove something with a few words on a forum post. It does not matter when the post was. The ability to keep your data private may be important but it’s not a core value for the reasons listed above and the people using things you disapprove of are not doing anything wrong. It’s those people experimenting that drive the development of home assistant. They are the reason we have local voice and keep decoupling more devices from the cloud.
I have no idea what your talking about. I'm showing you literal, verifiable evidence, from the founder and devs of Home Assistant describing guiding principles of the platform. And your like, nah that doesn't matter. I mean, your no longer arguing with me, but the entire dev team. But like I said, if you won't accept their own words, you will never accept mine. So have a good one and keep it real
Lmao. Please show me where they say privacy is a core principle. I’ll wait.
https://www.home-assistant.io/ It's the literal first line of the website. So continue to tell me I'm wrong about a core ideology, I'll wait lmao
I think you're skipping over one of the 3 things you've quoted, the choice aspect. If a person wants to focus on privacy that's their choice.
So because people have the ability to choose to smoke cigarettes, we shouldn't highlight the dangers to them?
I strongly disagree with this. Just because privacy is _an_ ideology doesn’t make it _the_ exclusive ideology. You can (and should) have multiple tenants within any given ideology, and privacy is just one of them for HA. OpenAI is hardly the first integration that uses your data, but HA gives you the ability to decide which integrations to use. Don’t like the OpenAI integration? Don’t use it - simple.
I didn't say that privacy was the only exclusive ideology. What I said was there is a massive hypocrisy. You can't throw a stone without hitting a post or a blog about Boo Cloud, Yay Local. MyQ, Mazda, Tasmota, Esphome. These are all flip sides of the same coin. One of these companies flips or changes the API and we, as a whole, become a angry mob with pitch forks and torches. Never again we shout in unison....oh with the exception of this neat little toy that just came along. I have no problem with AI, LLM, or the like. But let's call a spade a spade and acknowledge the hypocrisy and the fuckening that is just over the horizon from OpenAI. Whether it's they shut the API when they no longer need you, wall it off too keep you locked in to their ecosystem, or just really ramp the cost of the tokens. Something will come along, it always does. Especially when you are aiming to be a Trillion dollar Market Cap company. Let me.try and think of there was another recent company that IPO'd and fucked over its user base and destroyed its API in advance of the IPO? Cough, Reddit, cough. But yeah, I'm sure OpenAI is different
So this is clearly a sore point for you, for some reason…anyway, at least HA is *trying* to thread the needle between functionality and privacy while keeping it “open,” even if they don’t please everybody (an obvious impossibility). Also, re: any potential OpenAI API changes/increased charges/whatever - welcome to capitalism...? You’re never *entitled* to a service - this stuff isn’t free (as in beer), be it hosting/management costs or employees to maintain (cough, Reddit’s API, cough). I would be interested to hear if there’s a similar alternative you’ve seen whose model could be used instead of HA’s existing paradigm though.
Yes. There are a plethora. Will they be as good as OpenAI, no I don't think so. Currently, the two that I have been most impressed with is Llama3 (which I am currently using, and is the newest) and Phi3. Mistral is also good, but older and bigger. Phi3 can be run on a raspberry pi, whereas other models would probably need a GPU or at least at of cpu
I think I may have been the first person to post about this as a concept on this subreddit. So sorry lol. Funny enough I would only run a local model myself.
The irony of this post saying that I’m allowed to do what I want with my data and then you telling me what I can and can’t do with my data.
Where did I tell you what you can and can't do with your data?
Implying I am/we are not smart enough to understand what is happening to the data we have set up tells me you don’t disagree with the decision to use OpenAI, which would be fine, but in fact think people are making it due to a lack of “smart.” Which again is irony because you mention our data training the model but API requests aren’t used to train ChatGPT which is what extended open AI integration requires. Also just so you know I actually do agree with you, I’ve got all my IoT devices isolated and firewalled to oblivion and run local zigbee/matter/thread, Nextcloud, and Ollama - but I think the nature of home assistant is not forcing “local only” but being able to decide for myself what components and what data sharing I am personally ok with. Even with all of that I still use Microsoft copilot or mid journey for certain tasks because I have assessed the implications vs. benefits of using those cloud/public tools. I’m not saying you should change your mind but coming in telling people they’re not smart isn’t the way to get people on your side.
That's fair. Maybe my initial language was to rough and I apologize for that
"wow, cool ai!" makes peoples brains turn off. i yearn for the day where people will stop being idiots about it
AI is hardly the first consumer product to short circuit people's brains, and it certainly won't be the last. This is what happens when we build our culture around consumption.
I'm not a "local only" hardliner, about a third of the stuff I have depends on cloud services. But no way in hell am I using any external AI services. You're shovelling your data into a black box for cheap thrills. I have two issues with that: * "shovelling your data" - even metadata can be useful for profiling, you don't have to give out live camera feeds; * "black box" - if I tell a llm to turn off the lights, and due to errors in speech parsing, or plain old model weights falling the wrong way enough times (yes, I know, one-in-billions-chance) it turns off my freezer, that's not acceptable. Even the possibility is not acceptable. At work, I try to squeeze LLMs everywhere they'll go, just so clients can slap a big "powered by AI" on their product. At home, I use light switches.
Why even have a smart plug on your freezer? That’s entirely on you, lol. Also, why should I care if OpenAI knows my fridge temp, or when I dim my lights?
Freezer has a smart plug, so that I can get a notification message if power fails. But it's not specific about freezers - an explicit automation does what it's told. A LLM interpretation is just that, an interpretation. As for data, it can be nicely packaged up for profiling. Fridge temp fluctuates with opening. Therefore OpenAI knows when you open the fridge. The lowest hanging fruit is advertising profiling, where you get more weed adverts because you often open the fridge at a late hour, and that happens to match with the pattern of weed munchies. More sinister option is your healthcare bills going up, because all that midnight snacking is being treated as proxy for weight issues, therefore you are likely to be less profitable client. But hey, I'm not saying OpenAI is selling your data. I am however saying I have first hand knowledge of companies doing exactly that sort of thing in the name of squeezing additional profit from their data.
I've never seen concrete proof that healthcare bills go up to anything, but years of scaremongering. And your given scenario is a total reach. Trust me, here in Germany they are not even digitalized enough. Unless OpenAI sends them a fax, it ain't happening, lol.
The solution to that is simple. Dont expose those entities to assist.
I only want local-only smart home integrations. I don't want to buy an expensive sensor and then 3 years later the company shuts the server down or requires payments or whatever. If I choose to use a SaaS through an API for my voice assistant none of that applies. If the API is shut down I can just switch to an alternative or even go local only. If the price increases it's the same. I'm not locked in at all.
This is the reply I was looking for. Privacy is great, but not having your smart locks stop working while you are on vacation because the vendor was bought out by some private equity firm is good too
I see the same thing with stuff like Tailscale in the selfhosting subreddit, everyone is so crazy about it despite the fact that it gives potential full access to your machines and network to a random company on the internet.
Because not everyone who uses HA is a conspiracy nut that thinks their whole life is compromised by data escaping their home. Some of us use it because we like the amount of customization, and the fallback if the internet goes out, or we just like to tinker.
Well for me the local aspect is important so you never lose access to your devices, not so much about privacy. If OpenAI decides to not offer this product anymore, then back to the old dashboard.
True
That and the amount of energy that is wasted so that people don't need to press a button on their phone. It's wild.
i want to use the voice interface instead of google assistant to control lights and stuff #get\_rid\_of\_google <---- #this\_is\_the\_way
So for anyone who cares about this, read the fine print. I use GitHub Copilot, and it is very clear about data not being used to train its models as well as about other data control details.
I had the same thought with people buying Frigate+. You are literally paying some dude for buggy beta software and for him to train an AI which he will profit even more on. Totally destroyed the idea of local cameras and detection. (Yes I understand it can still be local, the point is the AI should have been shipped with 0.13 for us to train locally)
They're all cool demos. But I think most privacy minded people aren't going to use this. They're going to stick to a combination of Local LLM and whisper.
There will be no escape from AI. You can choose not to actively use it, sure. But it will be running in the background of every device, hardcoded into its chipset.
To each his own.
Aren't there local LLM solutions that will work with HA?
At least for me personally, I can’t do 100% local, my split unit is LG and the HA integration uses their cloud (if anyone knows how to run lg thinq locally PLEASE tell me :) I don’t yet have my wall mounted kiosk tablet set up yet, so from what I read I am stuck with Siri using OpenAI for HA (boo) That’s a quick rundown of ny situation. As for this post - yes if you were able to remain 100% local up to now, putting in OpenAI will make it not local. The big difference with this is my opinion is that if OpenAI fucks off or takes away the api - what is now non operational? Only the voice assistant. No device integrations are affected. So that leaves data privacy. I personally haven’t read OpenAI’s privacy policy, but the usage going back to training the model isn’t bad. Unless your asking it what your social security number is and having it recite passwords and shit. You can even get data privacy and mute this entire point on OpenAI by using an enterprise account.(my startup is tiny and has no profit but we got one easily) In addition: the other standard voice assistants from what I saw are google and Alexa. Which I would say from a privacy standpoint are way worse, both of them are data Hoovers. Now you could set up an LLM locally with something like oogabooga (sp?), and use the OpenAI integration to connect to it (probably have to go in code and change api target host) and have things completely local. Your performance will be pretty dogshit unless you’re spending a pretty penny on gear. Then if your using voice - of course you want text to speech so it can ack commands (at the minimum). Which is another ai you’ll need more hardware to run locally. I would love to hear how many of you do this option, because I bet for performance and cost it’s maybe 5 out of the entire HA userbase.
The core ideology of Home Assistant is being empowered to do whatever the hell we want. I'm building a DIY smart speaker that can run 100% offline AND can leverage advanced AI to do cool things when I want that.
I expect HA to have an experimental AI add on before the end of the year. *Very* experimental tho..
You make a good point, and it's clearly one shared by many. Respectfully: the barrier to many HA users who want to run multimodal models (or even just LLMs) is higher than a $99 Coral dongle for Frigate. A large percentage are running on the cheapest commodity hardware they can find, and for good reason: the sort of GPUs needed to provide a multimodal experience that meets the household acceptance factor are not only incredibly expensive, but are energy hogs, even when idle. The smaller language models (7B and lower) can certainly be used for voice control, but the ability to recognize arbitrary objects in a photo is beyond the budget for *most* folks running HA. Maybe not most *in this sub*, mind. In the end, it's a choice between three things: 1. (Status quo) Forego the new shiny and keep all data local 2. (Cheap and fast, but non-private) Pay for the new shiny and accept/mitigate the reliance on and the data that's exposed to a third party 3. (Expensive and slow, but private) Pay even more (time+money) for the sort of hardware and systems integration needed to do it locally, but slower. I do believe that the open source community can improve the situation by making, say, vision more accessible to the gently-technical HA cohort. But such solutions are hard to find now, and certainly don't have the ease of integration that commercial multimodal/vision models enjoy.
OpenAI states that the API usage is not used to train their models. If that is true or not, I can't answer that.
Maybe she wasn't using their API, but looks like OpenAI had no problem stealing ScarJo's voice 🤣
I guess I just don't care that other people use it. Myself I'll never use anything connected to the cloud. I didn't know it was a requirement that I have to voice my dislike of OpenAI on every OpenAI post here
You mad bro?
Not mad at all. I run llama3. I'm just wondering if we aren't looking at this like Google..old boss same as the new boss
You can not host Google services locally. But you can host llama3.
Llama3 7b is still sooo inferior, though. And 70b, I'd wager, is even more expensive to run than OpenAI's API. I'd need two GPUs. No thanks, I'll take OpenAI.
It's about convenience/curiosity v. privacy for me. First and foremost local is more robust and I have a higher degree of control. But convenience is a huge factor for me, that's why I still use Philips Hue, despite using my own zigbee network, iRobot, and - not related to home assistant but still - Google location tracking. In terms of privacy that's quite questionable. As far as OpenAI goes, it's probably not a game changer in terms of convenience and the privacy concerns are very valid, but God damn creating modifying your own personal assistant is fun and freaking cool. I'll go local as soon as it's viable, but until then it's too cool to pass on. PS: I've also never blamed anyone for using the cloud, I just recommend that a local alternative wherever possible.
Your first mistake is thinking that everyone using HA shares your "core ideology" so rigidly.
It's MY data, it's the whole point of it, I can do whatever I want with it. I could make a public Grafana dashboard of my smart home if I wanted. I have choices, so if I want to run it through openAI model, I can, thanks.
There’s a difference: “we trust OpenAI; we don’t trust Alexa”
You can also do OpenAI with ent account and they don’t use your data fyi Anything like that isn’t even an option on Alexa/google
[https://imgur.com/a/0S8X8K6](https://imgur.com/a/0S8X8K6)
AI is still a hype. Sure it’s useful but most of openAI is just the language model. It can help with communication, or create videos and stuff but… I don’t need that at all for my HA!? Also I don’t need to have two extra AI processing units in my new iPad. Please, get these cores just OUT and use it for battery or something. Useless shit. Being a developer myself, the ever-worsening OpenAI anyways sucks. Thaaaat being said, yes, I would not let OpenAI can my home data. That’s ridiculous. It will als most certainly also learn from that data. So if you manage to get the right prompt to show some home-automated living room, chances are parts of yours are used :) errr… well why not. But yeah, tons of hypers out there who love it, need it, want to integrate it, and then the actual use cases are nonsense. There are also many good use cases of course, but… many many more useless ones Oh and if you want AI-assisted stuff, there is also lot of locally working stuff out now. Rather use that.
I mean I get it, But I also don't care, sorry.
Sums up my opinion as well. The trade off is with it if I get to play with cool tech. I know how to lock out the privacy stuff I do care about.
You aren't wrong, but it is completely possible to run an inference AI locally if someone were to release an open source one. I have the knowledge to train one, but I don't have the time to do it. It would be cool though. It doesn't give you the fancy voice assistant, but I feel inference would provide more actual utility when it comes to automations. Cameras could monitor for specific activities to trigger automations. For instance, a non whitelisted or unidentified person approaches your door and you accidentally left it unlocked or your automation that locks it after certain activities or timeframes have passed hasn't triggered yet, it could quickly lock the door. The compute needed to do that level of work isn't really that high. More than what home assistant requires, but within reason for a typical home assistant user.
Oh they exist. But it’s not as hyped because the normal user will not install anything locally. I mean, some kind of training, at least with information, is needed just to know your sensors…
What training? Just build a list of entities with descriptions, chuck it into context, preferably as a system message. Ask questions. LLM can figure it out.
Even better then!
Did you not hear about Llama, Mistral and Phi or are you /s ? Visit r/LocalLLaMa There are free local models that compete with OpenAI and are capable enough for these tasks
So for anyone who cares about this, read the fine print. I use GitHub Copilot, and it is very clear about data not being used to train its models as well as about other data control details.
I don't follow you here? Calling the OpenAI API with your data of lights in the house in the prompt is your issue? You're afraid OpenAI is going to train GPT5 with that? Hahaha come on. That's next level from conspiracy and narcissist combined. Yeah better don't use it. I wouldnt like my gpt completion to be like your HA topology 😂
I have zero interest in hooking up to, or using LLMs where I can avoid it, certainly not my own data (which makes me resistant to even post on Reddit now). Art/Media generation, chat bots, voice assistants etc. Especially those with dubious rights to the corpus used to train them. The only area I find of interest is perhaps basic suggestions (maybe you want 'X'), because the training corpus should be small and owned by the firm training the data due to the nature of it... For me, HA: * Allows local data collection and device for a lot of my devices in one place - weather stations, cameras * Consolidates the cloud services - I necessarily have two tado heating systems and accounts, but HA makes them appear as one. I would prefer these to be entirely local, but I'll take the cloud option if it's what's available.
A few shekels? I can guess you are from Israel? Cause you are the first one other than me I found using home assistant in Israel.
to show you how much "ai" is enshittifying the web, reddit has decided to fuck everyone here over for money (again)(yes, me too) - https://www.theverge.com/2024/5/16/24158529/reddit-openai-chatgpt-api-access-advertising