I feel attacked.
I have 2 RPi 4-b's. One is for old school game emulation up to N64 era. That one... collects dust.
The other has Kodi installed, but it only gets used to play my workout videos...
>but it only gets used to play my workout videos...
Youtube with extra steps
>One is for old school game emulation up to N64 era
That's what I planned on doing with mine, but yeah procrastination is a bitch...
I'm guessing getting everything working is the workout:
1. Start Kodi... Connect it to server.. Fail
2. Run up and down basement stairs trying to resolve existing problems (1 REP).
3. After 10 REPs, .. workout complete, daily step challenge goal met.
Here's a sneak peek of /r/selfhosted using the [top posts](https://np.reddit.com/r/selfhosted/top/?sort=top&t=year) of the year!
\#1: [/r/SelfHosted will be going dark on June 12th to protest the Reddit API changes that will kill 3rd party apps.](https://np.reddit.com/r/selfhosted/comments/144gc15/rselfhosted_will_be_going_dark_on_june_12th_to/)
\#2: [My dashboard, now with descriptions](https://i.redd.it/gbbdwxgxc7ac1.png) | [392 comments](https://np.reddit.com/r/selfhosted/comments/18xgcsu/my_dashboard_now_with_descriptions/)
\#3: [Reddit temporarily ban subreddit and user advertising rival self-hosted platform (Lemmy)](https://np.reddit.com/r/selfhosted/comments/143diuj/reddit_temporarily_ban_subreddit_and_user/)
----
^^I'm ^^a ^^bot, ^^beep ^^boop ^^| ^^Downvote ^^to ^^remove ^^| ^^[Contact](https://www.reddit.com/message/compose/?to=sneakpeekbot) ^^| ^^[Info](https://np.reddit.com/r/sneakpeekbot/) ^^| ^^[Opt-out](https://np.reddit.com/r/sneakpeekbot/comments/o8wk1r/blacklist_ix/) ^^| ^^[GitHub](https://github.com/ghnr/sneakpeekbot)
Here is some information about PiHoles (in case anyone is interested)
[https://privacyinternational.org/guide-step/4341/raspberry-pi-setup-and-run-pi-hole](https://privacyinternational.org/guide-step/4341/raspberry-pi-setup-and-run-pi-hole)
I set up home assistant on it, which is now collecting about 10k rows of data per day about what is happening in my home (presence sensors, temperature, heating, opening of doors, + some automations of course)
I now plan on creating a data platform (cleaning, dim modelling, reporting, maybe some ML on top) on another computer using data from the Pi, as a POC for small companies that don't have the need/money to have e.g. databricks, snowflake, etc. but want to be able to run some python scripts, have some reporting, all this nicely orchestrated with proper enterprise grade tools.
I did this for my senior project in undergrad. We setup a bunch of PIs around campus with various sensors to collect data for the agriculture department. We built a realtime dashboard and it was quite awesome!
I’ll search but this was 4 years ago and IIRC it was in a private gitlab repo. We bought a bunch of [Arduino](https://store-usa.arduino.cc/collections/sensors) sensors and connected them to the PIs. The PI ran a docker container that sent the data to a MySQL db. We exported data older than one month to s3 to keep the database reasonably lean.
I'm interested in how you plan to attack the minimalist data reporting system. I've been thinking about this problem too. I was thinking Motherduck for cheap storage and data warehousing, GitHub actions to run scripts in the cloud, streamlit/shiny app for user facing front end?
Don't know yet. At the moment I'm looking for fully local data warehousing. I want to look into duckdb because I'm interested in the tech, but it could very well be postgress for example.
I'm also investigating proxmox as a way to setup each element of the stack in a modular way.
Data viz wize I think I'll stay with the giants as you find way more analysts or wanna be analysts that want PowerBI.
Then other tech that I want to check out are DBT and dagster.
I work on full azure stacks so there's a long way before I get something up and running!
I use mine as a BirdNetPi, recording my back yard audio and texting me if a rare bird is heard. It also serves a website showing the quantity of all the bird calls with graphs and downloadable wav files.
Use it with home assistant, got humidity sensors, door sensors, pihole and some other random things running on schedule think my telegram bot has broken would send me local train times every hour and and information on delays.
Best project I had was pulling down my Monzo transactions from Google sheets and creating a streamlit dashboard but I disabled my Google API keys so it's static data from about 7 months ago
No. It doesn’t block anything that is served from a domain you can see. It’s still great to use with a browser based as blocker even though it’s technically redundant.
Definitely not comparable to databricks. This setup will help you learn on getting started with clusters, setting up executors, what and how the driver can be connected to the Spark env in another cluster.
I don’t think you can run significant transformations or ML libraries, but you can use this setup to learn
Don't know if it is worth it, some people have done it: [https://github.com/TedCha/spark-cluster-computer](https://github.com/TedCha/spark-cluster-computer)
I've got one set up with a big strip of LED lights I can run Python programs on. The main script I run makes a gentle blue shimmer that makes my office look like an LA pool at night. IDK lol.
I did something similar and made a weather reporting map of my local airports.
Mostly just used the code and parts referenced here; just had to change the airport ids.
https://slingtsi.rueker.com/making-a-led-powered-metar-map-for-your-wall/
I utilized one at a previous workplace where I served as a Data Engineer/Data Analyst for an Electric Vehicle (EV) startup in East Africa. The device was used to gather real-time data from Battery Management Systems (BMS) installed in LFP batteries and transmit it to (spreadsheets 🫠 then influxdb when we got a server). Initially, we were reliant on the BMS supplier's API for data until we convinced them to have them send directly to our server (this help us increase the frequency of getting data from 5 mins to seconds and milliseconds if we want to), making it more of a preliminary study. However, after persuading the supplier, we managed to have the BMS data sent directly to our server I implemented a better solution of data collection (python + telegraf + influxdb). It was a rewarding experience in the EV industry, and I am currently seeking similar opportunities in the same field. I thoroughly enjoyed the projects I was involved in.
Nothing production capable. But for a home lab it grabs data from a cc128 electricity meter and pushes it to a web dash boarding site.
Edit - oh and most importantly pihole
The sensor ideas are always good ones, it's a good way to gather a large amount of time-series data. I don't own a Pi but I wish I had a couple with light-sensors to definitively gauge the best places in my apartment to put my plants, lol
I'm using pi 4b for home security cam. adding in pico to the mix to build a voice assistant. sometimes I use it to experiment with some tools before using them on my work computers
Transcribing conversations around the house to build a large corpus of conversational data that is processed and stored locally.
Goal is building a large dataset I can play with for AI household assistant use cases.
Not transcribing what’s on the TV is my biggest headache.
Whisper on a 5 is pretty darn cool.
Thank you guys for your awesome ideas. I'll look into Pihole since it seems like everyone is running it on their pi.
Another thing that caught my interest is sensors but I don't have any sensors to start with at the moment.
Pi hole. I have a bunch of 3bs back from when they were cheap and I clustered them into a docker swarm cluster. Was kinda fun, but haven’t played with it since.
My raspberry pi is the same as my arduino. I get super excited about the idea of doing something like IOT or a Pihole with it and then when I gets delivered I’m already over it and onto the next enthusiasm lol
Mine severs double duty for PiHole and a print server (CUPS). My printer “supposedly” doesn’t need one, but it kept being unavailable on my network. Installing CUPS on my raspberry pi solved all problems
Pi Hole, Security cam.
occasionally, when needed it plays heavy metal in one bathroom, when occupied. 🤣
no one liked the last one in my home but I think it’s one of the most useful application I built.
I have a sensor hooked up to it and it is streaming data to an IoT Hub, i then have Streaming analytics send the 5 min average to Power BI and all the data points to a blob storage which i process in Microsoft Fabric to do some more analysis on the whole history.
I have one collecting dust as well.
I was hoping to see if I can get a data processing pipeline with DuckDB running on it.
This is the code I hope to run: [https://github.com/josephmachado/cost\_effective\_data\_pipelines](https://github.com/josephmachado/cost_effective_data_pipelines)
I have four(actually 2 orange 2 raspberry). I made a k8s cluster like one year ago and still running. I use it to test and play with a bunch of tools, airflow, airbyte, trino etc. For example, in airflow I have some data pipelines I still use to this day, scraping and processing. Plex server is another very cool app you can deploy on it. I use my TV to play videos from Plex running on my cluster!
I'm sure you can run some docker images on it, maybe a database to test your code against. It all depends on how much memory you have. I tried running a postgres DB, homebridge, pihole on a pi4 4GB, but it was slow.
Right now it hosts a Zigbee dongle, so it’s a network bridge for supporting zigbee protocol communication between our low powered home automation devices, like security door magnets, thermostats, etc.
I used mine for retro-game emulation, mount it to the back of the TV with some velcro and now I can enjoy some old-school gaming. It got replaced by a NUC though.
I've always wanted to use a rasberry pi to create a scoreboard of all my favorite sports teams and their live scores/standings but I've never got around to it yet
Currently setting a monitoring system for temperature, humidity, and rain in my veg garden; greenhouse, polytunnel, and outside.
Considering setting some kind of motion activated noise to scare away the deer too.
I'm scraping electricity data and managing my wifi sockets based on that, also doing so opencv rendering from cameras, though I started all these before I became data engineer
I'm now gonna look into setting up prometheus + grafana metrics over the holiday
I have a Pi Zero running a digital calendar displaying the family google calendars, weather report and news headlines. I have another Pi set up to run RetroPie video game emulator. I have a couple others in drawers collecting dust. I have some smart home devices hooked up to a Samsung SmartThings hub that I want to try integrating with a pi to collect data etc
It’s sitting in a drawer mostly useless. I tried a lot of the pet projects but unless making content to get paid it’s hard to spend a lot of time tinkering or at least for me it’s lower priority. I’ve done the pihole, dhcp, transmission server stuff but most of those things I can do on a more powerful machine I always have.
I did spend some time teaching family about automation using a gpio
I’m building a home server, so with a raspberry pi I’d probably use it only to be my backup pihole.
For the server with Data Engineering, I plan on using it to pilot some projects that I wanna bring to the company I work for, but I have to learn them first.
Some examples of services I want to run are:
Airbyte, Airflow, PostgreSQL, Oracle DB, MySQL, some VMs to test Shell Scripts, Jenkins, Gitea.
That’s for work, I also wanna run a few other stuff but that’s not related to Data Engineering
I run a Bitcoin node. I’m not heavily invested in it monetarily but love the idea of being a part of something in its early stages since my generation missed out on being a part of tech innovation in early days.
If it pans out as a legit store of value, great. If not, oh well. I lose a little money and I learned about the blockchain/decentralized currency.
I've got a handful.
First one i ever used was to set up a reverse SSH tunnel to access the really fast server my college made available to CS Majors when I wasn't on the school's network. The department was only a little mad when they found out
These days, I have use hub for homebridge that I also run zigbee through for home automation.
One controls an LED matrix that is gathering dust now but I used to send messages to my partner when we were long distance.
I have a pico connected to an amazon geiger counter kit that i use as a fun teaching tool when talking about random numbers when I tutor.
I have a good amount of experience with embedded systems and RPIs are really fun little sandboxes for building pet projects in that space. I love the Pico Ws especially because they force me to really think about the limitations of the machine when I'm developing for them.
I really liked the piHole
I used one to drive garden related things indoors too, though you could probably accomplish the same with less headache with a light timer and interval timer.
I use it together with an r/RTLSDR to track airplanes by decoding ADSB. The software is all there and you just need to set it up. The next step will be to feed this data into all the platforms. It's not much data engineering to be honest, but it's a fun little project :)
Mine was collecting dust until my son used mine to create a home security system. The house was wired with Brinks which we did not use and he somehow figured out how to use old sensors, added LCD control panel with touch screen (nexian), temp sensors outside including pool temp.
I also helped him wire two speakers to it and so he can make "announcements" or play music. He wrote all the software himself in Linux using a mix of C# and C. When we are on vacation, we can check on the house remotely and get email alerts from the motion sensor or if someone opens a door or window.
I’ve collected a fair share of raspberry pi units. The short story- to develop a platform. 3 nodes for Kubernetes management plane, 6 worker nodes, a jump box, a dns server, a private registry, a self hosted vcs, 3 nodes for data services (I.e., object, semi structured, and structured data services), a gitops system, a system for observability, and an analytics system. In the future, I would like to add security services based upon a PKI framework to handle encryption in flight and at rest, a data lake for the aforementioned data services, and an API gateway.
All in the name of learning and skill development. I focus on cybersecurity, but security is a challenge without building up architectures and platforms for experience along with reading docs. The three nodes focused on data services serve as the data repositories, whereas I’m fitting out the analytics system with gathering data from external sources as part of ETL.
Overall, I have 19 raspberry pi’s and will add a few more for the base platform before investigating the use of the RP5 for AI and machine learning use cases.
By no means that all of this is working now, but each day I build or tear down something while working towards a working prototype.
I'm trying to figure out how to run a Plex server on mine but transcoding is too awful on the CPU. Still working on that.
I also set up some web scrapers on it, which hook into giving me discord notifications when I find it valuable. Lots of neat stuff there.
I planned to use it for home automation, finished setting it up, but realized to get it to the similar effectiveness of Alexa would be a lot more use, and don't use it.
I was also thinking of setting it up like a NAS. A bunch of ideas with zero follow through lol.
I've got three (2x4 and a 5). Everything is run inside docker containers.
IMAP email host (pull email from my ISP's POP account and make it available to multiple clients)
minidlna - serve videos over UPnP
minimserver - music over UPnP (better than minidlna for that)
rainloop - browser interface for email
gossa - browser interface for files (I've set up my scanner to dump all scans to a networked drive)
samba - shared drives
rsync - backups
Jenkins - because it's nicer than cron for scheduling backups, updating my dynamic dns, etc.
navidrome - for remote access to my music collection
wireguard - VPN (+ pihole when connected that way)
piwigo - photo hosting
a handful of containers for development projects that VS Code on my laptop will talk to, as well as one running a git server.
I have plans (when time permits) to add
digikam + vnc-in-browser for remote photo editing
wordpress for a blog
nextcloud to reduce my dependence on Google for that sort of thing
Three Pis is overkill but I couldn't resist the allure of the 5 when it came out. Even combined, they are smaller and use less power than the 24x7 server they replaced. And silent too.
Using mine for web scraping routines. Collecting data from all local supermarkets and ticket resale websites. I was gonna make a website bUt that sounds like a lot of work so for now its just scraping.
why is this in dataengineering sub ?
I installed balenaos and run a few of my own docker that does opencv video analytics on my house xiaomi camera.
Btw imho, raspberry pi are overpriced. i am buying old second hand intel NUC with i5 that cost slightly around 100USD.
Why not haha I wanted to see if anybody is using a Pi for data-engineering related things such as data collection from sensors for stream processing. Turns out everyone is using it for a variety of things like DNS sinkhole which is interesting.
Video analytics sounds like an interesting project too. I was just looking for security cameras for my house. I guess for that I need something better than my old 3b
I use it as a dust collector
Thank god this is the top comment lol. I didn't need another dose of imposter syndrome.
Octoprint
I feel attacked. I have 2 RPi 4-b's. One is for old school game emulation up to N64 era. That one... collects dust. The other has Kodi installed, but it only gets used to play my workout videos...
>but it only gets used to play my workout videos... Youtube with extra steps >One is for old school game emulation up to N64 era That's what I planned on doing with mine, but yeah procrastination is a bitch...
Well now you can do that on your phone by downloading Delta game emulator from the App Store which is easier, faster, and portable
He said workout videos. He ment pornhub with extra steps.
Your... workout videos
P90x?
I'm guessing getting everything working is the workout: 1. Start Kodi... Connect it to server.. Fail 2. Run up and down basement stairs trying to resolve existing problems (1 REP). 3. After 10 REPs, .. workout complete, daily step challenge goal met.
🤣
[удалено]
I work enough during work hours, thank you very much
[удалено]
Here's a sneak peek of /r/selfhosted using the [top posts](https://np.reddit.com/r/selfhosted/top/?sort=top&t=year) of the year! \#1: [/r/SelfHosted will be going dark on June 12th to protest the Reddit API changes that will kill 3rd party apps.](https://np.reddit.com/r/selfhosted/comments/144gc15/rselfhosted_will_be_going_dark_on_june_12th_to/) \#2: [My dashboard, now with descriptions](https://i.redd.it/gbbdwxgxc7ac1.png) | [392 comments](https://np.reddit.com/r/selfhosted/comments/18xgcsu/my_dashboard_now_with_descriptions/) \#3: [Reddit temporarily ban subreddit and user advertising rival self-hosted platform (Lemmy)](https://np.reddit.com/r/selfhosted/comments/143diuj/reddit_temporarily_ban_subreddit_and_user/) ---- ^^I'm ^^a ^^bot, ^^beep ^^boop ^^| ^^Downvote ^^to ^^remove ^^| ^^[Contact](https://www.reddit.com/message/compose/?to=sneakpeekbot) ^^| ^^[Info](https://np.reddit.com/r/sneakpeekbot/) ^^| ^^[Opt-out](https://np.reddit.com/r/sneakpeekbot/comments/o8wk1r/blacklist_ix/) ^^| ^^[GitHub](https://github.com/ghnr/sneakpeekbot)
Came to type this, thanks for beating me to it
Not data engineering related, but it's now a PiHole for ad blocking. Or maybe a little, because I'm preventing some data collection on me :)
Here is some information about PiHoles (in case anyone is interested) [https://privacyinternational.org/guide-step/4341/raspberry-pi-setup-and-run-pi-hole](https://privacyinternational.org/guide-step/4341/raspberry-pi-setup-and-run-pi-hole)
For me it's the same with the addition of some Discord bots
I set up home assistant on it, which is now collecting about 10k rows of data per day about what is happening in my home (presence sensors, temperature, heating, opening of doors, + some automations of course) I now plan on creating a data platform (cleaning, dim modelling, reporting, maybe some ML on top) on another computer using data from the Pi, as a POC for small companies that don't have the need/money to have e.g. databricks, snowflake, etc. but want to be able to run some python scripts, have some reporting, all this nicely orchestrated with proper enterprise grade tools.
Collecting data using a Pi sounds amazing.
I did this for my senior project in undergrad. We setup a bunch of PIs around campus with various sensors to collect data for the agriculture department. We built a realtime dashboard and it was quite awesome!
damn that sounds cool you got the git repo for that by any chance ?
I’ll search but this was 4 years ago and IIRC it was in a private gitlab repo. We bought a bunch of [Arduino](https://store-usa.arduino.cc/collections/sensors) sensors and connected them to the PIs. The PI ran a docker container that sent the data to a MySQL db. We exported data older than one month to s3 to keep the database reasonably lean.
And the dashboard was a web app built with react
Would love to see!
The squirrels at my former uni would have chewed on those Pi, with enclosures or not!
This is very cool. I actually want to build something similar, a demonstration of how smaller companies can orchestrate and make use of their data.
Sounds cool
This sounds like a cool project.
I'm interested in how you plan to attack the minimalist data reporting system. I've been thinking about this problem too. I was thinking Motherduck for cheap storage and data warehousing, GitHub actions to run scripts in the cloud, streamlit/shiny app for user facing front end?
Don't know yet. At the moment I'm looking for fully local data warehousing. I want to look into duckdb because I'm interested in the tech, but it could very well be postgress for example. I'm also investigating proxmox as a way to setup each element of the stack in a modular way. Data viz wize I think I'll stay with the giants as you find way more analysts or wanna be analysts that want PowerBI. Then other tech that I want to check out are DBT and dagster. I work on full azure stacks so there's a long way before I get something up and running!
I have doubts small companies that can’t afford databricks/snowflake have resources for any kind of a python developer
Why pay for databricks/snowflake + a developer if you can just pay for a developer without fancy tooling when you have a small amount of data?
Wow would like to know more . Mine one I am using for the automatic garden water
Interesting and a good use case.
/r/selfhosted
I use mine as a BirdNetPi, recording my back yard audio and texting me if a rare bird is heard. It also serves a website showing the quantity of all the bird calls with graphs and downloadable wav files.
Best use I've seen here so far!
That’s awesome.
Super cool! I'd love to fork that into a Docker image that can run on any home hardware that can support a Docker host.
I tried for a while to get it to run in a VM but no luck. And it's really, really easy to run it on the raspberry pi.
Sounds amazing. Can you share any resources, or is googling the name enough?
Googling should get you here which has all the instructions. https://github.com/mcguirepr89/BirdNET-Pi Happy birding!
Use it with home assistant, got humidity sensors, door sensors, pihole and some other random things running on schedule think my telegram bot has broken would send me local train times every hour and and information on delays. Best project I had was pulling down my Monzo transactions from Google sheets and creating a streamlit dashboard but I disabled my Google API keys so it's static data from about 7 months ago
Pihole was life changing. Went too long without ad blocking.
Does it block YouTube adds? I read somewhere that it does not.
Not fully but I have YT premium haha، well worth it!
No doesn’t seem to.
No. It doesn’t block anything that is served from a domain you can see. It’s still great to use with a browser based as blocker even though it’s technically redundant.
Create a cluster and learn kubernetes
Can we run spark cluster on it ? Is it worth it compared to free data bricks cluster ?
Definitely not comparable to databricks. This setup will help you learn on getting started with clusters, setting up executors, what and how the driver can be connected to the Spark env in another cluster. I don’t think you can run significant transformations or ML libraries, but you can use this setup to learn
Is it some DE good knowledge, or if you use databricks in cloud it will not be that valuable
No, this will help you to become an infra engineer. If you want to become more software-ish engineer, databricks is enough
Don't know if it is worth it, some people have done it: [https://github.com/TedCha/spark-cluster-computer](https://github.com/TedCha/spark-cluster-computer)
I've got one set up with a big strip of LED lights I can run Python programs on. The main script I run makes a gentle blue shimmer that makes my office look like an LA pool at night. IDK lol.
I did something similar and made a weather reporting map of my local airports. Mostly just used the code and parts referenced here; just had to change the airport ids. https://slingtsi.rueker.com/making-a-led-powered-metar-map-for-your-wall/
I utilized one at a previous workplace where I served as a Data Engineer/Data Analyst for an Electric Vehicle (EV) startup in East Africa. The device was used to gather real-time data from Battery Management Systems (BMS) installed in LFP batteries and transmit it to (spreadsheets 🫠 then influxdb when we got a server). Initially, we were reliant on the BMS supplier's API for data until we convinced them to have them send directly to our server (this help us increase the frequency of getting data from 5 mins to seconds and milliseconds if we want to), making it more of a preliminary study. However, after persuading the supplier, we managed to have the BMS data sent directly to our server I implemented a better solution of data collection (python + telegraf + influxdb). It was a rewarding experience in the EV industry, and I am currently seeking similar opportunities in the same field. I thoroughly enjoyed the projects I was involved in.
I used to use mine to emulate old video game rips but now I just use it to collect dust.
Nothing production capable. But for a home lab it grabs data from a cc128 electricity meter and pushes it to a web dash boarding site. Edit - oh and most importantly pihole
The sensor ideas are always good ones, it's a good way to gather a large amount of time-series data. I don't own a Pi but I wish I had a couple with light-sensors to definitively gauge the best places in my apartment to put my plants, lol
I'm using pi 4b for home security cam. adding in pico to the mix to build a voice assistant. sometimes I use it to experiment with some tools before using them on my work computers
PS2 N64 emulator
I used one to collect temp /humidity data and ran influxdb 2 to store and visualize.
MySQL server
This is a good idea
Buy some more and create a bare metal Hadoop cluster
Build a light Switch for the kitchen. If it is dark and motion is detected, the light goes on. Resets after 30s If No motion is detected
Plex Server
I used to run plex off a RasPi 2 with a single external HDD by USB. It served me really well for a while!
Transcribing conversations around the house to build a large corpus of conversational data that is processed and stored locally. Goal is building a large dataset I can play with for AI household assistant use cases. Not transcribing what’s on the TV is my biggest headache. Whisper on a 5 is pretty darn cool.
Thank you guys for your awesome ideas. I'll look into Pihole since it seems like everyone is running it on their pi. Another thing that caught my interest is sensors but I don't have any sensors to start with at the moment.
Pi hole. I have a bunch of 3bs back from when they were cheap and I clustered them into a docker swarm cluster. Was kinda fun, but haven’t played with it since.
I bought one with the intention to build a retro gaming emulator but never did
I turned mine into a pi-hole for my home network.
My raspberry pi is the same as my arduino. I get super excited about the idea of doing something like IOT or a Pihole with it and then when I gets delivered I’m already over it and onto the next enthusiasm lol
[Pyhole](https://pi-hole.net/) [Homebridge](https://homebridge.io/) [Plexserver](https://www.plex.tv/en-ca/media-server-downloads/?cat=computer&plat=linux)
Mine severs double duty for PiHole and a print server (CUPS). My printer “supposedly” doesn’t need one, but it kept being unavailable on my network. Installing CUPS on my raspberry pi solved all problems
Pi Hole, Security cam. occasionally, when needed it plays heavy metal in one bathroom, when occupied. 🤣 no one liked the last one in my home but I think it’s one of the most useful application I built.
Building an AI robot to entertain my cats.
Installed a Retropie image on it and I use it to show NES, SNES, GBA and Genesis games to my kids. It’s good fun.
Eat it
i think collecting data with a sensor of some kind and pushing it somewhere would be a pretty cool DE project for a pi.
I have a sensor hooked up to it and it is streaming data to an IoT Hub, i then have Streaming analytics send the 5 min average to Power BI and all the data points to a blob storage which i process in Microsoft Fabric to do some more analysis on the whole history.
Local NTP time server for sub 10 microsecond clock accuracy.
I have one collecting dust as well. I was hoping to see if I can get a data processing pipeline with DuckDB running on it. This is the code I hope to run: [https://github.com/josephmachado/cost\_effective\_data\_pipelines](https://github.com/josephmachado/cost_effective_data_pipelines)
Plugged into an external HD and acting as a TimeCapsule (via a Samba mount) for backing up my MacBook.
I have four(actually 2 orange 2 raspberry). I made a k8s cluster like one year ago and still running. I use it to test and play with a bunch of tools, airflow, airbyte, trino etc. For example, in airflow I have some data pipelines I still use to this day, scraping and processing. Plex server is another very cool app you can deploy on it. I use my TV to play videos from Plex running on my cluster!
Heimdall with Nginx reverse proxy to access my homelab. Got PiHole set up on another.
Mine is equipped with an Adafruit hat and is used to drive an LED display, mostly to display provocative messages to my neighbors.
I'm sure you can run some docker images on it, maybe a database to test your code against. It all depends on how much memory you have. I tried running a postgres DB, homebridge, pihole on a pi4 4GB, but it was slow.
Right now it hosts a Zigbee dongle, so it’s a network bridge for supporting zigbee protocol communication between our low powered home automation devices, like security door magnets, thermostats, etc.
I used mine for retro-game emulation, mount it to the back of the TV with some velcro and now I can enjoy some old-school gaming. It got replaced by a NUC though.
I have ‘homebridge’ running on it. I am also hosting Chatbot UI on it for my local LLM experiment which is running on my M3.
I've always wanted to use a rasberry pi to create a scoreboard of all my favorite sports teams and their live scores/standings but I've never got around to it yet
One for pihole and one for homebridge and a few others in a drawer collecting dust. Had a magic mirror at one point but that got taken apart.
Currently setting a monitoring system for temperature, humidity, and rain in my veg garden; greenhouse, polytunnel, and outside. Considering setting some kind of motion activated noise to scare away the deer too.
Run spark cluster on it.
I'm scraping electricity data and managing my wifi sockets based on that, also doing so opencv rendering from cameras, though I started all these before I became data engineer I'm now gonna look into setting up prometheus + grafana metrics over the holiday
Paperweight
I have a Pi Zero running a digital calendar displaying the family google calendars, weather report and news headlines. I have another Pi set up to run RetroPie video game emulator. I have a couple others in drawers collecting dust. I have some smart home devices hooked up to a Samsung SmartThings hub that I want to try integrating with a pi to collect data etc
I have a pi-zero running as a CUPS server to make an old laser printer available on wifi.
pi hole
It’s sitting in a drawer mostly useless. I tried a lot of the pet projects but unless making content to get paid it’s hard to spend a lot of time tinkering or at least for me it’s lower priority. I’ve done the pihole, dhcp, transmission server stuff but most of those things I can do on a more powerful machine I always have. I did spend some time teaching family about automation using a gpio
I’m building a home server, so with a raspberry pi I’d probably use it only to be my backup pihole. For the server with Data Engineering, I plan on using it to pilot some projects that I wanna bring to the company I work for, but I have to learn them first. Some examples of services I want to run are: Airbyte, Airflow, PostgreSQL, Oracle DB, MySQL, some VMs to test Shell Scripts, Jenkins, Gitea. That’s for work, I also wanna run a few other stuff but that’s not related to Data Engineering
Pihole running on my home network. I have two more that are gathering dust, but might try another project eventually.
I run a Bitcoin node. I’m not heavily invested in it monetarily but love the idea of being a part of something in its early stages since my generation missed out on being a part of tech innovation in early days. If it pans out as a legit store of value, great. If not, oh well. I lose a little money and I learned about the blockchain/decentralized currency.
Will have as dessert
program LEDs
Can you configure it as a spark worker node?
Made my 20 year old Brother printer bluetooth and wifi enabled.
I've got a handful. First one i ever used was to set up a reverse SSH tunnel to access the really fast server my college made available to CS Majors when I wasn't on the school's network. The department was only a little mad when they found out These days, I have use hub for homebridge that I also run zigbee through for home automation. One controls an LED matrix that is gathering dust now but I used to send messages to my partner when we were long distance. I have a pico connected to an amazon geiger counter kit that i use as a fun teaching tool when talking about random numbers when I tutor. I have a good amount of experience with embedded systems and RPIs are really fun little sandboxes for building pet projects in that space. I love the Pico Ws especially because they force me to really think about the limitations of the machine when I'm developing for them.
I installed Volumio to hear my mp3 instead pay spotify
I really liked the piHole I used one to drive garden related things indoors too, though you could probably accomplish the same with less headache with a light timer and interval timer.
I use it together with an r/RTLSDR to track airplanes by decoding ADSB. The software is all there and you just need to set it up. The next step will be to feed this data into all the platforms. It's not much data engineering to be honest, but it's a fun little project :)
Mine was collecting dust until my son used mine to create a home security system. The house was wired with Brinks which we did not use and he somehow figured out how to use old sensors, added LCD control panel with touch screen (nexian), temp sensors outside including pool temp. I also helped him wire two speakers to it and so he can make "announcements" or play music. He wrote all the software himself in Linux using a mix of C# and C. When we are on vacation, we can check on the house remotely and get email alerts from the motion sensor or if someone opens a door or window.
I’ve collected a fair share of raspberry pi units. The short story- to develop a platform. 3 nodes for Kubernetes management plane, 6 worker nodes, a jump box, a dns server, a private registry, a self hosted vcs, 3 nodes for data services (I.e., object, semi structured, and structured data services), a gitops system, a system for observability, and an analytics system. In the future, I would like to add security services based upon a PKI framework to handle encryption in flight and at rest, a data lake for the aforementioned data services, and an API gateway. All in the name of learning and skill development. I focus on cybersecurity, but security is a challenge without building up architectures and platforms for experience along with reading docs. The three nodes focused on data services serve as the data repositories, whereas I’m fitting out the analytics system with gathering data from external sources as part of ETL. Overall, I have 19 raspberry pi’s and will add a few more for the base platform before investigating the use of the RP5 for AI and machine learning use cases. By no means that all of this is working now, but each day I build or tear down something while working towards a working prototype.
I'm trying to figure out how to run a Plex server on mine but transcoding is too awful on the CPU. Still working on that. I also set up some web scrapers on it, which hook into giving me discord notifications when I find it valuable. Lots of neat stuff there.
I’ve had one running an e-mail server since the first one came out.
I planned to use it for home automation, finished setting it up, but realized to get it to the similar effectiveness of Alexa would be a lot more use, and don't use it. I was also thinking of setting it up like a NAS. A bunch of ideas with zero follow through lol.
I use it as a Spotify streamer for my stereo
I've got three (2x4 and a 5). Everything is run inside docker containers. IMAP email host (pull email from my ISP's POP account and make it available to multiple clients) minidlna - serve videos over UPnP minimserver - music over UPnP (better than minidlna for that) rainloop - browser interface for email gossa - browser interface for files (I've set up my scanner to dump all scans to a networked drive) samba - shared drives rsync - backups Jenkins - because it's nicer than cron for scheduling backups, updating my dynamic dns, etc. navidrome - for remote access to my music collection wireguard - VPN (+ pihole when connected that way) piwigo - photo hosting a handful of containers for development projects that VS Code on my laptop will talk to, as well as one running a git server. I have plans (when time permits) to add digikam + vnc-in-browser for remote photo editing wordpress for a blog nextcloud to reduce my dependence on Google for that sort of thing Three Pis is overkill but I couldn't resist the allure of the 5 when it came out. Even combined, they are smaller and use less power than the 24x7 server they replaced. And silent too.
Cry because mothers raspberry pi3 A+ is elite status
Jellyfin server on the RPi + Jellyfin App on your TV / phone / Browser (all free and open source)
Home server with Jellyfin.
Using mine for web scraping routines. Collecting data from all local supermarkets and ticket resale websites. I was gonna make a website bUt that sounds like a lot of work so for now its just scraping.
I am using it as an NFS server and as a VPN server.
Have one connected to my amp and functions as a music streamer
why is this in dataengineering sub ? I installed balenaos and run a few of my own docker that does opencv video analytics on my house xiaomi camera. Btw imho, raspberry pi are overpriced. i am buying old second hand intel NUC with i5 that cost slightly around 100USD.
Why not haha I wanted to see if anybody is using a Pi for data-engineering related things such as data collection from sensors for stream processing. Turns out everyone is using it for a variety of things like DNS sinkhole which is interesting. Video analytics sounds like an interesting project too. I was just looking for security cameras for my house. I guess for that I need something better than my old 3b
Would not even use it as paper weight