I’m currently traveling for months at a time and my homelab has become unreachable to me over VPN due to a unknown complication after a power outage.
Just as a learning experience for all, my mistake was that I set-up my VPN very far down the stack - as a wg-easy app inside TrueNAS SCALE’s apps ecosystem. My very important reason for doing it was that way was that wg-easy allows for setting up client devices with a QR code…
Anyway, the NAS is not booting back up nor do the TrueNAS apps. I should’ve set my VPN up right at the front of the network - on my MikroTik router that also supports Wireguard. The funny thing is I was so happy that my NAS has IPMI and whatnot but now I can’t even access it.
For now the NAS is kept powered on from what I know, it just doesn’t boot. This should help prevent bitrot until I’m back. All important files are backed up on a 3rd party service.
It’s a shame my Jellyfin and Navidrome inaccessible, but I’ll live.
Now I’m thinking about buying an UPS so that this doesn’t happen in the future. I’d like the UPS to be fanless and rackmount, so that limits me to ~700VA territory.
Devices in my homelab pull about 65W idle and spike to say 150W when everything is booting. ISP modem, router, POE+ switch, AP, NAS. I might add another 20W due to a Lenovo M920q in the future.
I only really care about NUT and graceful shutdown instead of long runtime on battery.
I was thinking about this: https://www.apc.com/us/en/product/SMT750RMI2U/
In my country I can get it with new batteries (no front panel) and a network card for NUT for a total of 180 EUR.
Would that work? Would you be afraid of leaving an UPS (it is kinda like a bomb after all) unattended an leaving your home for 6 months at a time?
…(it is kinda like a bomb after all)…
WAT? I’ve never heard a UPS referred to as “kinda like a bomb” before.
Keep your UPS maintained, replace the batteries when they age out, and it will be fine. If your UPS supports automated self-tests, use them.
My employer has UPS units spread all over the region we operate in, and we don’t have any issues, despite leaving them mostly unattended for years. I have several in my house and I’ve never given them a second thought aside from battery replacements.
Do they make them with LiPos? Everything I’ve ever owned had sealed Lead/Acid batteries. Really hard to cause any damage with those.
Lead acid can boil over and fill a badly ventilated room with flammable hydrogen and other toxic gasses.
A well maintained LiFePO4 battery that some modern UPS use it probably less risky, but I think the risk for both is very low.
To highlight the low risk, every (non-EV) car in the world uses a lead acid battery.
Had one of power powervar’s lead acid batteries swell up and overheat on us, but it was probably also installed several years ago.
As long as you replace them on a regular basis, totally fine.
There are probably newer ones that come with LiPos. But every consumer grade one I’ve seen is traditional lead acid batteries.
If you hit them with a flame thrower they explode
Man, I don’t even run my desktops without a UPS. Maybe I’m jaded from too much tech support, but I feel like not having a UPS is like not using a condom for a random hookup: it’s fine until it dramatically and seriously isn’t.
Almost every desk at my workplace has its own lead-acid battery. That’s well over 1000. We haven’t had any fires or explosions.
Line interactive basically means the battery will always be feeding the devices on UPS. Make sure you don’t go over the listed wattage. I learned the hard way and bought an under spec Tripplite, the battery kept dying. I now have a UPS specced out for my router and the Verizon box outside, not anything else on my rack. You should be fine, just saying in the future when you add devices, keep that in mind.
Slight clarification that you described a smart online ups. A line interactive device only supplements power grid with battery in order to keep a smooth 120v should it flutter, for ex if target voltage drops to prevent brownout conditions or fully kick over to battery if power goes out.
A smart online ups runs from the battery 24/7 and is back fed by the grid.
Thought they were synonymous. Thank you for making the distinction.
The device you’re thinking of has 42 decibels of sound. You should be aware of that, I don’t think it’s actually fanless
Noise is going to be a huge factor for your home lab, so make sure you look at the data sheet for whatever you’re about to buy and check what it’s rated noise level is
Always UPS everything. But also always have a simple backdoor. I generally have 1 little desktop like a NUC running some basic Win10 OS and an install of remote software like TeamViewer. It is connected to my hardware router right after the ISP router and a backup connection. Used to be LTE everywhere, now I’m half and half on Starlink. It is then also connected to the router ports needed for management but inactive.
If I have to, remote into the NUC over Starlink. I can then reboot my main ISP box. I can eventually get into my router and enable those ports which are pre-plugged in. From there I can then access all the stack management and all the IPMI ports like iLo. It’s a virtual interface through a virtual interface. It is slow, and painful. But it works.
And it works 99.99% of the time. But even then, I’ve had to do a call of shame and walk one of my friends through which button to press as I’m on the other side of the world. In my case it was also power related but the UPS I had decided to overheat. In reality over the summer, the temps were high. But also it is a super awesome double conversion UPS. The line voltage into the UPS was dropped to below standards from the utility because their grid was overworked with everyone’s AC’s. So the UPS saw this as a line failure, kicked in the double conversion and ran happily. But it did not count as a power failure, so none of my services scaled back. Essentially it was delivering 3KW of juice from the wall through a double conversion making the whole thing super hot. Eventually it shut down for safety automatically, just pulled the plug. My NUC is on a separate little backup along with the modems and an auto transfer switch which did its thing. But there was no way to press the reset button on the UPS for a critical safety shutdown like that. It had to be in person.
You want the ability to remote power cycle everything and to get a quick status. Get a single board computer and use it as a jumping off point. From there you can go get some smart plugs that can be connected to the single board computer.
If you can afford one, I would strongly recommend going with a dual-conversion UPS. A line-interactive UPS like the one you posted essentially acts as a pass-through for your mains power until it detects a power loss or a brown-out. This works most of the time, but there’s a short delay during the switch from line to batteries (just guessing, but most likely on the order of milliseconds). This might not sound like much, but you’re counting on the capacitors in your server’s power supply to hold enough charge until the UPS kicks in.
The other thing to consider is that a dual-conversion UPS also supplies “clean” power to your equipment. It essentially acts as a DC power supply connected to an inverter, so regardless of how bad your input power is, you’re always going to get the correct voltage and frequency out. I connected my old line-interactive UPS to a cheap generator at one point; the voltage and frequency regulation was so bad on the generator that my UPS continually switched on/off of battery (several times per second), and the equipment attached to it immediately shut down.
I can connect my dual-conversion UPS to the same generator, and it keeps humming along as if it was connected to mains voltage. According to the datasheet, anything from 60VAC to 150VAC, it’s still going to output clean 120V/60hz power.
They’re much more expensive. Mine is 1000VA, and if I remember correctly, I paid something like 600 or 700 USD for the UPS. An add-on rackmount battery pack was another $300 or so. It was well worth the cost, though.
UPDATE: Turned out that the culprit of the downtime was my switch - the D-Link DGS-1210-10P rev. B1.
The way the management web interface of the switch works is pretty unintuitive. Namely, if you change some settings in the web interface and hit save in one of the sections, the settings are saved in the volatile memory of the switch. This basically means that the settings are only saved in RAM, which is cleared on power loss. To save the settings into non-volatile memory which persists on reboots, you need to find the “Save” section at the top of the UI. This is described here: https://networkengineering.stackexchange.com/questions/20158/dlink-switch-loses-configuration-on-power-off
So basically, my problem was that the settings weren’t commied to nonvolatile memory and on a short 1 minute power loss the switch restarted.
I got an UPS anyway now, SMT750RMI2U
