I've been working with virtualization technology professionally for a couple of decades now. A lot has changed in that time and a lot has remained the same. In the beginning the go-to for an enterprise solution was VMWare, and I was all-in on ESX and then ESXi. Around 2010 I managed to get my hands on an IBM E-Chassis complete with blades for essentially nothing. The house I was in at the time had a spare 220v outlet so I was able to set the chassis up at home and get a little home lab going. After a few months I got tired of the noise (it was unbelievably loud) not to mention the electric bill. So, I tore the whole thing down and sold it to a coworker for basically nothing and for a couple of years didn't bother with a home lab setup. In 2013 I picked up a nearly new IBM tower server that another coworker had bought and then decided he didn't want. I put ESXi on it, booting from a thumb drive, and ran that for a couple of years. I was pretty pleased with it but the coveted enterprise functionality like HA was non-existent since I only had the one. So, again I parted with it to another eager coworker and spent the next few years once again with no home lab.

In 2019 I bought a raspberry pi zero W. I turned it into a pi-hole and I loved it (and still do). I picked up an old Lenovo W540 the same year with Windows preinstalled on it and used it as a spare computer for testing things. In 2021 the raspberry pi zero W microSD card wore out and it was then that I really started looking into getting a home lab set up again. The W540 it turns out has a 4th generation Intel Core processor that supports both VT-d and VT-x. I had been working professionally with ESXi and Nutanix AHV for awhile at that point but I had also been doing a lot of reading about Proxmox VE. I decided I wanted to learn something new so I gave Proxmox VE a try. Installing it on the W540 was very easy and within a couple of hours I had a new pi-hole up and running. I spent a little bit of time configuring it and before long had migrated my whole network over to the new pi-hole and retired the old one. This was really when my dive into Proxmox VE at home started. I had spent $100 on the W540, so that was the extent of my investment at that point.

Over time my usage of Proxmox VE organically grew. I knew early that I wanted to be able to migrate VMs to another host so that I could perform maintenance on hosts without interrupting the guests. I had a friend with an old MSI laptop and he was willing to part with it so he shipped it to me and in no time I had another Proxmox VE host up and crossover cabled the onboard NICs to build a simple 2-host cluster. With only 2 hosts and no witness I couldn't leverage HA but I could migrate VMs and that was good enough for me at the time. I still had only invested $100.

Within a few months the MSI laptop started having some stability issues and, despite my best efforts, I was unable to fix them. It seems that their gaming laptops are known for the embedded controllers glitching and the laptops just becoming simply unuseable. So, I decommissioned the laptop and replaced it with another W540 I picked up from eBay for $130. After a couple of years I was only in for about $250 on a 2 node Proxmox VE cluster.

Before long I had exhausted the capacity for a single host to support the entire load during maintenance operations. Rather than grow with another single host I decided I wanted to give myself more room to grow so I added 2 more. I picked up a couple of "for parts" W540s off of eBay again. One came with basically everything stripped out, another had no battery, a broken screen, and a keyboard missing letters. Thankfully I don't need these laptops for laptops so a broken screen or keyboard isn't a deal killer. I plugged in what I needed long enough to configure the host and get it plugged into the cluster. I spent a couple hundred more on these 2 hosts, so for 4 hosts (16 cores, 128 GB RAM total) I was in for around $450. I added 2 more NICs to each host (USB) to break out the migration interface and the VM interface, and an additional mPCIe disk for each, for a total investment in the hosts of about $600. Add in a couple of dumb switches for dedicated cluster and migration networks for $30 each to bring the total for the hardware to around $660. Finally, I put together a scrap wood cabinet to contain everything, 6 fans to cool everything, a UPS for the switches, and some wood stain and ethernet cables, bringing the total for everything altogether to about $750.

This is easily my favorite virtualization solution for home. I use scheduled replication for pseudo-HA so I can do maintenance on the hosts without any interruption to the guests. I experimented with hyperconverged storage but my poor gigabit links just couldn't keep up. Replication gives me the performance of local storage with the benefit of very small deltas so migrations take very little time. As you can see I've got a few VMs running with room for plenty more. I don't personally like containers but if I were to use them my capacity would be even greater. I find the W540, now 10 years old, to be a very capable platform for this.

Here's a shot of my scrap cabinet:

If you're looking for a virtualization solution for home I strongly recommend Proxmox VE. Despite my ongoing (daily) enterprise experience I also recommend you consider that you might not need a rack with a pizza box (or boxes), or a blade chassis, or a server tower, but that creative use of consumer hardware might be more convenient (and cheaper) for you.

- Andrew

topics proxmox ve