I am finally making the push to self host everything I possibly can and leave as many cloud services as I can.
I have years of linux server admin experience so this is not a technical post, more of an attempt to get some crowd wisdom on a complex migration.
I have a plan and have identified services i would like to implement. Take it as given that the hardware I have can handle all this. But it is a lot so it won’t happen at once.
I would appreciate thoughts about the order in which to implement services. Install is only phase one, migration of existing data and shaking everything down to test stability is also time consuming. So any insights, especially on services that might present extra challenges when I start to add my own data, or dependencies I haven’t thought of.
The list order is not significant yet, but I would like to have an incremental plan. Those marked with * are already running and hosting my data locally with no issues.
Thanks in advance.
Base system
- Proxmox VE 8.3
- ZFS for a time-machine like backup to a local hdd
- Docker VM with containers
- Home Assistant *
- Esphome *
- Paperless-ngx *
- Photo Prism
- Firefly III
- Jellyfin
- Gitea
- Authelia
- Vaultwarden
- Radicale
- Prometheus
- Grafana
I also started with a Docker host in Proxmox, but have since switched to k3s, as I think it has reduced maintenance (mainly through FluxCD). But this is only an option if you want to learn k8s or already have experience.
If Proxmox runs on a consumer ssd, I would keep an eye on the smart values, as it used up the disk quickly in my case. I then bought second-hand enterprise ssds and have had no more problems since then. You could also outsource the intensive write processes or use an hdd for root if possible.
I put my storage controller directly into the VM via PCI, as it makes backups via zfs easier and I was able to avoid a speed bottleneck. However, the bottleneck was mainly caused by the (virtualized) firewall and the internal communication via it. As a result, the internal services could only communicate with a little more than 1GBit/s, although they were running on ssds and nvme raids.
I use sqlite databases when I can, because the backups are much easier and the speed feels faster in most cases. However, the file should ideally be available locally for the vm.
Otherwise I would prioritize logging and alerts, but as an experienced server admin you have probably thought of that.
Good call out on the smart values. That’s on the priority list for my monitoring scheme now too.