Oh wow how things change in a month! I now have 3.5 homelabs, It’s an odd number but hear me out:
- OG Proxmox Server
- Parents Proxmox Server
- Mediaserver (I don’t classify this as a full lab)
- New Raspberry Pi Compute Module 5 Kubernetes Cluster
Migration to Kubernetes#
For a while now I’ve wanted my own cluster, I’ve even built a few in Proxmox but none of them have ever stuck around long enough for me to really grow attached to them. For those that don’t know what I do for my day job I manage a k8s cluster with around a 1000 namespaces it’s quite challenging! The problem is I don’t use k8s, I can talk for hours on how our cluster is built and how all the components fit together but I can count on one hand the amount of time I’ve actually deployed any applications to it. I have a hole in my knowledge when it comes to actually using k8s as an end user. Building a physical cluster and migrating my existing apps over to it should hopefully cover that.
Hardware#
I have a DeskPi Super6c carrier board and 2 Raspberry Pi Computer Module 5s with 8gb RAM and 16gb EMMC. I intend to fill this completely with a mix of more RPis and other manufacturers that offer drop in replacements. I’ve been quite critical of RPi in the past and continue to be so will see how things go with the new hardware.
Software#
I’m using Armbian as my base OS which has been ok so far. I’m new to the whole Compute Module scene so getting the boards setup with an OS is manual at the moment. I keep meaning to look into cloud-init but I don’t think Armbian supports it, they do have something similar but the documentation is quite lacking. Also configuring anything other than hostname in raspberry pi imager flat out doesn’t work in Armbian. For k8s itself I’m using k3s and their ansible playbook which is really nice. It’s super easy to uninstall it and rebuild if I do something wrong.
Next Steps#
I’m currently setting things up, I’ve already reinstalled k3s twice and will probably nuke it again by the end of the day. I’m trying to automate as much as I can so I’m using Ansible to do most of the heavy lifting. I’ve still to get provision the actual Pis more automated but it’s fine for the time being.
I need to setup Traefik and Metallb then I should be able to start looking into moving workloads over. I also need to start buying a few more NVME drives so I can add CEPH into the mix for PV storage with backups to my NAS.