Cloud provider like AWS, Azure, and GCP alredy have dyncamic provisioner setup. So when their user create resources like PersistentVolumeClaim, the resources gets automatically provisioned.

Homelab cluster does not have a dynamic provisioner set up already, we are going to create one.
The high level strategy is to use Synology as NFS server and have my pi cluster to deploy a NFS client as the provisioner.

Set up NFS server

Synology NAS

  • using this Youtube tutorial
    • create shared folder called nfs
    • only allow this ip range to access 192.168.1.192/255.255.255.0

Check that NFS server is working

# make sure the permission on worker node is correct
mkdir /opt/nfs
sudo chown pirate:pirate /opt/nfs
sudo chmod 755 /opt/nfs

# Update worker node to load this on startup
sudo vi /etc/fstab

# Add the following entry
mqiu-nas:volume1/nfs /opt/nfs defaults 0 0

/etc/fstab

Save the above file and use the following command to verify the you can mount successfully.

sudo mount /opt/nfs

Remember to umount disk

Now that we have verified the NFS mounting is working as expected.

sudo umount /opt/nfs

After umounting we are ready to have NFS client provisioner running for us.

Deploy nfs-licent-provisioner

The overall architecture is described by this following diagram: nfs client provisioner architecture

The provisioner is a pod, in order to have proper access control, we are going to set up correct role and account for the it as well.

Install Using Helm

If we think of k8s as an OS for a cluster of machines.
Helm is a package management tool for k8s cluster just as Pip is a package management tool for Python.

For nfs client, we can direcly install it from helm stable repo.

# create namespace for the nfs client deployment
kubectl create ns storage

# install nfs client via helm
$ helm install nfs-client-provisioner stable/nfs-client-provisioner   \
--namespace storage --set nfs.server=mqiu-nas --set nfs.path=/volume1/nfs \
--set image.repository=quay.io/external_storage/nfs-client-provisioner-arm

NAME: nfs-client-provisioner
LAST DEPLOYED: Fri Jul 10 18:34:23 2020
NAMESPACE: storage
STATUS: deployed
REVISION: 1
TEST SUITE: None

I installed this deployment to a storage namespace.
Because we are on an ARM based cluster, we need to set the -arm suffix to image.repository to include arm based docker image.
Since my Synology based NAS is at mqiu-nas:/volume1/nfs, I will set the nfs.server=mqiu-nas and nfs.path=/vluem1/nfs.

$ kc get all -n storage
NAME                                         READY   STATUS    RESTARTS   AGE
pod/nfs-client-provisioner-f4d9d5cdf-xxfzc   1/1     Running   0          117m

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nfs-client-provisioner   1/1     1            1           117m

NAME                                               DESIRED   CURRENT   READY   AGE
replicaset.apps/nfs-client-provisioner-f4d9d5cdf   1         1         1       117m

$ helm list -n storage
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                            APP VERSION
nfs-client-provisioner  storage         1               2020-07-10 22:20:31.6635529 -0400 EDT   deployed        nfs-client-provisioner-1.2.8     3.1.0

# Check role based access control is created
$ kc get clusterrole,clusterrolebinding | grep nfs
clusterrole.rbac.authorization.k8s.io/my-nfs-client-provisioner-runner          28m
clusterrolebinding.rbac.authorization.k8s.io/run-my-nfs-client-provisioner      28m

Create a PersistentVolumeClaim like this:

#pvc-nfs.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
        name: pvc-1
spec:
        storageClassName: nfs-client // <-- this is important; triggers dynamic provisioning
        resources:
                requests:
                        storage: 100Mi
        accessModes:
        - ReadWriteMany

In the future if we want to remove this installation:

helm delete my-nfs-client-provisioner -n storage

Install Manually

Solution is adapted from this Youtube Tutorial by Venkat. This tutorial breaks down the nfs-client-provisioner helm chart into small parts and apply them one by one to achieve the same goal.

Reference