casa/docs/k3s-cluster.md

Casa k3s cluster

Table of Contents:

• Nodes
  • Master Node
  • worker node

Nodes

Setup user for ssh access:

apt install sudo
usermod -aG sudo mf

Disable swap:

swapoff -a
Edit /etc/fstab and comment out any swap entries:
# /swapfile none swap sw 0 0

Other Packages:

sudo apt update -y | sudo apt install curl btop -y

Master Node

• debian 12
• 2g ram
• 24Gb disk

ip a # check ethernet name

# removes automatic vonfiguration as dhcp client
sed -i '/ens18/d' /etc/network/interfaces

cat <<EOF > /etc/network/interfaces.d/ens18
# my network configuration
auto ens18
iface ens18 inet static
  address 192.168.14.9/24
  gateway 192.168.0.1
EOF

cat <<EOF > /etc/resolv.conf       
domain dev.lan
search dev.lan. lan.
nameserver 192.168.14.1
EOF

curl -sfL https://get.k3s.io | sh -

Taint NoSchedule on master node:

kubectl taint nodes node-role.kubernetes.io/control-plane=:NoSchedule

kubectl taint nodes casa node-role.kubernetes.io/control-plane=:NoSchedule

worker node

• debian 12
• 4g ram
• 8Gb OS disk
• 16Gb data disk

# execute on server to get token
cat /var/lib/rancher/k3s/server/node-token

Setup worker node:

# install k3s as agent / worker node
TOKEN="???"
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --data-dir /dataDisk/k3s --server https://casa.dev.lan:6443 --token ${TOKEN}" sh -s -

Move default k3s folder:

apt install rsync
sudo systemctl stop k3s-agent
sudo rsync -a /var/lib/rancher/k3s/ /dataDisk/k3s/

Execute install k3s default update script with --data-dir /dataDisk/k3s argument.

Change kubectl -n kube-system edit configmap local-path-config on kube-system to set path to provisioner.

  config.json: |-
    {
      "nodePathMap":[
      {
        "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
        "paths":["/var/lib/rancher/k3s/storage"]
      },
      {
          "node":"casa-minion-01",
          "paths":["/dataDisk/k3s/storage"]
      }
      ]
    }

rm -rf /var/lib/rancher/k3s

Set node labels:

kubectl label node casa-minion-01 role=worker-node