0bf368d129
fix: log spam every minute "Liveness (or Readiness) probe already exists for container", for every container. Logs where being registered on k3s process (systemd logs).
188 lines
3.9 KiB
Markdown
188 lines
3.9 KiB
Markdown
# Casa k3s cluster
|
|
|
|
**k3s version:** v1.35.2+k3s1
|
|
|
|
**Table of Contents:**
|
|
|
|
- [Common](#common)
|
|
- [Master Node](#master-node)
|
|
- [Master Node - proxmox vm](#master-node---proxmox-vm)
|
|
- [Master Node - network configuration](#master-node---network-configuration)
|
|
- [Master Node - k3s setup](#master-node---k3s-setup)
|
|
- [minion01 - worker node](#minion01---worker-node)
|
|
- [Minion01 - proxmox vm](#minion01---proxmox-vm)
|
|
- [casa-minion-01 - k3s - setup](#casa-minion-01---k3s---setup)
|
|
|
|
## Common
|
|
|
|
**Disable swap:**
|
|
|
|
``` bash
|
|
swapoff -a
|
|
Edit /etc/fstab and comment out any swap entries:
|
|
# /swapfile none swap sw 0 0
|
|
```
|
|
|
|
**Other Packages:**
|
|
|
|
``` bash
|
|
sudo apt update -y && \
|
|
sudo apt install -y \
|
|
curl \
|
|
btop
|
|
|
|
```
|
|
|
|
**Update system:**
|
|
|
|
``` bash
|
|
sudo apt update -y && \
|
|
sudo apt upgrade -y && \
|
|
sudo apt auto-remove -y
|
|
```
|
|
|
|
**name servers:**
|
|
|
|
``` bash
|
|
cat <<EOF > /etc/resolv.conf
|
|
domain dev.lan
|
|
search dev.lan. lan.
|
|
nameserver 192.168.14.254
|
|
EOF
|
|
```
|
|
|
|
## Master Node
|
|
|
|
| Name | Value |
|
|
| --- | --- |
|
|
| **NAME** | casa.dev.lan |
|
|
| **VLAN** | casa |
|
|
| **IP** | 192.168.14.9 |
|
|
| **OS** | Debian GNU/Linux 12 (bookworm) |
|
|
|
|
### Master Node - proxmox vm
|
|
|
|
*hosted on surfacepro.*
|
|
|
|
``` yaml
|
|
agent: 1
|
|
balloon: 0
|
|
boot: order=scsi0;ide2;net0
|
|
cores: 2
|
|
cpu: host
|
|
ide2: none,media=cdrom
|
|
memory: 2355
|
|
meta: creation-qemu=10.1.2,ctime=1762626497
|
|
name: casa
|
|
net0: virtio=BXX:XX:XX:XX:XX:XX,bridge=vmbr0,tag=xx
|
|
numa: 0
|
|
onboot: 1
|
|
ostype: l26
|
|
scsi0: local-lvm:vm-XXX-disk-0,iothread=1,size=24G,ssd=1
|
|
scsihw: virtio-scsi-single
|
|
smbios1: uuid=cxxxx-xxxx-xxxx-xxxx-xxxx
|
|
sockets: 1
|
|
usb0: host=1-1.1
|
|
```
|
|
|
|
### Master Node - network configuration
|
|
|
|
``` bash
|
|
ip a # check ethernet name
|
|
|
|
# removes automatic configuration as dhcp client
|
|
sed -i '/ens18/d' /etc/network/interfaces
|
|
|
|
cat <<EOF > /etc/network/interfaces.d/ens18
|
|
# my network configuration
|
|
auto ens18
|
|
iface ens18 inet static
|
|
address 192.168.14.9/24
|
|
gateway 192.168.0.1
|
|
EOF
|
|
|
|
```
|
|
|
|
### Master Node - k3s setup
|
|
|
|
``` bash
|
|
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.35.2+k3s1 sh -
|
|
```
|
|
|
|
**Taint NoSchedule on master node:**
|
|
|
|
kubectl taint nodes <master-node-name> node-role.kubernetes.io/control-plane=:NoSchedule
|
|
|
|
``` bash
|
|
kubectl taint nodes casa node-role.kubernetes.io/control-plane=:NoSchedule
|
|
```
|
|
|
|
## minion01 - worker node
|
|
|
|
### Minion01 - proxmox vm
|
|
|
|
*hosted on gaia.*
|
|
|
|
```yaml
|
|
agent: 1
|
|
balloon: 0
|
|
boot: order=scsi0;ide2;net0
|
|
cores: 4
|
|
cpu: host
|
|
ide2: none,media=cdrom
|
|
memory: 4096
|
|
meta: creation-qemu=10.1.2,ctime=1763219351
|
|
name: casa-minion-01
|
|
net0: virtio=BXX:XX:XX:XX:XX:XX,bridge=vmbr0,tag=xx
|
|
numa: 0
|
|
onboot: 1
|
|
ostype: l26
|
|
scsi0: fastcore:vm-XXX-disk-0,iothread=1,size=8G,ssd=1
|
|
scsi1: fastcore:vm-XXX-disk-1,iothread=1,size=16G,ssd=1
|
|
scsihw: virtio-scsi-single
|
|
smbios1: xxxx-xxxx-xxxx-xxxx-xxxx
|
|
sockets: 1
|
|
```
|
|
|
|
| Name | Value |
|
|
| --- | --- |
|
|
| **NAME** | minion01 |
|
|
| **VLAN** | casa |
|
|
| **IP** | 192.168.14.10 |
|
|
| **OS** | Debian GNU/Linux 12 (bookworm) |
|
|
|
|
### casa-minion-01 - k3s - setup
|
|
|
|
``` bash
|
|
# install k3s as agent / worker node
|
|
# execute on server to get token
|
|
# cat /var/lib/rancher/k3s/server/node-token
|
|
|
|
K3S_TOKEN="???"
|
|
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.35.2+k3s1 INSTALL_K3S_EXEC="agent --data-dir /dataDisk/k3s --server https://casa.dev.lan:6443 --token ${K3S_TOKEN}" sh -s -
|
|
```
|
|
|
|
Change kubectl -n kube-system edit configmap local-path-config on kube-system to set path to provisioner.
|
|
|
|
``` yaml
|
|
config.json: |-
|
|
{
|
|
"nodePathMap":[
|
|
{
|
|
"node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
|
|
"paths":["/var/lib/rancher/k3s/storage"]
|
|
},
|
|
{
|
|
"node":"casa-minion-01",
|
|
"paths":["/dataDisk/k3s/storage"]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
**Set node labels:**
|
|
|
|
``` bash
|
|
kubectl label node casa-minion-01 role=worker-node
|
|
```
|