set cilium as CNI

cluster/README.md

@@ -186,8 +186,13 @@ EOF
 ### casa - control Plane - k3s setup
 
 ``` bash
-curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.36.1+k3s1 sh -
-```
+curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.36.1+k3s1 \
+INSTALL_K3S_EXEC="\
+--flannel-backend=none \
+--disable-network-policy \
+--disable=servicelb" \
+ sh -
+ ```
 
 **Taint NoSchedule on master node:**
 

services/cilium/deploy/README.md

@@ -0,0 +1,27 @@
+# Cilium
+
+<https://github.com/cilium/charts>
+<https://docs.cilium.io/en/stable/helm-reference/#helm-reference>
+
+## setup and deploy
+
+Using flux for reconciliation.
+
+**Restart:**
+
+Occasionally, Cilium's eBPF maps for L2 announcements can get "stale" after a configuration change (like switching from Local to Cluster). Perform a rolling restart of the Cilium pods to clear the state:
+
+```bash
+```
+
+``` bash
+POD_NAME=$(kubectl get pod -l 'app.kubernetes.io/name'=cilium-agent -n cilium -o jsonpath='{.items[0].metadata.name}')
+kubectl exec -n cilium  ${POD_NAME} -c cilium-agent -- cilium-dbg status
+```
+
+**list load balancers ip pools:**
+*Required after changes*
+
+```bash
+kubectl get CiliumLoadBalancerIPPool -A 
+```

services/cilium/deploy/app/helm-release.yaml

@@ -8,8 +8,17 @@ spec:
   chart:
     spec:
       chart: cilium
-      version: 1.18.2
+      version: 1.19.5
       sourceRef:
         kind: HelmRepository
         name: cilium
       interval: 40h
+      values:
+        kubeProxyReplacement: true
+        installCRDs: true
+        routingMode: native
+        enableIPv4Masquerade: false # using bpf.masquerade: true
+        autoDirectNodeRoutes: true
+        bpf:
+          masquerade: true
+          hostLegacyRouting: false # force use of bfp

services/cilium/deploy/flux/app-sync.yaml

@@ -7,6 +7,6 @@ spec:
   sourceRef:
     kind: GitRepository
     name: casa
-    namespace: flux-system
+    namespace: casa-limbosolutions-com
   path: ./services/cilium/deploy/app
   prune: true