limbosolutions.com/git.limbosolutions.com
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

migrate borg backups to ssh backup services

Browse Source
This commit is contained in:
Márcio Fernandes
2025-05-04 00:48:13 +00:00
parent 5454d9c35a
commit 622c84c6be
2 changed files with 23 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
README.md
View File

@@ -5,18 +5,19 @@ Welcome to public repository of my [Git Server](https://git.limbosolutions.com)
Using [gitea](https://git.limbosolutions.com/kb/gitea) as git server. Using [gitea](https://git.limbosolutions.com/kb/gitea) as git server.
- [Setup](#setup) - [Setup](#setup)
- [kubernetes Namespace](#kubernetes-namespace) - [kubernetes Namespace](#kubernetes-namespace)
- [Add Help Chart](#add-help-chart) - [Add Help Chart](#add-help-chart)
- [Update Helm charts](#update-helm-charts) - [Update Helm charts](#update-helm-charts)
- [Install / Update](#install--update) - [Install / Update](#install--update)
- [ssh ingress](#ssh-ingress) - [ssh ingress](#ssh-ingress)
- [Backups](#backups) - [Backups](#backups)
- [secrets](#secrets) - [secrets](#secrets)
- [Proxmox Backup Server (kubernetes cron job)](#proxmox-backup-server-kubernetes-cron-job) - [Proxmox Backup Server (kubernetes cron job)](#proxmox-backup-server-kubernetes-cron-job)
- [Borg and Offsite sync (kubernetes cron job)](#borg-and-offsite-sync-kubernetes-cron-job) - [Borg and Offsite sync (kubernetes cron job)](#borg-and-offsite-sync-kubernetes-cron-job)
- [borgbackup sidekick](#borgbackup-sidekick) - [borgbackup sidekick](#borgbackup-sidekick)
- [Create borgbackup-sidekick pod](#create-borgbackup-sidekick-pod) - [Create borgbackup-sidekick pod](#create-borgbackup-sidekick-pod)
- [Remove borgbackup-sidekick pod](#remove-borgbackup-sidekick-pod) - [Remove borgbackup-sidekick pod](#remove-borgbackup-sidekick-pod)
## Setup ## Setup
@@ -63,7 +64,7 @@ kubectl apply -f ./gitea/ssh-ingress.yaml
### secrets ### secrets
``` bash ```bash
set -a set -a
source ./backups/.env source ./backups/.env
set +a set +a
@@ -72,8 +73,8 @@ envsubst < ./backups/gitea-backup-secrets.yaml | kubectl apply -n git-limbosolut
SSH_ID_RSA=$(echo -n "$SSH_ID_RSA" | base64 -w 0) SSH_ID_RSA=$(echo -n "$SSH_ID_RSA" | base64 -w 0)
BORG_KEY_FILE=$(echo -n "$BORG_KEY_FILE" | base64 -w 0) BORG_KEY_FILE=$(echo -n "$BORG_KEY_FILE" | base64 -w 0)
kubectl patch secret gitea-backup-secret --patch "{\"data\":{\"SSH_ID_RSA\":\"$SSH_ID_RSA\"}}" kubectl patch secret gitea-backup-secret --patch "{\"data\":{\"SSH_ID_RSA\":\"$SSH_ID_RSA\"}}" -n git-limbosolutions-com
kubectl patch secret gitea-backup-secret --patch "{\"data\":{\"BORG_KEY_FILE\":\"$BORG_KEY_FILE\"}}" kubectl patch secret gitea-backup-secret --patch "{\"data\":{\"BORG_KEY_FILE\":\"$BORG_KEY_FILE\"}}" -n git-limbosolutions-com
``` ```
### Proxmox Backup Server (kubernetes cron job) ### Proxmox Backup Server (kubernetes cron job)

12
backups/borgbackup-offsite-cronjob.yaml
View File

@@ -61,7 +61,7 @@ spec:
- name: REPO_SYNC_MAX_SIZE - name: REPO_SYNC_MAX_SIZE
value: "4294967296" # 4GB value: "6442450944" # 6GB
- name: BORG_KEY_FILE - name: BORG_KEY_FILE
value: /root/.borg/key value: /root/.borg/key
@@ -73,6 +73,10 @@ spec:
SCRIPT_START_TIME=$(date +%s) SCRIPT_START_TIME=$(date +%s)
# while true; do
# sleep 5
# done
borg create ${BORG_REPO}::postgresql-export-$(date +%Y%m%d%H%M%S) /data/postgresql-export borg create ${BORG_REPO}::postgresql-export-$(date +%Y%m%d%H%M%S) /data/postgresql-export
borg create ${BORG_REPO}::gitea-data-$(date +%Y%m%d%H%M%S) /data/gitea-data borg create ${BORG_REPO}::gitea-data-$(date +%Y%m%d%H%M%S) /data/gitea-data
@@ -83,16 +87,16 @@ spec:
borg compact ${BORG_REPO} borg compact ${BORG_REPO}
# check repo size # check repo size
REPO_SIZE_IN_BYTES=$(${BORG_RSH} mf@backupsrv01.dev.lan "du -b ~/borg-repos/git.limbosolutions.com --max-depth=0 | cut -f1 ") REPO_SIZE_IN_BYTES=$(${BORG_RSH} mf@backupsrv01.dev.lan -p 2222 "du -b ~/borg-repos/git.limbosolutions.com -d 0 | cut -f1 ")
echo "Repository size: $((REPO_SIZE_IN_BYTES / 1024 / 1024)) MB" echo "Repository size: $((REPO_SIZE_IN_BYTES / 1024 / 1024)) MB"
if [ $REPO_SIZE_IN_BYTES -gt $REPO_SYNC_MAX_SIZE ]; then \ if [ $REPO_SIZE_IN_BYTES -gt $REPO_SYNC_MAX_SIZE ]; then \
echo 'Repository size exceeds $REPO_SYNC_MAX_SIZE'; echo "ERROR: Repository size exceeds $REPO_SYNC_MAX_SIZE";
exit 1; exit 1;
else else
# Repository size is within limits for offsite sync # Repository size is within limits for offsite sync
# ssh to backup server and enforce rclone to onedrive # ssh to backup server and enforce rclone to onedrive
${BORG_RSH} mf@backupsrv01.dev.lan \ ${BORG_RSH} mf@backupsrv01.dev.lan -p 2222 \
"rclone sync ~/borg-repos/git.limbosolutions.com mf.onedrive:.backups/git.limbosolutions.com/borg" && \ "rclone sync ~/borg-repos/git.limbosolutions.com mf.onedrive:.backups/git.limbosolutions.com/borg" && \
SCRIPT_DURATION=$(($(date +%s) - SCRIPT_START_TIME)) && \ SCRIPT_DURATION=$(($(date +%s) - SCRIPT_START_TIME)) && \
echo "INFO: Finished Backup of git.limbosolutions.com (offsite) ($((SCRIPT_DURATION / 60 / 60)):$((SCRIPT_DURATION / 60)):$((SCRIPT_DURATION % 60))) " echo "INFO: Finished Backup of git.limbosolutions.com (offsite) ($((SCRIPT_DURATION / 60 / 60)):$((SCRIPT_DURATION / 60)):$((SCRIPT_DURATION % 60))) "