You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 

12 KiB

title tags updated description
My backup system [bash shell backup rsync LUKS cryptsetup sync shred] 2020-02-08 17:30 A detailed exaplanation of my backup system which is both encrypted and unencrypted for different purposes.

Introduction

Rsync is a very useful and flexible tool to do incremental backups and I have been using it for years without any problems. Recently I discovered the power of Rsync and Cryptsetup with LUKS to do encrypted and incremental backups as well.

Warning: part of the content of this post is deprecated. See [this newer post]({% post_url 2020-02-08-my-new-backup-system %}).

Partitioning scheme

I have chosen a partitioning scheme that separates bigger and rarely used files (on HDDs) from smaller and commonly used files (on SSDs). For this reason I need two different hard disks for backup:

Disk name Disk type Size (order of magnitude) Backup of mountpoint
backup root [1,2] HDD Gigabyte /
backup data [1,2] HDD Terabyte /data

Two different types of backups

I also keep two different types of incremental backups. It's not exactly the 3 2 1 rule but it's fine for my use case.

Backup name Online Offsite Automatic Encrypted Frequency Reason
backup x x daily to recover files deleted by mistake
backup enc x x monthly (at best) in case some disaster occurs

Totals

Disk name Backup name Device
backup root 1 backup /dev/sdx
backup data 1 backup /dev/sdy
backup root 2 backup enc /dev/sdw
backup data 2 backup enc /dev/sdz

Preliminary actions

Notes

  • A table of the notation used from this point on

    Mountpoint Partition Notes
    /mnt/backup_root /dev/sdx1 unencrypted root partition backup
    /mnt/backup_data /dev/sdy1 unencrypted data partition backup
    /mnt/backup_root_enc /dev/sdw1 encrypted root partition backup
    /mnt/backup_data_enc /dev/sdz1 encrypted data partition backup
  • Bash wildcards will be used in this section.

Steps

  1. Install Rsync, Cryptsetup, msmtp, S-nail, GNU Bash and GNU awk

  2. Make the mountpoints and change the permissions. Make them accessible only to the root user and group, just to be on the safe side. The first thing to do is to unmount everything under /mnt

    # umount -R /mnt
    # pushd /mnt
    # mkdir -p backup_{root,data}{_enc,}
    # chown 700 backup_{root,data}{_enc,}
    # popd
    
  3. Partition, format and mount the unencrypted hard disks. You should use fstab instead of mounting the filesystems manually

    # fdisk /dev/sd{x,y}
    # mkfs.ext4 /dev/sd{x,y}1
    # mount /dev/sd{x,y}1 /mnt/backup_{root,data}
    
  4. Shred, partition and format the encrypted backups as written in the Arch Wiki page. We will use some non-default options to make the encryption stronger

    # shred /dev/sd{x,y} -n1 --random-source=/dev/urandom --verbose
    # fdisk /dev/sd{w,z}
    # cryptsetup -v --type luks2 --cipher aes-xts-plain64 --key-size 512 --hash sha512 --iter-time 5000 --use-urandom --verify-passphrase luksFormat /dev/sd{w,z}1
    
  5. We will now mount and create a filesystem on the encrypted partitions. You will be prompted for a password which will then be used every time you call cryptsetup

    # cryptsetup open /dev/sd{w,z}1 {root,data}_enc
    # mkfs.ext4 /dev/mapper/{root,data}_enc
    # mount /dev/mapper/root_enc /mnt/backup_{root,data}_enc
    
  6. Unmount

    # umount /mnt/backup_{root,data}_enc
    # cryptsetup close {root,data}_enc
    
  7. Backup the LUKS headers of the encrypted drives in case they need to be recovered in the future.

    # pushd "${a_safe_place_for_backups}"
    # cryptsetup luksHeaderBackup /dev/sd{w,z}1 --header-backup-file backup_{root,data}_enc_luks_header.img
    # popd
    

The scripts

Incremental backup script

This script, called backup.sh, was inspired by an Arch Wiki page dating back some years ago, so I have to use the same license. The following code is licensed under the GNU Free Documentation License 1.3 or later.

#!/bin/bash
#
# backup.sh
#
# Copyright (C)  2019  Franco Masotti <franco.masotti@live.com>.
# Permission is granted to copy, distribute and/or modify this document
# under the terms of the GNU Free Documentation License, Version 1.3
# or any later version published by the Free Software Foundation;
# with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
# A copy of the license is included in the section entitled "GNU
# Free Documentation License".

declare -A BACKUP_ALIAS

# Get correct path before loading the configuration file.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
. ""${DIR}"/backup.conf"

SRC="${1}"
DST="${2}"

{ command -V rsync && command -V bash && command -V mail \
    && command -V printf && command -V sync; } \
    || exit 1
{ [ -z "${SRC}" ] || [ -z "${DST}" ]; } \
    && printf "%s\n" 'check inputs' 2>&1 && exit 1
[ ${UID} -ne 0 ] && printf "%s\n" 'user must be root' 2>&1 && exit 1

START=$(date +%s)
if [ "${SRC}" = '/*' ]; then
    # Use the exclude option to avoid infinite loops due to symlinks and to 
    # avoid backing up the /data mountpoint itself.
     rsync --archive --acls --xattrs --hard-links --delete /* "${DST}" --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/data}
elif [ "${SRC}" = '/data/*' ]; then
    rsync --archive --acls --xattrs --hard-links --delete /data/* "${DST}"
else
    rsync --archive --acls --xattrs --hard-links --delete "${SRC}" "${DST}"
fi

ret_rsync=${?}
# Flush data.
sync

FINISH=$(date +%s)
if [ ${ret_rsync} -eq 0 ]; then
    if [ "${LOG_TO_FILE}" = 'true' ]; then
        printf "%s\n" "[$(date '+%Y-%m-%d, %T') OK backup_ext_disks] "${DST}" ; \
$((${FINISH}-${START}))s" >> "${LOG_FILE}"
    fi
    if [ "${LOG_TO_EMAIL}" = 'true' ]; then
        printf "%s\n" "Backup on "${BACKUP_ALIAS["${DST}"]}" took $((${FINISH}-${START}))s" \
            | mail -r "${EMAIL_SENDER}" -s 'Backup complete' "${EMAIL_RECIPIENT}"
    fi
else
    if [ "${LOG_TO_FILE}" = 'true' ]; then
        printf "%s\n" "[$(date '+%Y-%m-%d, %T') FAILED backup_ext_disks \
ret_rsync=${ret_rsync}] "${DST}"" \
        >> /var/log/rsync.log
    fi
    if [ "${LOG_TO_EMAIL}" = 'true' ]; then
        printf "%s\n" "Backup error on "${BACKUP_ALIAS["${DST}"]}". Error code ${ret_rsync}" \
            | mail -r "${EMAIL_SENDER}" -s 'Backup error' "${EMAIL_RECIPIENT}"
    fi
fi

Configuration file

#
# backup.conf
#
# Copyright (C)  2019  Franco Masotti <franco.masotti@live.com>.
# Permission is granted to copy, distribute and/or modify this document
# under the terms of the GNU Free Documentation License, Version 1.3
# or any later version published by the Free Software Foundation;
# with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
# A copy of the license is included in the section entitled "GNU
# Free Documentation License".

# Friendlier names used in logs and emails.
BACKUP_ALIAS['/mnt/backup_root']='backup-root'
BACKUP_ALIAS['/mnt/backup_root_enc']='backup-root-encrypted'
BACKUP_ALIAS['/mnt/backup_data']='backup-data'
BACKUP_ALIAS['/mnt/backup_data_enc']='backup-data-encrypted'

LOG_TO_EMAIL='true'
EMAIL_SENDER='Computer <your.email@address.com>'
# You can use aliases if your mailing system is set up correctly.
EMAIL_RECIPIENT='all'

LOG_TO_FILE='true'
LOG_FILE='/var/log/rsync.log'

Encrypted backup script

To do our encrypted backups we will use LUKS and the previous script as a base. The idea is quite simple:

  1. mount the encrypted partition.
  2. do the backup using the previous script with the appropriate parameters.
  3. unmount the partition.

This new script is called backup_enc.sh and was inspired by this Arch Wiki page.

#!/bin/bash
#
# backup_enc.sh
#
# Copyright (C)  2019  Franco Masotti <franco.masotti@live.com>.
# Permission is granted to copy, distribute and/or modify this document
# under the terms of the GNU Free Documentation License, Version 1.3
# or any later version published by the Free Software Foundation;
# with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
# A copy of the license is included in the section entitled "GNU
# Free Documentation License".

#
# This backup is intended to be run manually.
#

declare -A BACKUP_ALIAS_ROOT
declare -A BACKUP_ALIAS_DATA

# Get correct path before loading the configuration file.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
. ""${DIR}"/backup_enc.conf"

WHAT="${1}"

{ command -V cryptsetup && command -V bash && command -V printf \
    && command -V mount && command -V umount && command -V awk; } \
    || exit 1
[ ${UID} -ne 0 ] && printf "%s\n" 'user must be root' 2>&1 && exit 1

if [ "${WHAT}" = 'root' ]; then
    SRC="${BACKUP_ALIAS_ROOT['SRC']}"
    DST="${BACKUP_ALIAS_ROOT['DST']}"
    UUID="${BACKUP_ALIAS_ROOT['UUID']}"
    NAME="${BACKUP_ALIAS_ROOT['DEVICE_MAPPER_NAME']}"
elif [ "${WHAT}" = 'data' ]; then
    SRC="${BACKUP_ALIAS_DATA['SRC']}"
    DST="${BACKUP_ALIAS_DATA['DST']}"
    UUID="${BACKUP_ALIAS_DATA['UUID']}"
    NAME="${BACKUP_ALIAS_DATA['DEVICE_MAPPER_NAME']}"
else
    printf "%s\n" 'error: configuration not set' 2>&1 && exit 1
fi

# Find partition via UUID.
dev="/dev/"$(lsblk -x name -i -o name,uuid | grep "${UUID}" | awk '{print $1}')""
[ -z "${dev}" ] && exit 1

cryptsetup open "${dev}" "${NAME}" && mount /dev/mapper/"${NAME}" "${DST}" \
&& ""${DIR}"/backup.sh" "${SRC}" "${DST}"

umount "${DST}" && cryptsetup close "${NAME}"

Configuration file

You must put the correct UUIDs of the partitions in the configuration file. To get these run:

$ lsblk -o name,uuid

and copy the appropriate ones.

#
# backup_enc.conf
#
# Copyright (C)  2019  Franco Masotti <franco.masotti@live.com>.
# Permission is granted to copy, distribute and/or modify this document
# under the terms of the GNU Free Documentation License, Version 1.3
# or any later version published by the Free Software Foundation;
# with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
# A copy of the license is included in the section entitled "GNU
# Free Documentation License".

BACKUP_ALIAS_ROOT['SRC']='/*'
BACKUP_ALIAS_ROOT['DST']='/mnt/backup_root_enc'
BACKUP_ALIAS_ROOT['UUID']='<put the UUID here>'
BACKUP_ALIAS_ROOT['DEVICE_MAPPER_NAME']='root_enc'

BACKUP_ALIAS_DATA['SRC']='/data/*'
BACKUP_ALIAS_DATA['DST']='/mnt/backup_data_enc'
BACKUP_ALIAS_DATA['UUID']='<put the UUID here>'
BACKUP_ALIAS_DATA['DEVICE_MAPPER_NAME']='data_enc'

First backups

Once you have everything in place, including the configuration files, you may start backups manually along with Rsync's --dry-run option in the script.

# ./backup.sh '/*' /mnt/backup_root
# ./backup.sh '/data/*' /mnt/backup_data

If everything works as expected, remove that option.

You can use Cron for the automatic backups. In my case I use Cronie. Just remember to call the backup scripts from their directory otherwise the configuration files won't be sourced.

~

Till next time!