Added a new post.

_posts/2018-08-04-android-backup-with-rsync.md

@@ -0,0 +1,162 @@
+---
+title: Syncing files from Android to GNU/Linux
+tags: [android, rsync, sshfs, ssh, sync]
+updated: 2018-08-04 12:00
+description: Syncronization of files from Android to a GNU/Linux system
+---
+
+# Overview
+
+Backups of mobile phone content have been a problem I wanted to solve for years 
+but unfortunately the tools to do a clean job have always been inadeguate.
+Also, I certainly didn't want to rely on third party services such as
+Dropbox, Google, etc...
+
+The first solution was to use bluetooth with an APP I can't even remember the 
+name of. The problem was its slow speed and non-secureness.
+
+So I started using micro SD cards and manual copying... with the result of 
+losing several times all media on the card because its filesystem got 
+corrupted.
+
+After searching intensively I finally found a reliable and automatic way for 
+mobile phone backups. There is this neat APP called 
+[Primitive FTPD](https://github.com/wolpi/prim-ftpd) that includes an SFTP 
+server. The advantages of SFTP instead of using other systems are:
+- simpleness
+- encryption and authentication by default
+- no need to open yet another port
+- possibility to use a variety of copying tools such as Rsync.
+
+# Steps
+
+1. Set a fixed IP address for the phone.
+2. Download the [APP from Fdroid](https://f-droid.org/en/packages/org.primftpd/).
+3. Open the APP and choose a username and password in the settings menu.
+4. Start the server and tap `yes` when asked for key generation.
+5. Install SSHFS and Rsync on the target computer.
+6. Adapt the following script I have written for the purpose:
+
+```shell
+# Get the value from argc.
+ALIAS="$1"
+
+# checksum option because it avoids problems with partial file transfers. 
+# Archive without keeping original user and group file attributes.
+RSYNC_MAIN_OPTS='--checksum -rltD'
+
+# Set the devices variables here.
+# Just edit this part and you should be good to go.
+if [ "${ALIAS}" = "user0" ]; then
+        USERNAME='username0'
+        PASSWORD='password0'
+        ADDRESS='1.2.3.4'
+        PORT='65432'
+        MOUNT_DIR='/home/user/mountpoints/user0'
+        DST_DIR='/home/user/backup/user0'
+        AKA='user 0'
+elif [ "${ALIAS}" = "user1" ]; then
+        USERNAME='username1'
+        PASSWORD='password1'
+        ADDRESS='5.6.7.8'
+        PORT='65432'
+        MOUNT_DIR='/home/user/mountpoints/user1'
+        DST_DIR='/home/user/backup/user1'
+        AKA='user 1'
+elif [ "${ALIAS}" = '' ]; then
+        printf "unspecified device" 2>&1
+        exit 1
+else
+        printf "unkown device" 2>&1
+        exit 1
+fi
+
+# Mount only if not already mounted.
+exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
+if [ -z "${exists_mount}" ]; then
+        echo "${PASSWORD}" | sshfs -o password_stdin -p "${PORT}" "${USERNAME}"@"${ADDRESS}": "${MOUNT_DIR}"
+fi
+
+# Might not be mounted if device not in range or if its server is offline.
+exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
+if [ -z "${exists_mount}" ]; then
+        echo ""${AKA}" not connected"
+        exit 1
+fi
+
+rsync "${RSYNC_MAIN_OPTS}" "${MOUNT_DIR}"/* "${DST_DIR}"
+ret_rsync=$?
+# You may want to send mails or write to log files instead of a simple print
+# operation.
+if [ "$ret_rsync" -ne 0 ]; then
+        echo "FAILED "${AKA}" ret_rsync=$ret_rsync"
+else
+        echo "OK "${AKA}"
+fi
+
+# Unmount after finishing.
+fusermount3 -u "${MOUNT_DIR}"
+```
+
+Finally run it. If it works you can add it to crontab for automatic backups.
+To run the script you simply need to pass the alias as argument:
+```shell
+$ <path_to_script> <alias>
+```
+
+with `<alias>` being the value of the `ALIAS` variable in the script.
+
+# Problems and explanation
+
+## (Rsync over SFTP) or (Rsync over directory over SSHFS)
+
+Since Primitive FTPD is an SFTP server, we can't use rsync
+directly on SFTP because this is not supported by rsync (see: `$ man 1 rsync`). 
+For this reason we must first mount the directory using SSHFS and then 
+*rsyncit*.
+
+## (Public key) or (password authentication)
+
+I was not able to use public key authentication because of Pftpd reported 
+some obscure error with it. Initially I wasn't able to find Pftpd's log 
+file in the phone's flash memory but I noticed it after several sync 
+operations. Going through the file didn't clear the situation at all.
+Even the
+[project issues](https://github.com/wolpi/prim-ftpd/issues?utf8=%E2%9C%93&q=is%3Aissue+public+key+) 
+didn't solve the problem. So, the only remaining solution was to use 
+password authentication.  Since all the process needs to be automatic, the 
+password needs to be stored plaintext and passed through the `password_stdin` 
+argument. This is of course *not the best setup*.
+
+## Storage type
+
+Depending on the Android version and device, the type of storage selected
+from Pftpd might not work and behave differently than expected, so it is 
+kind of a trial and error operation. In my case the option `Plain old 
+filesystem` worked best, although the location that will be synced will
+usually be `/storage/emulated/0` and cannot be changed to use, for example,
+the external SD card.
+
+## Partial file transfers
+
+As you can see from the script's code, the option ``--checksum`` is useful
+in case of partial file transfers, especially with flaky connections such
+as the ones we find in mobile devices.
+
+# Alternatives
+
+There aren't many *free software* alternatives that work like this for Android. 
+What I found either didn't involve SSH or lack some functionalities.
+[Syncopoli](https://gitlab.com/fengshaun/syncopoli), for example, was 
+another possible option. The readme file states that it is
+an Rsync-based application optionally working via SSH and public key 
+authentication, however it is a client, not a server. 
+
+Using a client avoids the computer polling for the connection, so it's a plus.
+The only problem is that I need to do extra operations like maling and writing 
+to log files after the syncronization finishes. This is not possible with 
+Syncopoli.
+
+~
+
+Have fun!