Added a new post.

This commit is contained in:
Franco Masotti 2018-08-07 17:10:11 +02:00
parent 6109674b1a
commit 56c65ceff5
1 changed files with 162 additions and 0 deletions

View File

@ -0,0 +1,162 @@
---
title: Syncing files from Android to GNU/Linux
tags: [android, rsync, sshfs, ssh, sync]
updated: 2018-08-04 12:00
description: Syncronization of files from Android to a GNU/Linux system
---
# Overview
Backups of mobile phone content have been a problem I wanted to solve for years
but unfortunately the tools to do a clean job have always been inadeguate.
Also, I certainly didn't want to rely on third party services such as
Dropbox, Google, etc...
The first solution was to use bluetooth with an APP I can't even remember the
name of. The problem was its slow speed and non-secureness.
So I started using micro SD cards and manual copying... with the result of
losing several times all media on the card because its filesystem got
corrupted.
After searching intensively I finally found a reliable and automatic way for
mobile phone backups. There is this neat APP called
[Primitive FTPD](https://github.com/wolpi/prim-ftpd) that includes an SFTP
server. The advantages of SFTP instead of using other systems are:
- simpleness
- encryption and authentication by default
- no need to open yet another port
- possibility to use a variety of copying tools such as Rsync.
# Steps
1. Set a fixed IP address for the phone.
2. Download the [APP from Fdroid](https://f-droid.org/en/packages/org.primftpd/).
3. Open the APP and choose a username and password in the settings menu.
4. Start the server and tap `yes` when asked for key generation.
5. Install SSHFS and Rsync on the target computer.
6. Adapt the following script I have written for the purpose:
```shell
# Get the value from argc.
ALIAS="$1"
# checksum option because it avoids problems with partial file transfers.
# Archive without keeping original user and group file attributes.
RSYNC_MAIN_OPTS='--checksum -rltD'
# Set the devices variables here.
# Just edit this part and you should be good to go.
if [ "${ALIAS}" = "user0" ]; then
USERNAME='username0'
PASSWORD='password0'
ADDRESS='1.2.3.4'
PORT='65432'
MOUNT_DIR='/home/user/mountpoints/user0'
DST_DIR='/home/user/backup/user0'
AKA='user 0'
elif [ "${ALIAS}" = "user1" ]; then
USERNAME='username1'
PASSWORD='password1'
ADDRESS='5.6.7.8'
PORT='65432'
MOUNT_DIR='/home/user/mountpoints/user1'
DST_DIR='/home/user/backup/user1'
AKA='user 1'
elif [ "${ALIAS}" = '' ]; then
printf "unspecified device" 2>&1
exit 1
else
printf "unkown device" 2>&1
exit 1
fi
# Mount only if not already mounted.
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
if [ -z "${exists_mount}" ]; then
echo "${PASSWORD}" | sshfs -o password_stdin -p "${PORT}" "${USERNAME}"@"${ADDRESS}": "${MOUNT_DIR}"
fi
# Might not be mounted if device not in range or if its server is offline.
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
if [ -z "${exists_mount}" ]; then
echo ""${AKA}" not connected"
exit 1
fi
rsync "${RSYNC_MAIN_OPTS}" "${MOUNT_DIR}"/* "${DST_DIR}"
ret_rsync=$?
# You may want to send mails or write to log files instead of a simple print
# operation.
if [ "$ret_rsync" -ne 0 ]; then
echo "FAILED "${AKA}" ret_rsync=$ret_rsync"
else
echo "OK "${AKA}"
fi
# Unmount after finishing.
fusermount3 -u "${MOUNT_DIR}"
```
Finally run it. If it works you can add it to crontab for automatic backups.
To run the script you simply need to pass the alias as argument:
```shell
$ <path_to_script> <alias>
```
with `<alias>` being the value of the `ALIAS` variable in the script.
# Problems and explanation
## (Rsync over SFTP) or (Rsync over directory over SSHFS)
Since Primitive FTPD is an SFTP server, we can't use rsync
directly on SFTP because this is not supported by rsync (see: `$ man 1 rsync`).
For this reason we must first mount the directory using SSHFS and then
*rsyncit*.
## (Public key) or (password authentication)
I was not able to use public key authentication because of Pftpd reported
some obscure error with it. Initially I wasn't able to find Pftpd's log
file in the phone's flash memory but I noticed it after several sync
operations. Going through the file didn't clear the situation at all.
Even the
[project issues](https://github.com/wolpi/prim-ftpd/issues?utf8=%E2%9C%93&q=is%3Aissue+public+key+)
didn't solve the problem. So, the only remaining solution was to use
password authentication. Since all the process needs to be automatic, the
password needs to be stored plaintext and passed through the `password_stdin`
argument. This is of course *not the best setup*.
## Storage type
Depending on the Android version and device, the type of storage selected
from Pftpd might not work and behave differently than expected, so it is
kind of a trial and error operation. In my case the option `Plain old
filesystem` worked best, although the location that will be synced will
usually be `/storage/emulated/0` and cannot be changed to use, for example,
the external SD card.
## Partial file transfers
As you can see from the script's code, the option ``--checksum`` is useful
in case of partial file transfers, especially with flaky connections such
as the ones we find in mobile devices.
# Alternatives
There aren't many *free software* alternatives that work like this for Android.
What I found either didn't involve SSH or lack some functionalities.
[Syncopoli](https://gitlab.com/fengshaun/syncopoli), for example, was
another possible option. The readme file states that it is
an Rsync-based application optionally working via SSH and public key
authentication, however it is a client, not a server.
Using a client avoids the computer polling for the connection, so it's a plus.
The only problem is that I need to do extra operations like maling and writing
to log files after the syncronization finishes. This is not possible with
Syncopoli.
~
Have fun!