1 changed files with 162 additions and 0 deletions
@ -0,0 +1,162 @@
|
||||
--- |
||||
title: Syncing files from Android to GNU/Linux |
||||
tags: [android, rsync, sshfs, ssh, sync] |
||||
updated: 2018-08-04 12:00 |
||||
description: Syncronization of files from Android to a GNU/Linux system |
||||
--- |
||||
|
||||
# Overview |
||||
|
||||
Backups of mobile phone content have been a problem I wanted to solve for years |
||||
but unfortunately the tools to do a clean job have always been inadeguate. |
||||
Also, I certainly didn't want to rely on third party services such as |
||||
Dropbox, Google, etc... |
||||
|
||||
The first solution was to use bluetooth with an APP I can't even remember the |
||||
name of. The problem was its slow speed and non-secureness. |
||||
|
||||
So I started using micro SD cards and manual copying... with the result of |
||||
losing several times all media on the card because its filesystem got |
||||
corrupted. |
||||
|
||||
After searching intensively I finally found a reliable and automatic way for |
||||
mobile phone backups. There is this neat APP called |
||||
[Primitive FTPD](https://github.com/wolpi/prim-ftpd) that includes an SFTP |
||||
server. The advantages of SFTP instead of using other systems are: |
||||
- simpleness |
||||
- encryption and authentication by default |
||||
- no need to open yet another port |
||||
- possibility to use a variety of copying tools such as Rsync. |
||||
|
||||
# Steps |
||||
|
||||
1. Set a fixed IP address for the phone. |
||||
2. Download the [APP from Fdroid](https://f-droid.org/en/packages/org.primftpd/). |
||||
3. Open the APP and choose a username and password in the settings menu. |
||||
4. Start the server and tap `yes` when asked for key generation. |
||||
5. Install SSHFS and Rsync on the target computer. |
||||
6. Adapt the following script I have written for the purpose: |
||||
|
||||
```shell |
||||
# Get the value from argc. |
||||
ALIAS="$1" |
||||
|
||||
# checksum option because it avoids problems with partial file transfers. |
||||
# Archive without keeping original user and group file attributes. |
||||
RSYNC_MAIN_OPTS='--checksum -rltD' |
||||
|
||||
# Set the devices variables here. |
||||
# Just edit this part and you should be good to go. |
||||
if [ "${ALIAS}" = "user0" ]; then |
||||
USERNAME='username0' |
||||
PASSWORD='password0' |
||||
ADDRESS='1.2.3.4' |
||||
PORT='65432' |
||||
MOUNT_DIR='/home/user/mountpoints/user0' |
||||
DST_DIR='/home/user/backup/user0' |
||||
AKA='user 0' |
||||
elif [ "${ALIAS}" = "user1" ]; then |
||||
USERNAME='username1' |
||||
PASSWORD='password1' |
||||
ADDRESS='5.6.7.8' |
||||
PORT='65432' |
||||
MOUNT_DIR='/home/user/mountpoints/user1' |
||||
DST_DIR='/home/user/backup/user1' |
||||
AKA='user 1' |
||||
elif [ "${ALIAS}" = '' ]; then |
||||
printf "unspecified device" 2>&1 |
||||
exit 1 |
||||
else |
||||
printf "unkown device" 2>&1 |
||||
exit 1 |
||||
fi |
||||
|
||||
# Mount only if not already mounted. |
||||
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")" |
||||
if [ -z "${exists_mount}" ]; then |
||||
echo "${PASSWORD}" | sshfs -o password_stdin -p "${PORT}" "${USERNAME}"@"${ADDRESS}": "${MOUNT_DIR}" |
||||
fi |
||||
|
||||
# Might not be mounted if device not in range or if its server is offline. |
||||
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")" |
||||
if [ -z "${exists_mount}" ]; then |
||||
echo ""${AKA}" not connected" |
||||
exit 1 |
||||
fi |
||||
|
||||
rsync "${RSYNC_MAIN_OPTS}" "${MOUNT_DIR}"/* "${DST_DIR}" |
||||
ret_rsync=$? |
||||
# You may want to send mails or write to log files instead of a simple print |
||||
# operation. |
||||
if [ "$ret_rsync" -ne 0 ]; then |
||||
echo "FAILED "${AKA}" ret_rsync=$ret_rsync" |
||||
else |
||||
echo "OK "${AKA}" |
||||
fi |
||||
|
||||
# Unmount after finishing. |
||||
fusermount3 -u "${MOUNT_DIR}" |
||||
``` |
||||
|
||||
Finally run it. If it works you can add it to crontab for automatic backups. |
||||
To run the script you simply need to pass the alias as argument: |
||||
```shell |
||||
$ <path_to_script> <alias> |
||||
``` |
||||
|
||||
with `<alias>` being the value of the `ALIAS` variable in the script. |
||||
|
||||
# Problems and explanation |
||||
|
||||
## (Rsync over SFTP) or (Rsync over directory over SSHFS) |
||||
|
||||
Since Primitive FTPD is an SFTP server, we can't use rsync |
||||
directly on SFTP because this is not supported by rsync (see: `$ man 1 rsync`). |
||||
For this reason we must first mount the directory using SSHFS and then |
||||
*rsyncit*. |
||||
|
||||
## (Public key) or (password authentication) |
||||
|
||||
I was not able to use public key authentication because of Pftpd reported |
||||
some obscure error with it. Initially I wasn't able to find Pftpd's log |
||||
file in the phone's flash memory but I noticed it after several sync |
||||
operations. Going through the file didn't clear the situation at all. |
||||
Even the |
||||
[project issues](https://github.com/wolpi/prim-ftpd/issues?utf8=%E2%9C%93&q=is%3Aissue+public+key+) |
||||
didn't solve the problem. So, the only remaining solution was to use |
||||
password authentication. Since all the process needs to be automatic, the |
||||
password needs to be stored plaintext and passed through the `password_stdin` |
||||
argument. This is of course *not the best setup*. |
||||
|
||||
## Storage type |
||||
|
||||
Depending on the Android version and device, the type of storage selected |
||||
from Pftpd might not work and behave differently than expected, so it is |
||||
kind of a trial and error operation. In my case the option `Plain old |
||||
filesystem` worked best, although the location that will be synced will |
||||
usually be `/storage/emulated/0` and cannot be changed to use, for example, |
||||
the external SD card. |
||||
|
||||
## Partial file transfers |
||||
|
||||
As you can see from the script's code, the option ``--checksum`` is useful |
||||
in case of partial file transfers, especially with flaky connections such |
||||
as the ones we find in mobile devices. |
||||
|
||||
# Alternatives |
||||
|
||||
There aren't many *free software* alternatives that work like this for Android. |
||||
What I found either didn't involve SSH or lack some functionalities. |
||||
[Syncopoli](https://gitlab.com/fengshaun/syncopoli), for example, was |
||||
another possible option. The readme file states that it is |
||||
an Rsync-based application optionally working via SSH and public key |
||||
authentication, however it is a client, not a server. |
||||
|
||||
Using a client avoids the computer polling for the connection, so it's a plus. |
||||
The only problem is that I need to do extra operations like maling and writing |
||||
to log files after the syncronization finishes. This is not possible with |
||||
Syncopoli. |
||||
|
||||
~ |
||||
|
||||
Have fun! |
Loading…
Reference in new issue