Added a new post.
This commit is contained in:
parent
6109674b1a
commit
56c65ceff5
|
|
@ -0,0 +1,162 @@
|
|||
---
|
||||
title: Syncing files from Android to GNU/Linux
|
||||
tags: [android, rsync, sshfs, ssh, sync]
|
||||
updated: 2018-08-04 12:00
|
||||
description: Syncronization of files from Android to a GNU/Linux system
|
||||
---
|
||||
|
||||
# Overview
|
||||
|
||||
Backups of mobile phone content have been a problem I wanted to solve for years
|
||||
but unfortunately the tools to do a clean job have always been inadeguate.
|
||||
Also, I certainly didn't want to rely on third party services such as
|
||||
Dropbox, Google, etc...
|
||||
|
||||
The first solution was to use bluetooth with an APP I can't even remember the
|
||||
name of. The problem was its slow speed and non-secureness.
|
||||
|
||||
So I started using micro SD cards and manual copying... with the result of
|
||||
losing several times all media on the card because its filesystem got
|
||||
corrupted.
|
||||
|
||||
After searching intensively I finally found a reliable and automatic way for
|
||||
mobile phone backups. There is this neat APP called
|
||||
[Primitive FTPD](https://github.com/wolpi/prim-ftpd) that includes an SFTP
|
||||
server. The advantages of SFTP instead of using other systems are:
|
||||
- simpleness
|
||||
- encryption and authentication by default
|
||||
- no need to open yet another port
|
||||
- possibility to use a variety of copying tools such as Rsync.
|
||||
|
||||
# Steps
|
||||
|
||||
1. Set a fixed IP address for the phone.
|
||||
2. Download the [APP from Fdroid](https://f-droid.org/en/packages/org.primftpd/).
|
||||
3. Open the APP and choose a username and password in the settings menu.
|
||||
4. Start the server and tap `yes` when asked for key generation.
|
||||
5. Install SSHFS and Rsync on the target computer.
|
||||
6. Adapt the following script I have written for the purpose:
|
||||
|
||||
```shell
|
||||
# Get the value from argc.
|
||||
ALIAS="$1"
|
||||
|
||||
# checksum option because it avoids problems with partial file transfers.
|
||||
# Archive without keeping original user and group file attributes.
|
||||
RSYNC_MAIN_OPTS='--checksum -rltD'
|
||||
|
||||
# Set the devices variables here.
|
||||
# Just edit this part and you should be good to go.
|
||||
if [ "${ALIAS}" = "user0" ]; then
|
||||
USERNAME='username0'
|
||||
PASSWORD='password0'
|
||||
ADDRESS='1.2.3.4'
|
||||
PORT='65432'
|
||||
MOUNT_DIR='/home/user/mountpoints/user0'
|
||||
DST_DIR='/home/user/backup/user0'
|
||||
AKA='user 0'
|
||||
elif [ "${ALIAS}" = "user1" ]; then
|
||||
USERNAME='username1'
|
||||
PASSWORD='password1'
|
||||
ADDRESS='5.6.7.8'
|
||||
PORT='65432'
|
||||
MOUNT_DIR='/home/user/mountpoints/user1'
|
||||
DST_DIR='/home/user/backup/user1'
|
||||
AKA='user 1'
|
||||
elif [ "${ALIAS}" = '' ]; then
|
||||
printf "unspecified device" 2>&1
|
||||
exit 1
|
||||
else
|
||||
printf "unkown device" 2>&1
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Mount only if not already mounted.
|
||||
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
|
||||
if [ -z "${exists_mount}" ]; then
|
||||
echo "${PASSWORD}" | sshfs -o password_stdin -p "${PORT}" "${USERNAME}"@"${ADDRESS}": "${MOUNT_DIR}"
|
||||
fi
|
||||
|
||||
# Might not be mounted if device not in range or if its server is offline.
|
||||
exists_mount="$(mount | grep ""${USERNAME}"@"${ADDRESS}":")"
|
||||
if [ -z "${exists_mount}" ]; then
|
||||
echo ""${AKA}" not connected"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rsync "${RSYNC_MAIN_OPTS}" "${MOUNT_DIR}"/* "${DST_DIR}"
|
||||
ret_rsync=$?
|
||||
# You may want to send mails or write to log files instead of a simple print
|
||||
# operation.
|
||||
if [ "$ret_rsync" -ne 0 ]; then
|
||||
echo "FAILED "${AKA}" ret_rsync=$ret_rsync"
|
||||
else
|
||||
echo "OK "${AKA}"
|
||||
fi
|
||||
|
||||
# Unmount after finishing.
|
||||
fusermount3 -u "${MOUNT_DIR}"
|
||||
```
|
||||
|
||||
Finally run it. If it works you can add it to crontab for automatic backups.
|
||||
To run the script you simply need to pass the alias as argument:
|
||||
```shell
|
||||
$ <path_to_script> <alias>
|
||||
```
|
||||
|
||||
with `<alias>` being the value of the `ALIAS` variable in the script.
|
||||
|
||||
# Problems and explanation
|
||||
|
||||
## (Rsync over SFTP) or (Rsync over directory over SSHFS)
|
||||
|
||||
Since Primitive FTPD is an SFTP server, we can't use rsync
|
||||
directly on SFTP because this is not supported by rsync (see: `$ man 1 rsync`).
|
||||
For this reason we must first mount the directory using SSHFS and then
|
||||
*rsyncit*.
|
||||
|
||||
## (Public key) or (password authentication)
|
||||
|
||||
I was not able to use public key authentication because of Pftpd reported
|
||||
some obscure error with it. Initially I wasn't able to find Pftpd's log
|
||||
file in the phone's flash memory but I noticed it after several sync
|
||||
operations. Going through the file didn't clear the situation at all.
|
||||
Even the
|
||||
[project issues](https://github.com/wolpi/prim-ftpd/issues?utf8=%E2%9C%93&q=is%3Aissue+public+key+)
|
||||
didn't solve the problem. So, the only remaining solution was to use
|
||||
password authentication. Since all the process needs to be automatic, the
|
||||
password needs to be stored plaintext and passed through the `password_stdin`
|
||||
argument. This is of course *not the best setup*.
|
||||
|
||||
## Storage type
|
||||
|
||||
Depending on the Android version and device, the type of storage selected
|
||||
from Pftpd might not work and behave differently than expected, so it is
|
||||
kind of a trial and error operation. In my case the option `Plain old
|
||||
filesystem` worked best, although the location that will be synced will
|
||||
usually be `/storage/emulated/0` and cannot be changed to use, for example,
|
||||
the external SD card.
|
||||
|
||||
## Partial file transfers
|
||||
|
||||
As you can see from the script's code, the option ``--checksum`` is useful
|
||||
in case of partial file transfers, especially with flaky connections such
|
||||
as the ones we find in mobile devices.
|
||||
|
||||
# Alternatives
|
||||
|
||||
There aren't many *free software* alternatives that work like this for Android.
|
||||
What I found either didn't involve SSH or lack some functionalities.
|
||||
[Syncopoli](https://gitlab.com/fengshaun/syncopoli), for example, was
|
||||
another possible option. The readme file states that it is
|
||||
an Rsync-based application optionally working via SSH and public key
|
||||
authentication, however it is a client, not a server.
|
||||
|
||||
Using a client avoids the computer polling for the connection, so it's a plus.
|
||||
The only problem is that I need to do extra operations like maling and writing
|
||||
to log files after the syncronization finishes. This is not possible with
|
||||
Syncopoli.
|
||||
|
||||
~
|
||||
|
||||
Have fun!
|
||||
Loading…
Reference in New Issue