Commit Graph

176 Commits

Author SHA1 Message Date
Manorit Chawdhry 19204c7fd2
Update to v6.0.0
This version includes the following changes
-  Uses bind mounts instead of symlinks
-  Flock is used to handle the multiple syncs happening together
-  Backups are compressed using zstd
-  The backup process is optimized and uses link-dest flag to decrease the size of backups
2022-03-26 22:34:42 +05:30
Manorit Chawdhry bd8eb21318
update version code 2022-03-26 22:28:28 +05:30
Manorit Chawdhry 3c06867a5d
update documentation 2022-03-26 21:56:09 +05:30
Manorit Chawdhry 8d4b08b9b5 Implement bind mounting (#73)
This will avoid symlinking issues and race conditions which leave the
directory in inconsistent state. The change was inspired by the
implementation of @ThibaultLemaire with some minor tweaks to keep the
backup functionality intact. Directories with hardlinks aren't supported
anymore to keep the backup functionality intact

Co-authored-by: Thibault Lemaire <>
2022-03-26 21:32:57 +05:30
Manorit Chawdhry 418430a5cc optimize the syncing process using rsync --link-dest (#72)
This change won't preserve the hardlinks anymore right now, but will
allow the backups to be much faster and space efficient.

Also, this change maintains two states now, one directory refers to the
directory before running asd and the other directory keeps track of the
directory after running asd

If a proper unsync happens. The new directory will be moved atomically
to the place where the original directory existed, if not, the previous
directory will be backed up in tar.zstd format and the updated directory
will be moved back to the place of original directory ( since this
directory might have suffered syncing issues, backup is formed from the
old directory)
2022-03-26 21:32:57 +05:30
Manorit Chawdhry f0a09e5e8a compress backup directories using zstd (#71)
This can help save the storage space as well as decrease the writes to
HDD ( the main goal of asd ) incase of some inconsistent state

It adds a dependency of zstd as it is very efficient in terms of
performance as well as compression ratio which shouldn't give much
overhead while compression
2022-03-26 21:32:57 +05:30
Manorit Chawdhry 42a96f3408 verbose lock message 2022-03-26 21:32:56 +05:30
Manorit Chawdhry 8f6ba2741a shellcheck fixes 2022-03-26 21:32:56 +05:30
Manorit Chawdhry 1c5ae0cb3a fix naming while finding backup crash directories 2022-03-26 21:32:56 +05:30
Manorit Chawdhry db3da72032 change the lockfile name
instead of locking all instances, we will lock the seperate asd
instances based on ASDNAME. This will allow parallel rsync in cases
where asd don't interact with each other
2022-03-26 21:32:56 +05:30
Manorit Chawdhry dc283164f8 add flock to avoid simultaneous syncs
flock will now be a dependency of asd, asd will automatically lock
itself whenever run so that at a point of time, only one instance of asd
can be run. This can avoid race conditions that causes the rsyncs to
lockup due to multiple parallel syncs happening
2022-03-26 21:32:56 +05:30
Manorit Chawdhry aa35f5069a Bug fixes and robustness
- During unsync, it was trying to remove all the directories in all the
  directories instead of just the directory which had been unsynced

- In rare scenarios when some race condition occurs, the directory as
  well as the backup directory both of them existed which caused the move
  to fail. So, during ungraceful state check, since we have a backup, We
  can delete the directory so that the move is successful

  NOTE: This might still fail if in the short period of time, any other
  process/OS creates our sync directory again before the move is done
  ( This race can only be fixed by bind mounting suggestions I think )
2022-03-26 21:32:56 +05:30
Manorit Chawdhry 15c36ce002 leftover refactor fix 2022-03-26 21:32:56 +05:30
Manorit Chawdhry c8561f7e95
fixup! remove 7 year old warning 2022-03-26 19:33:54 +05:30
graysky f7f11a6122 remove 7 year old warning 2022-03-26 09:53:58 -04:00
graysky 8038b76de9 fixup docs: closes #76 2022-01-21 05:00:08 -05:00
Manorit f37e672de6
version bump: 5.86 2021-11-29 12:05:00 +05:30
Manorit Chawdhry 997b235140
add debug functionality (#69)
running asd with `DEBUG=1` environment variable will give additional 
insights on the working of asd and what it is doing, it can helpful for 
debugging issues in future
2021-11-29 11:30:41 +05:30
Manorit ee705e85a6
minor fixes 2021-11-27 13:41:21 +05:30
Simon Frei 6fe71d0fb6
Document ASDCONF and use it for initial check of config file existence (#48)
* Document ASDCONF and use it for initial check of config file existence

* Ensure no collisions between different instances

* review

* fix double hyphen when no ASDCONF
2021-10-11 12:30:47 +05:30
Manorit Chawdhry 82a09a590b
make crash recovery robust (#67) 2021-09-19 04:15:52 +05:30
Manorit Chawdhry 68ae792a64
shellcheck ci (#66) 2021-09-18 15:48:02 +05:30
Manorit Chawdhry f84ee22d9b
shellcheck fixes (#65) 2021-09-18 15:30:01 +05:30
Manorit a63a2e47ca rearrange du commands
This will allow the user to see why asd p command is taking too much
time as the directory sizes are being calculated at that point of time
2021-09-17 17:39:39 +05:30
Manorit 58309991af don't overwrite asd.conf if already present 2021-09-17 17:39:39 +05:30
Manorit d4b2928c38 fix typo 2021-09-17 17:39:39 +05:30
Manorit Chawdhry 5746934aef
update year in copyright (#63) 2021-09-15 13:31:33 +05:30
Danilo Spinella aba994cdf3 Rename license to follow naming convention 2021-09-14 15:55:56 +05:30
Danilo Spinella 0781b39172 Install asd.conf beside main script
asd.conf is needed by the main script in order to run, so it
would be better to install them togheter, improving installation
for users who are not using systemd nor upstart.
2021-09-14 15:55:56 +05:30
Glandos 9a5018af65
Allow using zram based device (#54)
Update bail filter to allow using Zram devices, which are not tmpfs. Also, better filtering of tmpfs, to be sure the grep does not match for path like /path/to/fake/tmpfs mounted.
2021-09-14 15:46:02 +05:30
graysky 616671210b Merge pull request #49 from bryant1410/master
Fix broken headings in Markdown files
2017-04-26 23:16:57 -04:00
Santiago Castro dc267127c0 Fix broken Markdown headings 2017-04-18 05:29:48 -03:00
graysky 5b3471ea25 bail if $VOLATILE is not tmpfs, fixes #33 2016-11-26 11:17:16 -05:00
graysky 81f2a3f8ad port feature #191 from psd into asd 2016-11-25 17:07:32 -05:00
graysky a74f67ade5 fix vim modeline 2016-11-25 16:46:01 -05:00
graysky e3d72ce9aa more robust internals 2016-11-25 16:42:37 -05:00
graysky ffe01d09d2 replace calls to ls with find to define arrays
update INSTALL with new dependency
2016-10-01 10:53:25 -04:00
graysky 20f81fad0d bump ver 2016-09-04 02:50:04 -04:00
graysky 18c5897c68 Merge pull request #43 from Naughtylus/autofix-trailing-slashes
Autofix trailing slashes
2016-09-04 02:47:35 -04:00
Naughtylus 1c8be59127 Auto-removing trailing slashes for VOLATILE 2016-09-03 22:49:30 +02:00
Naughtylus 2401e43d5d Auto-removing trailing slashes for directories in WHATTOSYNC 2016-09-03 22:45:58 +02:00
graysky 04d077a231 don't double prefix BACKUP target if it's already prefixed with a period 2016-09-01 02:10:59 -04:00
Moez Bouhlel 932a0d3510 du - dont deference all symbolic links
allow lines starting with double-quotes variables

enable space before comments and variables

update to 5.82
2016-08-04 15:08:54 -04:00
graysky ee89bbf22a force ln in Makefile 2016-07-19 03:06:31 -04:00
graysky e929e8797f support of older version of egrep to fix #39 2016-07-19 02:45:38 -04:00
graysky a2c0606d91 fix issue #39 2016-07-15 14:24:45 -04:00
graysky 8d0c621045 bump ver 2016-07-06 20:57:11 -04:00
graysky 9b929f64fc Merge pull request #38 from xorian/master
Use --no-target-directory with mv
2016-07-06 16:03:21 -04:00
Kenneth C. Schalk 6dbef1e3bb Use --no-target-directory with mv to make sure we don't move the backup or target inside another directory. This could happen if a parallel process creates the directory. 2016-07-06 11:24:16 -04:00
graysky 1c15a9112e add a check for unwanted characters in asd.conf 2016-07-04 16:52:33 -04:00