(asd) is a tiny pseudo-daemon designed to manage user specified directories referred to as sync targets from here on out, in tmpfs and to periodically sync them back to the physical disc (HDD/SSD). This is accomplished via a symlinking step and an innovative use of rsync to maintain synchronization between a tmpfs copy and media-bound backups. Additionally, asd features several crash recovery features.
Asd or Psd?
Benefits of Asd
Design goals of asd:
- Completely transparent user experience.
- Reduced wear to physical discs,
Since the sync target(s) is relocated into tmpfs (RAM disk), the corresponding onslaught of I/O associated with system usage of them is also redirected from the physical disc to RAM, thus reducing wear to the physical disc and also improving speed and responsiveness. The access time of RAM is on the order of nanoseconds while the access time of physical discs is on the order of milliseconds. This is a difference of six orders of magnitude or 1,000,000 times faster.
Setup and Installation
Install the package.
User managed settings are defined in
/etc/asd.conf which is included in the package.
- At a minimum, define the sync target(s) to be managed by asd in the WHATTOSYNC array. Syntax below.
- Optionally uncomment and define the location of your distro's tmpfs in the VOLATILE variable.
- Optionally enable the use of overlayfs to improve sync speed even further and use a smaller memory footprint. Note that this option requires your kernel be configured to use either the 'overlay' (Arch default) or 'overlayfs' (Ubuntu <15.05) kernel module. See the FAQ below for additional details on this feature.
WHATTOSYNC=('/var/lib/monitorix' '/srv/http' '/foo/bar')
WHATTOSYNC=( '/var/lib/monitorix' '/srv/http' '/foo/bar' )
Preview mode (parse)
The 'parse' option can be called to show users exactly what asd will do/is doing based on the entries in /etc/asd.conf as well as print out useful information such as dir size, paths, and if any recovery snapshots have been created.
$ asd p Anything-sync-daemon v5.61 on Arch Linux. Systemd service is currently active. Systemd resync service is currently active. Overlayfs v23 is currently active. Asd will manage the following per /run/asd.conf settings: owner/group id: root/0 target to manage: /srv/http/serve sync target: /srv/http/.serve-backup_asd tmpfs target: /dev/shm/asd-root/srv/http/serve dir size: 21M overlayfs size: 15M recovery dirs: 2 <- delete with the c option dir path/size: /srv/http/.serve-backup_asd-crashrecovery-20141105_124948 (17M) dir path/size: /srv/http/.serve-backup_asd-crashrecovery-20150124_062311 (21M) owner/group id: facade/100 target to manage: /home/facade/logs sync target: /home/facade/.logs-backup_asd tmpfs target: /dev/shm/asd-facadey/home/facade/logs dir size: 1.5M overlayfs size: 480K recovery dirs: none
The clean mode will delete ALL recovery snapshots that have accumulated. Run this only if you are sure that you want to delete them.
# asd c
Anything-sync-daemon v5.61 on Arch Linux. Deleting 2 crashrecovery dir(s) for sync target /srv/http/serve /srv/http/.serve-backup_asd-crashrecovery-20141105_124948 /srv/http/.serve-backup_asd-crashrecovery-20150124_062311
Do not call
/usr/bin/anything-sync-daemon to sync or to unsync directly. Instead use the provided service files.
Both a systemd service file and a timer are provided and should be used to interact with asd. The role of the timer is update the tmpfs copy/copies back to the disk which it does once per hour. The timer starts automatically with
Sync at more frequent intervals (optional)
The package provided timer syncs once per hour. Users may optionally redefine this behavior simply by extending the systemd unit. The example below changes the timer to sync once every ten minutes:
[Unit] Description=Timer for Arofile-sync-daemon - 10min [Timer] # Empty value resets the list of timers OnUnitActiveSec= OnUnitActiveSec=10min
Seefor additional options.
What is overlayfs and why do I want to use it?
Overlayfs is a simple union file-system mainlined in the Linux kernel version 3.18.0. Starting with asd version 5.54, overlayfs can be used to reduce the memory footprint of asd's tmpfs space and to speed up sync and unsync operations. The magic is in how the overlay mount only writes out data that has changed rather than the entire profile. The same recovery features asd uses in its default mode are also active when running in overlayfs mode. Overlayfs mode is enabled by uncommenting the USE_OVERLAYFS="yes" line in
/etc/asd.conf followed by a restart of the daemon.
There are several versions of overlayfs available to the Linux kernel in production in various distros. Versions 22 and lower have a module called 'overlayfs' while newer versions (23 and higher) have a module called 'overlay' -- note the lack of the 'fs' in the newer version. Asd will automatically detect the overlayfs available to your kernel if it is configured to use one of them.
My system crashed and did not sync back. What do I do?
Odds are the "last good" backup of your sync target(s) is just fine still sitting happily on your filesystem. Upon restarting asd (on a reboot for example), a check is preformed to see if the symlink to the tmpfs copy of your sync target is valid. If it is invalid, asd will snapshot the "last good" backup before it rotates it back into place. This is more for a sanity check that asd did no harm and that any data loss was a function of something else.
Where can I find this snapshot?
You will find the snapshot in the same directory as the sync target and it will contain a date-time-stamp that corresponds to the time at which the recovery took place. For example, a /foo/bar snapshot will be /foo/.bar-backup_asd-crashrecovery-20141221_070112 -- of course, the date_time suffix will be different for you.
How can I restore the snapshot?
- Confirm that there is no symlink to the sync target. If there is, asd did not stop correctly for other reasons.
- Move the "bad" copy of the sync target to a backup (do not blindly delete anything).
- Copy the snapshot directory to the expected sync target.
Example using /foo/bar
mv /foo/bar /foo/bar-bad cp -a /foo/.bar-backup_asd-crashrecovery-20141221_070112 /foo/bar
Can asd delete the snapshots automatically?
Yes, run asd with the "clean" switch to delete snapshots.
Post in the discussion thread with comments or concerns.