# Copyright (c) 2009, Mike Harrelson # All rights reserved. # /* Copyright (C) 1997-2009 Mike Harrelson SyncDir is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. SyncDir is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. The GNU General Public License is often shipped with GNU software, and is generally kept in a file called COPYING or LICENSE. If you do not have a copy of the license, write to the Free Software Foundation, 59 Temple Place, Suite 330, Boston, MA 02111 USA. */ The SyncDir Backup Suite is a simple backup/synchronization suite that I wrote in my spare time, mainly as an experiment. It is based on the SyncDir code based (included, but released seperately as well). It is composed of 6 utilities: syncback, syncbatch, syncrest, syncxpire, synclist, and syncdir. It is not intended as an enterprise level backup suite, but is vaguely similar to rdiff-backup (http://www.nongnu.org/rdiff-backup/) in concept. Basically, it creates a mirror of a directory tree, and then subsequent executions archive only changed versions of the files. === SyncBack 1.2.x The syncback utility is used to create the archives. It's usage is as follows: Usage: syncback [-fsqdci] [-X=] -f : do not wait for verification -s : skip crossing of file system boundaries -d : show differences only, do not backup anything -q : quiet mode, show no output -c : compare regular file contents not just timestamps -i : include .snapshot directories -X= : file with exclude list Normally syncback prompts for verification. But if being run as part of a script, the -f flag forces it to proceed non-interactively. The -s flag tells syncback to avoid crossing filesystems that may have other mount points within them. The -q flag tells syncback not to output anything (except on serious errors). Normally syncback ignores .snapshot directories. The -i flag forces it to include them. Syncback bases change determination on the modification time and/or size of regular files. The -c flag tells it to also examine the file contents. The -d flag tells syncback to only report what it would backup, but does not actually change anything. Finally, the -X flag allows you to specify a file containing a list of object names that are to be excluded from being backed up. The format of the file is one object name per line. Names can include a path (which must include the given source path) or just a filename with no path elements. The former will only match a single object name in the given directory tree. The latter will match any object with that name in any directory. As an example, if the _src_tree_ is '/var' and '/var/sadm/patch' is given, then '/var/sadm/patch' will be skipped when encountered. It must match exactly, however, including the leading'/'. 'var/sadm/patch' or 'sadm/patch' will not match. The backup repository tree for the same path should also be added to the excludes as well. This ensures that the given path, if present from an older backup, does not get inadvertently archived/deleted in the backup tree by mistake. This would happen because an excluded file or directory in the source tree would be treated as non-existant on the repository tree if not listed to be ignored as well. is the directory tree that is being backed-up. is the backup archive repository. Be sure not to reverse those arguments, as that would be VERY bad. The flags are always at the end. Syncback works by creating 'dot' files in the archive directory. These dot files are in the form of '..sbk__.__DATEID' and contain the directory contents for that particular backup set for that directory. This is used later by syncrest to determine which files/directories belong to that particular directory on restore. When files are changed from what was last archived, the previous files are renamed as '..sbk__FILENAME__DATEID' and the "current" version of the file becomes the new baseline in the archive. Directories are only renamed as such if they are not present in the directory being backed-up. Before starting the backup, the program outputs the new backup set ID that will be associated with this backup. NOTE that for archived files and directories, the DATEID portion of the filename represents the previous version of that object, NOT the version that existed in the source directory when that backup was run. While it's running, SyncBack reports feedback on what it is doing. There are typically 4 things you may see: "Adding ..." : This is a new file/directory that was not in the repository before and is being saved for the first time. "Updating ..." : This is a file object of the same type that is being updated with changes. "Archiving ..." : This is an object that is no longer present on the live directory, so it is being archived as a deleted item. "Syncing ..." : You should not normally see this, but the ownership and permissions of the most recent backup of that file are being synchronized with the live version being backed up, but the data content is the same. If an object changes file types, you may see both an "Archiving" and a "Adding" for the old and new item. === SyncBatch 1.2.x SyncBatch is a "batch" mode version of Syncback. It takes a single argument, a configuration file: Usage: syncbatch All syncback arguments and flags are put in the configuration file exactly as they would be if one were calling syncback directly. See the section on SyncBack (above) for details on the flags and order of arguments/parameters. The basic layout is: SOURCE_PATH ARCHIVE_PATH [-fsqdci] [-X=exclude_file] The meanings of the flags are: -f : do not wait for verification -s : skip crossing of file system boundaries -d : show differences only, do not backup anything -q : quiet mode, show no output -c : compare regular file contents not just timestamps -i : include .snapshot directories -X= : file with exclude list Even though SyncBatch is usually intended for non-interactive use, the -f flag is still required to force non-interactivity. Comments can be put in the configuration file and are denoted by a leading '#'. Of course, no paths or filename can include a '#' as a result. Blank lines are ignored. Lines with errors are skipped, though an error message is printed out on stderr. The primary difference between SyncBatch and multiple calls to SyncBack is that the same backup ID is used for all sets defined in the configuration file. === SyncList 1.2.x Synclist is another part of the SyncDir Backup Suite. The primary function of the utility is to list available backup sets that are currently stored in the archive repository. It's usage is simple: Usage: synclist [-rq] -r : reverse order, most recent first -q : quiet mode, print only backup ID# where is the directory containing the backup sets. This utility is used along with SyncRest and SyncXpire to determine which backup sets can be used for restoration or expiration. The format of the output is: Backup ID# Date/time -------------- --------------------- BACKUPSET_ID (mo/day/year hour:minute:seconds) ... ... The BACKUP_SET_ID is an encoded date identifier used to identify a specific backup set. The backup sets are listed in order from oldest to latest. '-r' can be used to reverse the order from newest to oldest. The '-q' flag causes only the backup ID#'s to be displayed. === SyncRest 1.2.x Syncrest is the utility used to actually restore from a previous backup. It's arguments are as follows: Usage: syncrest [-fqd] [-#] -f : do not wait for verification -d : show differences only, do not restore anything -q : quiet mode, show no output -# : specify directory retore depth, 0-9 The defines which directory contains the backup set repository. The specifies where the backup will be restored to. All existing content in the destination tree that does not match with what is in the backup set will be deleted. The is the backup set ID. This value must match to an existing backup set in the archive repository or an error will be returned. The -f is used to turn off interactive mode, mainly for scripting purposes. The -q flag is used to turn off all output except for serious errors. The -d flag is used to only report what will be restored, but does not actually change anything on the file system. To restore a subdirectory, simply specify the corresponding path within the archive repository that matches the specific directory to be restored. Finally the -# flag specifies an optional directory depth that can be restored. The default, 0, means infinite depth. A value of 1 will restore just the top level directory, but no subdirectories. 2 will include its subdirectories' content, but no further, etc. A maximum depth of 9 subdirectories can be specified without defaulting to unlimited. === SyncXpire 1.2.x Syncxpire is used to expire and remove old backup sets that are no longer wanted or needed. Its usage is as follows: Usage: syncxpire [-fqd] -f : do not wait for verification -d : show expires only, do not remove anything -q : quiet mode, show no output specifies the directory containing the backup set repository. The argument can be of two types. If then it specifies the number days (from the current date) that a backup set must be older than in order to be removed. Every backup set that is old or older is deleted. Optionally, a backup set ID can be specified instead of days. That backup ID does not have to actually exist as all backup sets equal to or older than the given set will be deleted. The -f flag forces the program to act non-interactively. The -q flag suppresses all output except for serious errors. The -d flag forces the program to output what would be deleted, but does not actually remove anything. === Syncdir 2.6.2 Usage: syncdir [-fmusqdicl] syncdir updates files and directories in to match what is in . Since this operation can be dangerous, syncdir normally prompts the user for verification. The following flags are defined: -f : Forces syncdir to bypass user verification and proceed directly with operations. Normally syncdir prompts to verify or confirm the operation with the user. -m : Tells syncdir that it is to mirror with exactly, thus making it an exact copy of . Anything in that is NOT in is deleted. Care should be taken when using this flag. -u : {default} Tells syncdir only to update files in with those in . Files in that are not in are preserved (unlike with the -m option). Also, for regular files, if the file in is "newer" than the version in , then it will NOT be updated. -s : Skip file system boundaries. Do not process source directories that reside on another file system. Only the directory itself will be sync'ed (permissions, ownership, etc), but not its contents. If this flag is specified, and a destination directory exists, then anything in the destination directory will be untouched. -q : Quiet mode. Do not print any progress output to stdout. Same as redirecting syncdir's processing output to /dev/null. -d : Show differences only. Do not make any changes to the filesystem. This option produces virtually the same output as if syncdir was run normally but does not really do anything. It is usefule for seeing changes to a filesystem before syncing them. -i : Include .snapshots directories which are normally excluded -c : Compare contents for regular files when timestamps and file sizes are the same. -l : If initial argument(s) are symlink(s), synchronize what they point to (aka follow) instead of treating them as symlinks. The -m and -u options should not be specified at the same time since they enable conflicting operation modes. Syncdir will use whichever one is specified last. Syncdir updates file and directory contents of so that they are in sync with those in . Affected are file contents, owner, group, permissions, and modify time. Syncdir can handle regular files, directories, symbolic links, character special files, block special files, named pipes (FIFOs), Unix sockets, and sparse files (regular files with space holes in them). Other files types like doors, etc. are not yet supported. File contents are only updated when the files' modification times or sizes are out of sync. Missing files, directores, links, etc. are duplicated if missing from . Since modification times are synchronized, successive runs of Syncdir on the same directories will only update anything in them if the modification times on those items have changed. Starting with version 2.5.9, Syncdir can now be run as a non-root user. However, ownership/group settings on objects may not be allowed if your OS does not support it. Also, certain special device files cannot be created without root priviledges. As of 2.6.2, hardlinks are not yet differentiated (planned for future release, maybe). Syncdir will NOT run if the effective UID is not the same as the real UID. Care should also be taken if the -l option is used. Verify the link targets before running syncdir. NOTE: It is possible for the owner of a file to set the modification time (and/or size) of a file in a way that can fool syncdir into thinking that it has not changed. The 'touch' program typically allows one to do this. Other programs, like tar, also usually set the modification time on objects. So much for security! In such cases -c can be used to force the contents to be compared for changes. There is a considerable speed cost, however. Please e-mail comments, suggestions, bugs to mikeh@mindspring.com Since this is the initial release of SyncBack Suite, it's possible (likely?) that there are a few lingering bugs, so please report any problems you encounter. Last updated: 11/18/2009