Wiki‎ > ‎

Cron, rsync and backup

posted Nov 21, 2014, 7:14 PM by Dong Xu   [ updated Apr 9, 2015, 10:15 PM ]

Rsync local files and folders

To copy the contents of one local folder into another, replacing the files in the destination folder, we use:

rsync -rtv source_folder/ destination_folder/

In the source_folder notice that I added a slash at the end, doing this prevents a new folder from being created, if we don't add the slash, a new folder named as the source folder will be created in the destination folder. So, if you want to copy the contents of a folder called Pictures into an existent folder which is also called Pictures but in a different location, you need to add the trailing slash, otherwise, a folder called Pictures is created inside the Pictures folder that we specify as destination.


Rsync creates a directory with the same name inside of destination directory

If you don't want another tests directory created, the correct command would be

rsync -avzp --del -e "ssh -p myport" user@hostname:/var/www/tests/ /var/www/tests

Note the / at the end of user@hostname:/var/www/tests/

http://serverfault.com/questions/529287/rsync-creates-a-directory-with-the-same-name-inside-of-destination-directory

When copying to the local file system I always use the following rsync options:
# rsync -avhW --no-compress --progress /src/ /dst/

Here's my reasoning:

-a is for archive, which preserves ownership, permissions etc.
-v is for verbose, so I can see what's happening (optional)
-h is for human-readable, so the transfer rate and file sizes are easier to read (optional)
-W is for copying whole files only, without delta-xfer algorithm which should reduce CPU 
load
--no-compress as there's no lack of bandwidth between local devices --progress so I can see the progress of large files (optional)
I've seen 17% faster transfers using the above rsync settings over the following tar command as suggested by another answer

http://serverfault.com/questions/43014/copying-a-large-directory-tree-locally-cp-or-rsync


backup scripts using rsync, not using --delete at this time because I want to keep deleted files for now.

vi /garlic/allspice_backup.sh
#!/bin/bash
backup_log=/var/log/backup_`date +%m-%d-%Y`.log
echo "Backup Start: `date +%m-%d-%Y-%T`" >> $backup_log
rsync -avhW --no-compress --progress /garlic /backup >> $backup_log 2>&1
echo "Backup End: `date +%m-%d-%Y-%T`" >> $backup_log

add crontab, daily backup at 11:55pm.

vi /etc/crontab
* * * * * root /usr/bin/touch /tmp/cron.lastrun
55 23 * * * root sh /root/backup.sh
00 12 1 * * root rm -rf /backup/*


Great backup tutorial

http://www.mikerubel.org/computers/rsync_snapshots/
http://techgage.com/article/mastering-rsync-and-bash-to-backup-your-linux-desktop-or-server/1/
http://www.howtogeek.com/135533/how-to-use-rsync-to-backup-your-data-on-linux/
http://www.evbackup.com/support-rsync-scripting-automation/
http://troy.jdmz.net/rsync/
https://www.lullabot.com/blog/article/simple-site-backups-rsync-ssh-and-sudo

Rsync tutorials:
http://everythinglinux.org/rsync/


Cron on Suse:
for root, vi /etc/crontab
for users, crontab -e
https://en.opensuse.org/SDB:Cron
https://www.novell.com/support/kb/doc.php?id=3842311

To check cron status:
ps -el | grep cron
an easier way to determine if cron is actually running, is via "rccron status", executed as root

Any output generated by cron jobs is usually sent via email to the owning user of the cron tab, in your case that should be the root user. Anything to find there, in addition to any messages in syslog (grep CRON /var/log/messages) as asked by Hans?

Another simple test entry could be

* * * * * /usr/bin/touch /tmp/cron.lastrun
The time stamp of /tmp/cron.lastrun should follow the current time, verified by "ls -l /tmp/cron.lastrun".

Do the status files of cron.daily etc suggest that cron is actually doing something, by displaying an appropriate (current) time stamp?

# ls -l /var/spool/cron/lastrun/
total 0
-rw-r--r-- 1 root root 0 Feb 13 13:15 cron.daily
-rw-r--r-- 1 root root 0 Feb 7 16:00 cron.weekly


For advanced backup, try rsnapshot: http://www.rsnapshot.org/
Read https://wiki.archlinux.org/index.php/Rsnapshot
http://blog.mypapit.net/2011/07/how-to-back-up-files-periodically-using-rsnapshot-and-nfs-in-ubuntu-linux.html

Comments