rsync : Java Glossary

rsync is a file transfer program for Unix systems. rsync uses the rsync algorithm which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand. Rsync is usually teamed with SSH (Secure Shell) for privacy. Without rsync, with ordinary FTP (File Transfer Protocol), your passwords are sent in plain text, easy pickings for snoops. Normally you have a full time rsync server on TCP (Transmission Control Protocol) port 873, with clients coming in via SSH Rsync client software, though it is possible to run the server, as needed, via a remote SSH login.

FTP is utterly incompetent when it comes to preserving timestamps. It gets confused by time zones and DST (Daylight Saving Time). It was designed before the global Internet. Rsync fixes these problems.

rsync is clever. If the source and target files are similar but not identical it arranges to send just the differences. Further, it compresses the transmissions.

The problems with rsync are primarily political.

1. You need permission to install and run rsync on some server.
2. You need permission to tunnel through the client and server firewalls to get to the rsync server.

It uses XCOPY-like (Unix rcp-like) commands to copy groups of files from one machine to another, but much more efficiently than a classic copy. Security is handled by a combination of SSH encrypted communications, passwords, IP (Internet Protocol) lists permitted access, domain lists permitted access and userids permitted access.

To use rsync, you run a bash script on your desktop. The bash script, among other things invokes the rsync utility to efficiently copy files from your desktop machine to the server. You can also get rsync to run shell commands on the server to select files.

I wrote The Replicator, an rsync replacement, which uses just ordinary FTP and HTTP (Hypertext Transfer Protocol) to get around those two problems. You don’t need anything but vanilla generic FTP/FTPS and HTTP software running on the server. It is not as clever as rsync, but it fills the same niche.

Outstanding Questions

• If I try to upload a file to my server, while someone out on the web is downloading it, it will be busy. Does rsync just abort? Does it retry later? Using Netload, I often find it very difficult to complete an upload because it aborts every time I upload a file somebody is downloading.
• Do I need bash on my Windows desktop computer or can I use some other command processor?
• My normal case is I just want to upload a few files that have recently changed. Does rsync end up exchanging lists of filenames of my complete website on every connect, or is it clever and remembers the basic state of each side from the last exchange.
• Is there a way to do atomic uploads, so that the uploads show up to the outside world all at once?
• How does rsync compare in speed with FTP uploading? or other uploading techniques.