rsync is a utility software and network protocol for Unix-like systems (with ports to Windows) that synchronizes files and directories from one location to another while minimizing data transfer by using delta encoding when appropriate.
You can also transfer files by rsync in the background, e.g.
rsync -e 'ssh -x' -avxSDHP email@example.com:/data/cournia .
type in the password, the transfer starts
To run the Schrodinger Suite on the cluster as well as many other programs, you require passwordless ssh.
To use passwordless ssh, the hosts to which you want to connect must be configured to satisfy the following requirements:
• Ansshd server must be running.
• RSA public key authentication must be enabled and empty passphrases must be allowed
in the sshd configuration.
Note: Public key authentication is enabled in OpenSSH by default.
The following steps allow you to use ssh between computers that share your login directory
without specifying a password.
1. Generate a public/private RSA key pair on a host of your choice, whose home directory is
shared with the remote hosts that you want to run jobs on:
ssh-keygen -t rsa
Note: When asked for a passphrase do not enter one; just press ENTER. If you specify a
passphrase it defeats the purpose of configuring passwordless ssh.
2. Add your public key to the list of keys allowed to log in to your account:
cat id_rsa.pub >> authorized_keys
cat id_rsa.pub >> authorized_keys2
The two separate files are necessary to support both OpenSSH 1.5 and OpenSSH 2.0 protocols.
Some versions use just one or the other of these files.
3. Suppress the confirmation dialog you ordinarily get when you connect to a machine for
the first time:
echo "StrictHostKeyChecking no" >> config
This is necessary if you want to use ssh non-interactively and you cannot get RSA signatures
for every host to which you want to allow connections in your known_hosts file
ahead of time.
4. Remove your known_hosts file:
This is necessary so that the new RSA key-pair mechanism is used for every host. Otherwise,
hosts to which you previously connected using passwords might not use the new
5. Make sure your home directory cannot be written by anyone but you:
chmod go-w ~
This is required before ssh will allow passwordless access to your account.
For each home directory that is not shared with that of the chosen host:
1. Copy the public and private keys to that home directory:
scp ~/.ssh/id_rsa* other-host/~/.ssh/
2. Connect to the host on which that home directory is mounted and change to the .ssh
3. Repeat Step 2 through Step 5 above.
4. Ensure that id_rsa (the private key) is readable and writeable only by the user:
chmod go-rwx ~/.ssh/id_rsa
Taken from the Job Control Guide of the Schrodinger Suite
Christina, Alexis, Matina, Dimitra, Aggeliki,