Securely Transfer Files Over a Virtual Private Cloud (VPC) with SCP or Rsync

Introduction

When using commands like scp or rsync with the public IP address of your server, you will automatically transfer the files over the (public) internet. If you have have two instances with private networking enabled you can also transfer these files over the private network and prevent extra charges for the used bandwidth.

Prerequisites

• Two instances (we'll call them server_a and server_b) with private networking enabled
• Both instances should have private IP addresses configured in the same subnet (see Configuring Private Network)
• SSH connectivity should be possible between both instances
• A user which is not root (we'll call it your_user)

Generating and using SSH keys

Transferring files is possible using username and password authentication, but it's much safer to use SSH keys. The generation of the public-private key is described in How Do I Generate SSH Keys?. If you decide to skip this section then you'll have to enter the remote user's password in every command.

Assuming that the public-private key pair on server_a is located in ~/.ssh/id_rsa, you can run the following command to transfer your public key to server_b. Use the private IP address of server_b.

ssh-copy-id your_user@192.168.0.101

When prompted, give the password for your_user.

ssh-copy-id your_user@192.168.0.101 /usr/bin/ssh-copy-id: INFO: Source of
key(s) to be installed: "/home/your_user/.ssh/id_rsa.pub" The
authenticity of host '192.168.0.101 (192.168.0.101)' can't be
established. ECDSA key fingerprint is
SHA256:g9dfqycqU25b567/HDjPTqaQqKhep/fysNCQAG9yJG4. ECDSA key
fingerprint is MD5:41:67:be:68:51:9b:38:a8:95:82:71:47:f1:35:39:66.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s),
to filter out any that are already installed /usr/bin/ssh-copy-id:
INFO: 1 key(s) remain to be installed -- if you are prompted now it is
to install the new keys your_user@192.168.0.101's password:

Number of key(s) added: 1

Now try logging into the machine.

ssh your_user@192.168.0.101

Check to make sure that only the key(s) you wanted were added.

If you now SSH from server_a to server_b, it will no longer ask for a password. In case the public-private key pair is not located in ~/.ssh/id_rsa then you can specify it's location with the -i parameter (you will also need this parameter with the scp and rsync command).

ssh-copy-id -i /path/to/your/public_key your_user@192.168.0.101 
ssh -i /path/to/your/private_key your_user@192.168.0.101 

Transferring files with SCP

SCP stands for Secure Copy and it transfers all data over an SSH connection. To copy a single file, ~/myfile, from server_a to the /tmp directory on server_b we can issue the following command.

scp ~./myfile your_user@192.168.0.101:/tmp

To copy a complete folder with all it's contents (including symbolic links) you can add the -r option

scp -r ~./mydir your_user@192.168.0.101:/tmp

Transferring files with Rsync

Rsync is a versatile tool to copy files, it's most often used to synchronize the content of two locations. It uses the same secure SSH tunnel to transfer data. A complete folder can be synchronized to a remote /tmp dir with the following command

rsync -av ~/mydir your_user@192.168.0.101:/tmp

The -v option increases verbosity so that you can follow the progress of the transfer. The -a option enables 'archive mode' which copies your files recursively while preserving attributes like owner, group and permissions.