SSH, SFTP and SCP
1. |
Overview |
SSH — or, more accurately, the SSH suite of utilities — is a means of securely:
- accessing (i.e., logging into) remote systems, e.g., HPC clusters;
- transferring files to and from such systems.
2. |
Other SSH Tutorials |
- Suso Technology Services Inc. SSH Tutorial for Linux.
3. |
Implementations of SSH |
There are many different implementations of SSH available, i.e., different SSH clients (and servers) which one can download and install. Almost all Unix-like operating systems, including Linux, OS-X and Solaris, now use OpenSSH, or an implementation based on this. There is no official version of SSH for Microsoft Windows; the de facto standard is PuTTY.
4. |
Using SSH |
To connect from the command-line of your local desktop machine or laptop to a remote machine, hpc.rcs.manchester.ac.uk, say, with username mpciixyz, enter
localprompt> ssh mpciixyz@hpc.rcs.manchester.ac.ukor
localprompt> ssh -l mpciixyz hpc.rcs.manchester.ac.uk
5. |
Host Fingerprints and the OpenSSH Known Hosts File |
5.1. |
First Time Login: The Key Fingerprint Challenge |
The first time you attempt to login to a remote system you will be prompted like this:
The authenticity of host 'hpc.rcs.manchester.ac.uk (130.88.xyz.mno)' can't be established. DSA key fingerprint is ba:1d:..:..:..:..:..:..:..:..:..:..:..:..:63:0a. Are you sure you want to continue connecting (yes/no)?This feature of SSH is called host validation. If you answer "yes", the key fingerprint (ba:1d...) will be stored in your known hosts file, ~/.ssh/known_hosts — the next time you SSH to hpc.rcs.manchester.ac.uk you will not be prompted like this.
5.2. |
Warning: host key verfication failed! |
On rare occasion, when you SSH to a system you will see a message like this
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ The DSA host key for hpc.rcs.manchester.ac.uk has changed, and the key for the according IP address 130.88.xyz.mno is unchanged. This could either mean that DNS SPOOFING is happening or the IP address for the host and its host key have changed at the same time. Offending key for IP in /home/suso/.ssh/known_hosts:10or
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that the DSA host key has just been changed. The fingerprint for the DSA key sent by the remote host is 96:92:..:..:..:..:..:..:..:..:..:..:..:..:df:5b. Please contact your system administrator. Add correct host key in /home/suso/.ssh/known_hosts to get rid of this message. Offending key in /home/suso/.ssh/known_hosts:53 DSA host key for arvo.suso.org has changed and you have requested strict checking. Host key verification failed.
Given a warning like this, before simply deleting the "offending" key from your known hosts file, you should determine if there is a good reason for the key to have changed. If there is (e.g., OS upgrade, or new hardware) then go ahead, delete the old key fingerprint and save the new one; otherwise contact the system administrator for advice.
Can't anyone pretend to be hpc.rcs.manchester.ac.uk?
Can't anyone grab the key fingerprint and pretend to be hpc.rcs.manchester.ac.uk? In short, no!
The key on each system — host in the jargon — comes in two parts, the private key and the public key. The private key is kept secure on the host. There is some mathematical jiggery-pokery going on — beyond the scope of this tutorial — which proves that the server has access to the private key associated with the public key. Assuming the private key really is secure, then, as stated in the previous paragraph, the SSH protocol (which the user does not see) ensures hosts cannot "spoof" key fingerprints.
More on Host Key Verification
- Brian Hatch's SSH Host Key Protection.
6. |
Transferring Files: SFTP and SCP |
- command line
- gFTP
- Konqueror
7. |
Using SSH/SCP/SFTP on OS-X |
OS-X comes with OpenSSH by default — simply start a terminal and and type in the commands as described above.
8. |
Using SSH/SCP/SFTP on MS Windows |
There is no SSH client included in Microsoft Windows. PuTTY is the de facto client used. A simple GUI wraps the initial connection; once a connection has succeeded, a standard terminal is opened.
WinSCP. . .
9. |
Tunnelling of X-Windows (X11) |
X-Windows allows the graphical output of a program running on one machine to appear on a different machine. For example, the user-interface of a programme running on a central HPC machine can be displayed on a user's local desktop (or laptop) by use of X-Windows (e.g., the GUI of a programme submitted to the batch system SGE via "qrsh").
X-Windows should usually be tunnelled through an SSH connection; without such a tunnel the connection is unencrypted. Furthermore, tunnelling X-Windows connections means that SSH compression may be used — X11 is a verbose protocol and benefits from greatly compression on slow connections. (For example, on my very asymmetrical 1Mb connection from home, plain X11 is unusable; SSH-compressed X11 is fine.)
For details, see the dedicated X-Windows page in this group.
10. |
Tunnelling of VNC |
Virtual Network Computing (VNC) is a graphical virtual desktop-sharing system that uses the RFB protocol to remotely access another computer.
Superficially, from a user perspective, it seems similar to X-Windows. However, the server does not require a physical display, i.e., a screen/monitor; VNC can uses a virtual desktop.
What does this mean in practice for users?
A user can start a VNC server — that is, a virtual desktop — on a remote system, e.g., an HPC cluster, and view this virtual desktop in a (single) window on their office machine using a VNC viewer. (e.g, the virtual desktop could be used to display the GUI of an interactive job running on an HPC cluster — such as a qrsh job under SGE.) Furthermore, several such virtual desktops can be used simultaneously.
10.1. |
The Killer Feature |
Such a qrsh job may take several hours:
- the user may wish to leave the office and go home during that time, yet continue to monitor or interact with the job;
- the computer on which the job is being displayed may be shutdown (or may crash) which will kill the running job.
10.2. |
Tunnelling |
Like X-Windows, VNC is not encrypted; it is also verbose. So VNC also benefits from SSH tunnelling for reasons of security and speed. For details, see the dedicated VNC page in this group.
11. |
KDE Apps are Network-Transparent |
Do you actually need to copy the file? e.g., Kwrite. . .
12. |
Authentication Methods |
The SSH family supports several authentication methods. From a user's perspective, two methods dominate: username/password and key-based authentication. (Username/password authentication may actually be implemented by different SSH authentication methods, e.g., password or keyboard interactive, but most users don't know or care, or need to know!)
- Username/Password
- Time-honoured username/password combination. Just like the Telnet/FTP authentication system your Mom and Dad used to know (though encrypted). Users are very strongly encouraged to use long passwords with non-alpha-numeric characters in them.
- Public Key
-
SSH can also authenticate using a private/public key-pair combination.
The private key lives on the user's client desktop or laptop and should
itself be protected by a (long) password (with non-alpha-numberic
characters in it); the public key lives on the remote host (e.g.
HPC cluster). Details of using this method are given in the
section below [Section 13.].
This authentication method is considered much more secure than using a simple username/password combo.
13. |
SSH Keys |
Overview
-- overview of how-it-works
Practical details
-- generate -- copy public -- test
More
More details. . .
Generating the Key-Pair
A SSH private/public key-pair is generated using the ssh-keygen utility, for example:
prompt> ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/mpciish2/.ssh/id_rsa): # ...accept the default... Created directory '/home/mpciish2/.ssh'. Enter passphrase (empty for no passphrase): # ...enter a long password which includes non-alpha-numeric characters... Enter same passphrase again: Your identification has been saved in /home/mpciish2/.ssh/id_rsa. # ...i.e., private key... Your public key has been saved in /home/mpciish2/.ssh/id_rsa.pub. The key fingerprint is: 52:86:5d:78:9f:3e:00:7a:a0:88:f3:e1:0d:51:d1:08 mpciish2@man2e.nw-grid.ac.ukTwo files are created: ~/.ssh/id_rsa, which is your private key — under no circumstances should this ever be copied to remote machines; ~/.ssh/id_rsa.pub, your public key, which you must install on the remote machine you wish to login to — see below.
ssh-keygen -b 2048 -t rsa
Installing the Public Key
To enable authentication to a remote server, e.g., the head node of a HPC cluster, the public key (only) must be copied to that remote system. Details are given in this section — but first, there is a gotcha!
If key-based authentication is required — i.e., username/password authentication is disabled — then how can one copy over a key? The assumption must be that either username/password authentication is currently enabled but will be disabled once key-based authentication is working; or, that the system-administrator will help by some means.
The required steps on the remote system are:
- If a directory called .ssh does not exist within your home-directory, then create it.
- Ensure the permissions of .ssh are set to be drwx------, i.e., only the owner has any access.
- Copy the public key into .ssh: append to .ssh/authorized_keys, if it exists; if it does not, create it!
Testing the Key-Pair
It is worth testing using the verbose mode of your SSH client both to ensure that the key-pair is working as expected and also to increase your understanding of SSH authentication. For example:
ssh -v sekhmet.umist.ac.uk OpenSSH_5.1p1 Debian-5, OpenSSL 0.9.8g 19 Oct 2007 # ...the version of your SSH client... . debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * # ...the client reads your desktop/laptop SSH configuration... . debug1: Connecting to sekhmet.umist.ac.uk [130.88.99.13] port 22. debug1: Connection established. # ...a network connection has been made to the remote system... . debug1: identity file /home/simonh/.ssh/id_rsa type 1 # ...the client has found your private key... . debug1: Host 'sekhmet.umist.ac.uk' is known and matches the RSA host key. debug1: Found key in /home/simonh/.ssh/known_hosts:3 debug1: ssh_rsa_verify: signature correct # ...the remote system matches the entry in your "known_hosts" file... . debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive debug1: Next authentication method: gssapi-keyex # ...the various authentication methods which the client is gonna try # manifest themselves... . debug1: Next authentication method: publickey debug1: Trying private key: /home/simonh/.ssh/identity debug1: Offering public key: /home/simonh/.ssh/id_rsa . debug1: Enter passphrase for key '/home/simonh/.ssh/id_rsa # ...you are requested to enter the passphrase protecting your private key... . debug1: Server accepts key: pkalg ssh-rsa blen 277 debug1: read PEM private key done: type RSA # ...the server accepts the offered key; authentication succeeds... . .
14. |
SSH Agent |
SSH Agent is a helper utility for key-based authentication; it does not work with username/password authentication.
If one frequently accesses a remote system using key-based authentication, repeatedly entering a key passphrase can become tiresome. Having a blank password is not a good idea — use SSH Agent instead: SSH Agent remembers passphrases associated with keys for a limited time and supplies them for you.
Example Usage of SSH Agent
Without use of ssh-agent, a passphrase is required:
prompt> ssh sekhmet.umist.ac.uk Enter passphrase for key '/home/simonh/.ssh/id_rsa': Last login: Tue Apr 28 08:37:06 2009 from doolittle.rcs.m [simonh@sekhmet:~>
Set up ssh-sgent:
prompt> eval `ssh-agent` Agent pid 7758
Add a key to the agent — for one hour:
prompt> ssh-add -t 3600 Enter passphrase for /home/simonh/.ssh/id_rsa: Identity added: /home/simonh/.ssh/id_rsa (/home/simonh/.ssh/id_rsa) Lifetime set to 3600 seconds
Now authentication to sekhmet requires no manual input:
prompt> ssh sekhmet.umist.ac.uk Last login: Tue Apr 28 08:56:09 2009 from doolittle.rcs.m [simonh@sekhmet:~>