RCS::Intro Linux and HPC::Editors

Page Contents:



About this document

Page Group


Related Pages

Courses:



SSH, SFTP and SCP

1. 

Overview

SSH — or, more accurately, the SSH suite of utilities — is a means of securely:

  1. accessing (i.e., logging into) remote systems, e.g., HPC clusters;
  2. transferring files to and from such systems.
All network traffic in an SSH connection is encrypted. (It is a replacement for telnet, FTP, rsh and rcp.)

2. 

Other SSH Tutorials

3. 

Implementations of SSH

There are many different implementations of SSH available, i.e., different SSH clients (and servers) which one can download and install. Almost all Unix-like operating systems, including Linux, OS-X and Solaris, now use OpenSSH, or an implementation based on this. There is no official version of SSH for Microsoft Windows; the de facto standard is PuTTY.

4. 

Using SSH

To connect from the command-line of your local desktop machine or laptop to a remote machine, hpc.rcs.manchester.ac.uk, say, with username mpciixyz, enter 

  localprompt>  ssh mpciixyz@hpc.rcs.manchester.ac.uk
or 
  localprompt>  ssh -l mpciixyz hpc.rcs.manchester.ac.uk

5. 

Host Fingerprints and the OpenSSH Known Hosts File

5.1. 

First Time Login: The Key Fingerprint Challenge

The first time you attempt to login to a remote system you will be prompted like this: 

  The authenticity of host 'hpc.rcs.manchester.ac.uk (130.88.xyz.mno)' can't be established.  
  DSA key fingerprint is ba:1d:..:..:..:..:..:..:..:..:..:..:..:..:63:0a.  Are you sure you 
  want to continue connecting (yes/no)?
This feature of SSH is called host validation. If you answer "yes", the key fingerprint (ba:1d...) will be stored in your known hosts file, ~/.ssh/known_hosts — the next time you SSH to hpc.rcs.manchester.ac.uk you will not be prompted like this.

5.2. 

Warning: host key verfication failed!

On rare occasion, when you SSH to a system you will see a message like this 

  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  @       WARNING: POSSIBLE DNS SPOOFING DETECTED!          @
  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  The DSA host key for hpc.rcs.manchester.ac.uk has changed,
  and the key for the according IP address 130.88.xyz.mno
  is unchanged.  This could either mean that DNS SPOOFING is 
  happening or the IP address for the host and its host key 
  have changed at the same time.  Offending key for IP in 
  /home/suso/.ssh/known_hosts:10
or 
  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
  Someone could be eavesdropping on you right now (man-in-the-middle attack)!
  It is also possible that the DSA host key has just been changed.
  The fingerprint for the DSA key sent by the remote host is
  96:92:..:..:..:..:..:..:..:..:..:..:..:..:df:5b.
  Please contact your system administrator.
  Add correct host key in /home/suso/.ssh/known_hosts to get rid of this message.
  Offending key in /home/suso/.ssh/known_hosts:53
  DSA host key for arvo.suso.org has changed and you have requested strict
  checking.
  Host key verification failed.

Given a warning like this, before simply deleting the "offending" key from your known hosts file, you should determine if there is a good reason for the key to have changed. If there is (e.g., OS upgrade, or new hardware) then go ahead, delete the old key fingerprint and save the new one; otherwise contact the system administrator for advice.

Can't anyone pretend to be hpc.rcs.manchester.ac.uk?

Can't anyone grab the key fingerprint and pretend to be hpc.rcs.manchester.ac.uk? In short, no!

The key on each system — host in the jargon — comes in two parts, the private key and the public key. The private key is kept secure on the host. There is some mathematical jiggery-pokery going on — beyond the scope of this tutorial — which proves that the server has access to the private key associated with the public key. Assuming the private key really is secure, then, as stated in the previous paragraph, the SSH protocol (which the user does not see) ensures hosts cannot "spoof" key fingerprints.

More on Host Key Verification

6. 

Transferring Files: SFTP and SCP

7. 

Using SSH/SCP/SFTP on OS-X

OS-X comes with OpenSSH by default — simply start a terminal and and type in the commands as described above.

8. 

Using SSH/SCP/SFTP on MS Windows

There is no SSH client included in Microsoft Windows. PuTTY is the de facto client used. A simple GUI wraps the initial connection; once a connection has succeeded, a standard terminal is opened.

WinSCP. . .

9. 

Tunnelling of X-Windows (X11)

X-Windows allows the graphical output of a program running on one machine to appear on a different machine. For example, the user-interface of a programme running on a central HPC machine can be displayed on a user's local desktop (or laptop) by use of X-Windows (e.g., the GUI of a programme submitted to the batch system SGE via "qrsh").

X-Windows should usually be tunnelled through an SSH connection; without such a tunnel the connection is unencrypted. Furthermore, tunnelling X-Windows connections means that SSH compression may be used — X11 is a verbose protocol and benefits from greatly compression on slow connections. (For example, on my very asymmetrical 1Mb connection from home, plain X11 is unusable; SSH-compressed X11 is fine.)

For details, see the dedicated X-Windows page in this group.

10. 

Tunnelling of VNC

Virtual Network Computing (VNC) is a graphical virtual desktop-sharing system that uses the RFB protocol to remotely access another computer.

Superficially, from a user perspective, it seems similar to X-Windows. However, the server does not require a physical display, i.e., a screen/monitor; VNC can uses a virtual desktop.

What does this mean in practice for users?

A user can start a VNC server — that is, a virtual desktop — on a remote system, e.g., an HPC cluster, and view this virtual desktop in a (single) window on their office machine using a VNC viewer. (e.g, the virtual desktop could be used to display the GUI of an interactive job running on an HPC cluster — such as a qrsh job under SGE.) Furthermore, several such virtual desktops can be used simultaneously.

10.1. 

The Killer Feature

Such a qrsh job may take several hours:

In both cases, VNC comes to the rescue via its virtual desktop. Applications will run displayed on the virtual desktop whether or not this virtual desktop is itself currently being displayed on a physical monitor. This means that a user can start an application on a virtual desktop, then disconnect the viewer and reconnect later from the same computer or from elsewhere, as required, while the application continues to run untroubled.

10.2. 

Tunnelling

Like X-Windows, VNC is not encrypted; it is also verbose. So VNC also benefits from SSH tunnelling for reasons of security and speed. For details, see the dedicated VNC page in this group.

11. 

KDE Apps are Network-Transparent

Do you actually need to copy the file? e.g., Kwrite. . .

12. 

Authentication Methods

The SSH family supports several authentication methods. From a user's perspective, two methods dominate: username/password and key-based authentication. (Username/password authentication may actually be implemented by different SSH authentication methods, e.g., password or keyboard interactive, but most users don't know or care, or need to know!)

Username/Password
Time-honoured username/password combination. Just like the Telnet/FTP authentication system your Mom and Dad used to know (though encrypted). Users are very strongly encouraged to use long passwords with non-alpha-numeric characters in them.
Public Key
SSH can also authenticate using a private/public key-pair combination. The private key lives on the user's client desktop or laptop and should itself be protected by a (long) password (with non-alpha-numberic characters in it); the public key lives on the remote host (e.g. HPC cluster). Details of using this method are given in the section below [Section 13.].

This authentication method is considered much more secure than using a simple username/password combo.

13. 

SSH Keys

Overview

 -- overview of how-it-works

Practical details 

 -- generate
 -- copy public
 -- test

More

More details. . .

Generating the Key-Pair

A SSH private/public key-pair is generated using the ssh-keygen utility, for example: 

  prompt> ssh-keygen 

  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/mpciish2/.ssh/id_rsa): 
      # ...accept the default...

  Created directory '/home/mpciish2/.ssh'.
  Enter passphrase (empty for no passphrase): 
      # ...enter a long password which includes non-alpha-numeric characters...

  Enter same passphrase again: 
  Your identification has been saved in /home/mpciish2/.ssh/id_rsa.
      # ...i.e., private key...

  Your public key has been saved in /home/mpciish2/.ssh/id_rsa.pub.
  The key fingerprint is:
  52:86:5d:78:9f:3e:00:7a:a0:88:f3:e1:0d:51:d1:08 mpciish2@man2e.nw-grid.ac.uk
Two files are created: ~/.ssh/id_rsa, which is your private key — under no circumstances should this ever be copied to remote machines; ~/.ssh/id_rsa.pub, your public key, which you must install on the remote machine you wish to login to — see below. 

  ssh-keygen -b 2048 -t rsa

Installing the Public Key

To enable authentication to a remote server, e.g., the head node of a HPC cluster, the public key (only) must be copied to that remote system. Details are given in this section — but first, there is a gotcha!

If key-based authentication is required — i.e., username/password authentication is disabled — then how can one copy over a key? The assumption must be that either username/password authentication is currently enabled but will be disabled once key-based authentication is working; or, that the system-administrator will help by some means.

The required steps on the remote system are:

  1. If a directory called .ssh does not exist within your home-directory, then create it.
  2. Ensure the permissions of .ssh are set to be drwx------, i.e., only the owner has any access.
  3. Copy the public key into .ssh: append to .ssh/authorized_keys, if it exists; if it does not, create it!
These steps are automated by the ssh-copy-id script which comes as part of the OpenSSH distro (and is installed as standard on most Linuces).

Testing the Key-Pair

It is worth testing using the verbose mode of your SSH client both to ensure that the key-pair is working as expected and also to increase your understanding of SSH authentication. For example: 

  ssh -v sekhmet.umist.ac.uk

  OpenSSH_5.1p1 Debian-5, OpenSSL 0.9.8g 19 Oct 2007
      # ...the version of your SSH client...
  .
  debug1: Reading configuration data /etc/ssh/ssh_config
  debug1: Applying options for *
      # ...the client reads your desktop/laptop SSH configuration...
  .
  debug1: Connecting to sekhmet.umist.ac.uk [130.88.99.13] port 22.
  debug1: Connection established.
      # ...a network connection has been made to the remote system...
  .
  debug1: identity file /home/simonh/.ssh/id_rsa type 1
      # ...the client has found your private key...
  .
  debug1: Host 'sekhmet.umist.ac.uk' is known and matches the RSA host key.
  debug1: Found key in /home/simonh/.ssh/known_hosts:3
  debug1: ssh_rsa_verify: signature correct
      # ...the remote system matches the entry in your "known_hosts" file...
  .
  debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
  debug1: Next authentication method: gssapi-keyex
      # ...the various authentication methods which the client is gonna try 
      #    manifest themselves...
  .
  debug1: Next authentication method: publickey
  debug1: Trying private key: /home/simonh/.ssh/identity
  debug1: Offering public key: /home/simonh/.ssh/id_rsa
  .
  debug1: Enter passphrase for key '/home/simonh/.ssh/id_rsa
      # ...you are requested to enter the passphrase protecting your private key...
  .
  debug1: Server accepts key: pkalg ssh-rsa blen 277
  debug1: read PEM private key done: type RSA
      # ...the server accepts the offered key;  authentication succeeds...
  .
  .

14. 

SSH Agent

SSH Agent is a helper utility for key-based authentication; it does not work with username/password authentication.

If one frequently accesses a remote system using key-based authentication, repeatedly entering a key passphrase can become tiresome. Having a blank password is not a good idea — use SSH Agent instead: SSH Agent remembers passphrases associated with keys for a limited time and supplies them for you.

Example Usage of SSH Agent

Without use of ssh-agent, a passphrase is required: 

  prompt> ssh sekhmet.umist.ac.uk
  Enter passphrase for key '/home/simonh/.ssh/id_rsa': 
  Last login: Tue Apr 28 08:37:06 2009 from doolittle.rcs.m
  [simonh@sekhmet:~>

Set up ssh-sgent: 

  prompt> eval `ssh-agent`
  Agent pid 7758

Add a key to the agent — for one hour: 

  prompt> ssh-add -t 3600
  Enter passphrase for /home/simonh/.ssh/id_rsa: 
  Identity added: /home/simonh/.ssh/id_rsa (/home/simonh/.ssh/id_rsa)
  Lifetime set to 3600 seconds

Now authentication to sekhmet requires no manual input: 

  prompt> ssh sekhmet.umist.ac.uk
  Last login: Tue Apr 28 08:56:09 2009 from doolittle.rcs.m
  [simonh@sekhmet:~>