142

I've seen this with so many consoles (on Linux, Mac, ...), and with lots of different machines in many different networks. I can never pinpoint the exact reason, why this happens: All you have to do is log in to a machine via SSH. If the connection breaks for some reason (for simplicity, let's say the network cable was pulled), then sometimes the console just hangs forever - at other times, it just exits fine to the parent shell.

It's so annoying when this happens (e.g. you lose the command history.) Is there maybe a secret keyboard shortcut which can force an exit (Ctrl-C or Ctrl-D don't work)? And what's the reason for this random "bug" across all the implementations anyway?

Chris Lercher
  • 4,152
  • 9
  • 35
  • 41
  • This thread seems appropriate to mention [Mosh](https://mosh.org/) (Mobile Shell) which deals well with connection failures, including roaming (change of IP) and other stuff. – Ciprian Tomoiagă Mar 22 '17 at 08:14

6 Answers6

230

There is a "secret" keyboard shortcut to force an exit :~) From the frozen session, hit these keys in order: Enter~. The tilde (only after a newline) is recognized as an escape sequence by the ssh client, and the period tells the client to terminate it's business without further ado.

The long-hang behavior on communication issues is not a bug, the SSH session is hanging out hoping the other side will come back. If the network breaks, sometimes even days later you can get an SSH session back. Of course you can specifically tell it to give up and die with the sequence above. There are also various things you can do such as setting keep-alive timeouts in your client so that if it doesn't have an active link for a certain amount of time it shuts off on it's own, but the default behavior is to stay as connected as possible!

Edit: Another useful application of this interrupt key is to get the attention of the local ssh client and background it to get back to your local shell for a minute —say to get something from your history— then forground it to keep working remotely. Enter~ Ctrl+Z to send the ssh client to the background job queue of your local shell, then fg as normal to get it back.

Edit: When dealing with nested SSH sessions, you can add multiple tilde characters to only break out of one of the SSH sessions in the chain, but retain the others. For example, if you're nested in 3 levels, (i.e. you ssh from local->Machine1->Machine2->Machine3), Enter~. will get you back to your local session, Enter~~. will leave you in Machine1, and Enter~~~. will leave you in Machine2. This works for other escape sequences as well, such as moving the ssh session to background temporarily. The above works for any level of nesting, by just adding more tilde's.

Finally, you can use Enter~? to print a help menu of available escape commands.

TL;DR - the supported escape commands are Supported escape sequences:

 ~.   - terminate connection (and any multiplexed sessions)
 ~B   - send a BREAK to the remote system
 ~C   - open a command line
 ~R   - request rekey
 ~V/v - decrease/increase verbosity (LogLevel)
 ~^Z  - suspend ssh
 ~#   - list forwarded connections
 ~&   - background ssh (when waiting for connections to terminate)
 ~?   - this message
 ~~   - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)
kapad
  • 113
  • 6
Caleb
  • 11,813
  • 4
  • 36
  • 49
  • 12
    +1 for knowing the dodecatouple-secret-probation password for shooting sshd in the head. Did you find out about it the same way I did (mistyping `~/.somethingorother` after hitting enter)? – voretaq7 Jun 22 '11 at 18:22
  • Okay, +1 very cool - there seem to be some possible solutions. On my OS X terminal (which is currently frozen), the key strokes don't work (but that may have something to do with the (German) Mac keyboard, where tilde itself is a key combination: Alt-n)... I'll really need to find out, how to configure the keep-alive timeouts on my client! – Chris Lercher Jun 22 '11 at 18:27
  • On my keyboard it's a combination too, Shift+\` ... and I am able to do it from other keyboard where tilde is in an altgr key, but it should work. You only have to hit the keys in order, those are not a combination to be pressed at the same time. Three keys in sequence. – Caleb Jun 22 '11 at 18:31
  • 4
    @voretaq7: No I wasn't that smart, but when somebody did clue me in I went *"Really? So that's what was happening all those times my shell just went BANG! for no reason when I was typing?"*. It's not a common sequence except in mistypes of the one you mention, but it can happen. – Caleb Jun 22 '11 at 18:34
  • 1
    @Chris Lercher: I found some guys referencing this in an osx terminal ssh client, so I think the sequence is the same for you, but it is possible to start ssh with the `-e` flag to set a different character of your preference. – Caleb Jun 22 '11 at 19:01
  • 22
    Whereby "secret" he means "in the man page". – larsks Jun 22 '11 at 19:05
  • @Caleb: Thanks, I just found out, that the key sequence works in principle: I tried it on a non-frozen OS X terminal, and it worked. I also tried it again on another frozen terminal, and it worked there, too. Only on my original one, it doesn't work for some strange reason. – Chris Lercher Jun 22 '11 at 19:05
  • 4
    While many people don't know about this, it's really a secret in plain view. `man ssh` covers this under the `ESCAPE CHARACTERS` section. `~.` (Disconnect) and `~^Z` (background ssh) are very handy. – Stefan Lasiewski Jun 22 '11 at 19:56
  • 11
    Of course it's in the man page :) I only used the word secret because the OP did, and I was using tongue in cheek. The problem with this feature is that people don't know where to look for it, they expect control characters and such signals to be part of the shell or such. Once you know where to look or even what to ask, of course it's there. – Caleb Jun 22 '11 at 20:17
  • @Caleb: Exactly. I find 99% of the answers to my questions in man pages/blogs/... And I'm very happy, if somebody can help me find answers to the remaining 1% - and additionally gives me background information like you did. – Chris Lercher Jun 22 '11 at 20:49
  • *everything* is in the man page - but sometimes you don't go looking until you see odd behavior and debugging messages. Of course I'm also blessed with reliable connections :) – voretaq7 Jun 23 '11 at 04:39
21

SSH offers a keep-alive facility. Add the following to your local ~/.ssh/config (create if it doesn't exist):

ServerAliveInterval 15
ServerAliveCount 3

This setting will establish a keep-alive signal sent every 15 seconds through the secure tunnel. After three consecutive failures, the SSH client will exit.

Note that on some systems (including macOS 10.14) instead it needs to be:

ServerAliveInterval 15
ServerAliveCountMax 3

Taken from this answer on ask.ubuntu: https://askubuntu.com/a/29967/30266

Daniel Compton
  • 196
  • 10
krlmlr
  • 523
  • 1
  • 5
  • 17
  • ServerAliveCount will not work on Ubuntu 20.04 lts and RHEL 8 instead use ServerAliveCountMax for these two OS just like Krlmlr said. – MoonLight Dec 01 '22 at 08:42
10

The fact that it hangs is a function of TCP, not SSH. The application has no way of knowing that the TCP session/connection has been severed unless TCP informs the application using the connection that the connection no longer exists. From the perspective of each host, the TCP session is still in the Established state and there's nothing to say that a long idle (no data flowing) session is not valid other than a RST or a lack of response to a TCP keepalive packet (which is not universally implemented). That doesn't appear to be a bug in SSH to me, I'd expect that behavior.

joeqwerty
  • 109,901
  • 6
  • 81
  • 172
  • _The application has no way of knowing that the TCP session/connection has been severed unless TCP informs the application using the connection that the connection no longer exists._ The application can implement and use 'heart-beat' packets if it wants to know the effective connection status. SSH actually does this if ClientAlive*/ServerAlive* parameters are specified. – Alex Che Dec 12 '22 at 21:19
5

If the connection is properly hung any magic keystroke will not get through as the connection is already hung before you issue the key presses. You can tell the client to terminate but this won't affect the history kept (or not) at the server end.

While this doesn't actually answer the question it may help reduce the effects of a connection hang: when I'm working remotely (and even usually when I'm not) I run through screen (with or without the byobu wrapper depending on its availability) so that if there is any sort of connection drop my session, with all its history, is preserved and available in the state I left it when I reconnect.

David Spillett
  • 22,754
  • 45
  • 67
  • 7
    Your suggestion about screen is fine, but the first bit is actually not applicable to the issue. You don't need to get keys through to the remote side, you only have to get them through to the LOCAL ssh client! The OP want's his local shell back including history. SSH takes it over and doesn't respond to normal break sequences like CTRL-C because it passes those on. There is a way to get through and terminate the local client, see my answer. History on the local end is usually kept anyway if you login again, depending on shell configuration. – Caleb Jun 22 '11 at 18:20
  • 2
    @Caleb: the question specifically mentioned losing history, and the command history is stored server-side. To tell the server to do anything different with the history you need to get a message to the server to tell it to do something different with the history. Of course there are ways of making bash (and some other shells) record history lines immediately rather than collecting them in RAM until the user properly exists the shell with `exit`/`logout` which would solve the history issue without using `screen`. – David Spillett Jun 22 '11 at 19:44
1

In my case problem was in large MTU size. You can change MTU on router if you using NAT, but I change MTU on server:

sudo /sbin/ifconfig eth0 mtu 1036
sudo /etc/init.d/networking restart

On Windows you can also increase this key:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"TcpMaxDataRetransmissions"=dword:00000010
Vasin Yuriy
  • 111
  • 4
1

If you are on a windows machine and using PuTTY, you can go to Connection tab on the left and set Seconds between keep alive for this. eg: 15 secs.

DharsanB
  • 11
  • 1