I've got a problem with SSH on a Debian Lenny based server (it's a vHost within a Xen environment, booted on a Xen kernel). I hope someone can help me with this.
The SSH connection seems somehow getting screwed up frequently when the terminal overflows (new lines beyond the bottom of the terminal, usually forcing it to scroll). The connection gets lost but not regularly disconnected. It nearly always happens when I do the following:
- an existing SSH connection gets disconnected (regularly)
- I order putty to reestablish the connection
- login-prompt appears at the very bottom of the putty terminal window
- I enter my login-name, press the enter key
- I'm asked for the password, I enter it, press the enter key and BOOM! Nothing more happens. I have to reconnect again.
So it is reproducable.
I'm not totally sure if the connection crashes before or after I enter the password.
Furthermore it also happens when there is much text to be displayed (for example when I compile something or do an ls -l on a directory with many entries).
Using 'screen', however, helps to reduces the frequency of occurence but doesn't solve the problem completely.
It's occurence is independent from which terminal software I use. I mostly use putty but it also happens with other clients.
I certainly hope somebody can help me solving this problem.
Thanks in advance!
//edit: I've just made a Wireshark trace of the ssh connection and there is nothing, I repeat, nothing different between the working and the failing connection (at least aside from frame numbers, ports and times that obviously can't be equal). This leads me to the assumption that the error has to happen on the server's side.
-
From your initial descriptions, it appears that some special character communication is being escaped to do something to the SSH shell itself.
These are some things you can try,- Keep a wireshark track to check if the SSH connection actually shows a TCP-FIN sequence when you get a stall (check how this happens in the wireshark view when you exit a normal connection for a reference).
- if the FIN sequence is not seen the SSH has stalled in some way (not closed)
- When you are stalled try the key sequence Ctrl+Q
You can also try to eliminate specific end-point behavior with these things if any of them can be done in your setup,
- Try to reproduce from another client machine and the same server
- Try to reproduce with the same client machine on a different server
- Try to run the client with '
-v -v -v
' debug logging options - Try to run the server with '
-d -d -d
' debug logging options (look at/var/log/syslog
or/var/log/auth.log
for the server debug messages - When you get a reproduction with
screen
does it continue to stall when you attempt a disconnect and reconnect: "-D -R
" on the screen?
SeveQ : I have tried starting the ssh server with the triple -d. It logged everything on the console. So here is what I got... first the connection that worked: http://pastebin.com/bWZabYaG and the connection that got lost while logging in (where the login prompt was at the very bottom of the terminal): http://pastebin.com/rA5fziZ6Marco Ramos : +1 for the excellent and detailed answer. I'd also recommend a strace to trace system calls.From nik - Keep a wireshark track to check if the SSH connection actually shows a TCP-FIN sequence when you get a stall (check how this happens in the wireshark view when you exit a normal connection for a reference).
-
Sounds like your connection gets dropped when you start to reach the maximum packet size for the TCP connection. With SSH, normally transfers are of fairly small data and you remain well below MTU.
Is there a firewall inbetween which is dropping all ICMP packets? Is there a router numbered in RFC1918 address space?
As a crude hack, if you run, as root, on the server:
ifconfig eth0 mtu 1420
does that alleviate the problem? If so, Path MTU Discovery is broken (misconfigured firewall somewhere) and you have a link in the way which has a lower MTU than the links local to each end-point. You haven't solved the problem, you've hacked around it by lowering the MTU at the server side.
From Phil P