ssh immediately disconnects "Write failed: broken pipe"
When I try to log in to a remote machine via ssh, I immediately get disconnected with "Write failed: broken pipe". I've tried logging in via mosh instead and the mosh client freezes.
I've tried googling around and the only answers I can come up with are related to timeouts (for example, this Unix stack exchange question). I'm pretty certain it's not a timeout related issue since I made no personal configuration changes since last night (when it was last working) and other users are able to log in just fine.
Running ssh with -vvv (very very verbose logs) revealed nothing helpful.
I already solved my specific issue (see my answer below), but I was wondering what other problems might cause the issue (ssh disconnecting immediately stating only "Write failed: broken pipe")?
I further wonder if sshd/mosh/etc could report on being unable to spawn a session to help diagnose problems?
3 Answers
Another solution is detailed here:
I started with the same problem and during the investigation I had another user try to su as my user. He got the error that lead me to the link above (cannot set user id: Resource temporarily unavailable). I was indeed above the allowed thread limit for my user and increasing that limit allowed me login again.
Basically, run this against your user (this is the current number of threads active for that user):
ps -u testuser -L | wc -lIf the result is bigger than whatever limit is set in /etc/security/limits.d/90-nproc.conf (default is 1024) for that user, than you need to increase it. You will need to do that with another user that has not yet exceeded his limit (or somehow kill some of the threads to be able to login).
To increase it, just add a line similar to the existing one, but specific for your user:
testuser soft nproc 2048I know it's an old question, but this might help people that have this issue.
1So, in my specific case, I had left a screen open running gdb attached to a process trying to reproduce an issue a customer was having related to our product spawning too many threads. Well apparently the issue had been reproduced overnight and my user had reached the system's user thread limit.
So sshd could not spawn a new process under my user due to the process and thread limit. Ultimately what we had to do was to log in with a different user and kill my running screen, gdb, and the process being debugged. This freed up process and thread resources allowing me to finally reconnect.
In case this is useful to others, I had this same issue, and in my case it turned out to be that I had added a user to an sftp users group disallowed for ssh access. I removed the user from the group and voila, no more broken pipe.
1
