Recently I found myself having to execute some long running tasks (running
-exec on a directory with millions of files) on a linux server over SSH. This can cause a bit of an issue if you want to go away and do something else or if you have an unreliable internet connection. This is because as soon as you SSH session disconnects the current process ends. This happens wether you close the connection yourself or the frustrating
packet_write_wait: Connection to some.ip port 22: Broken pipe error. There are a couple of solutions to this:
Screen is a program that allows you to have multiple “windows” open without having to create separate SSH sessions for each process. I had always meant to learn how to use this program but there were a lot of options and I always put it off, however like many programs I found there are only a few options you need to learn to get work done.
To get started first SSH into the server we need to run the task on and run
In my case I got this:
-bash: screen: command not found
Not a problem, just install with
sudo yum install screen and select yes when asked to confirm.
Now it’s installed lets try again. You may be presented with a welcome screen but you can just tap space or enter to continue. This will take to to a new empty shell. From here you can run tasks a normal. Lets say you are trying to find all .txt files in your home directory you might use something like
find ~/ -name *.txt and depending on how many files it has to search through this may take a while. Not to worry. Press
d and your screen will be “detached” and you will be returned to the main shell:
[detached from7938.pts-2.newbury] [craig@newbury ~]$
From here you can close your SSH connection and come back in your own time. To “re-attach” simply
screen -r yourSessionId in my case this would be
screen -r from7938.pts-2.newbury If you can’t remember the session ID you can also run
screen -ls which will show you something like:
[craig@newbury ~]$ screen -ls There is a screen on: 7938.pts-2.newbury (Detached) 1 Socket in /var/run/screen/S-craig. [craig@newbury ~]$
screen -r 7938.pts-2.newbury and you will be re-attached back to the session.
Theres also a handy feature to name sessions. From within the session type
This is another program that allows you to run programs without the risk of them being killed on disconnect. In-fact they way I have been using this I haven’t needed to come back, leaving it run on it’s won was fine so
nohup some-command & did the job fine. The
& tells the OS to put the job into the background. If you need to see if its still running then
jobs -l will do the trick for you. You will get output similar to this:
jobs -l + 13100 Running nohup find / &
Where “13100” is the PID and “nohup find / &” is the command you ran. You can still kill the process, say for example services start having latency/responsiveness issues by using
kill 13100 (SIGTERM/Graceful process end) or
kill -9 13100 (SIGKILL/Forcing the process to end).