Fixing IPython Pipe Handling
On my Laptop, I use IPython as the default shell. Lately, I noticed pipe handling to be broken in some specific cases. I could trace this back to Python ignoring the SIGPIPE signal.
SIGPIPE gets sent to a process when it tries to write to a pipe that is not read by another process. The default action for a process under these circumstances is to terminate.
Imagine a process yielding a huge amount of text, piped to head to print the first ten lines. With SIGPIPE ignored, the process will continue after head has stopped reading. Usually, this will only affect runtime – however, if the original process outputs an endless data stream, the shell will hang.
To check whether your shell ignores SIGPIPE, try the following command: yes | tee | head. It should print 10 lines with a single y each, then terminate. If the shell hangs, printing a message similar to tee: standard output: Broken pipe, SIGPIPE is ignored.
TL;DR: In IPython, proper SIGPIPE handling can be restored using the signal module: signal.signal(signal.SIGPIPE, signal.SIG_DFL).
Further reading
- Python tracker: Issue 1652: subprocess should have an option to restore SIGPIPE to default action
- Python SIGPIPE Handling
- A Very Subtle Bug