This makes no sense. Ctrl-C is just a way to tell your terminal to send a SIGINT signal to the current process. How that process handles the signal is up to it! It's by definition ignorable, as the author points out, but it's not rocket science to handle it in a sane fashion even in a multi-threaded application. Modern languages make this trivial. The author makes it sound like some dark art but in reality you just have to read the manpages.
SIGINT is really designed for interactive applications. Most processes should simply treat it like a SIGTERM unless they have some sort of REPL. Unless you need graceful shutdown, most processes shouldn't mask either signal. If they do, the polite thing is to unmask after receiving the first signal so subsequent signals immediately terminate.
The SIGINT (if the terminal is configured to generate one) goes to the
foreground process group, not the current process. See the termios man
page, look for INTR. This gets complicated in shell pipelines (oh look,
a process group) where one or more of the tools involved are fiddling
with the global terminal state, in which case there may be a process
group signal, or there might instead be a key for some random program of
the pipeline to to read, which it may ignore.
With an important productivity app like rogue(6) there is (probably)
only one process in the foreground process group, and curses has
(probably) set the terminal to have control+c either ignored or
delivered to the rogue process as a key event. The player probably does
not want to have their rogue process vanish because they hit control+c
by habit, like trek(6) likes to do, but someone wrote blocksig(1) as an
exec wrapper so that SIGINT can be ignored. With a complicated shell pipeline, the
player probably does want the whole thing to go away, but that may be
difficult depending on exactly how complicated the shell pipeline is and
whether any processes in that pipeline are fiddling with the global
terminal state. (Global to the process groups and their PIDs under that
particular terminal, not the whole system or universe or whatever. But
global enough to cause problems.)
Opinions also vary here, some want that LISP image to never go away
(those emacs users, probably), others may want control+c to murder the LISP
image so they can go back to hacking in vi. POSIX probably says
something about shell pipelines and control+c and such, which various
shells may or may not follow, exactly. Etc...
There is an entire world here beyond registering for a signal that comment seems unaware of. Even the simplest of preliminaries: registering for a signal is arguably non-trivial and incorrectly specified in many places since sigaction() supersedes signal().
> it's not rocket science to handle it in a sane fashion even in a multi-threaded application. Modern languages make this trivial. The author makes it sound like some dark art
Which language? I'll specify one so we can begin the process of picking each apart. Python? There is a sibling thread indicating Python issues. I don't know what the actual internal status is with Python signal handling but I am guessing the interpreter actually doesn't handle it correctly if I spent any time digging. Do you mean apps implemented in Python? They will almost certainly not be internally data-consistent. Exposing a signal handling wrapper means very little particularly when they frequently do this by ignoring all of the bad implications. I just checked Python's docs, and not surprisingly, Python guarantees you'll be stuck in tight loops: https://docs.python.org/3/library/signal.html That's just one gotcha of many that they probably aren't treating. This dialogue is going to play out the same way regardless of which language you choose.
Do you mean Postgres? I haven't used it recently but the last comment I read on HN seemed to indicate you needed to kill it in order to stop ongoing queries in at least some situations. If by a stroke of luck it does support SIGINT recovery (which would be great), what about the hundreds of other db applications that have appeared recently? You can't just call the signal handler wrapper and declare victory.
> it's not rocket science to handle it in a sane fashion even in a multi-threaded application
It's not, though you need to be careful if you want to exit cleanly -- you can't just exit() or _exit(). You have to get all the threads to exit, and that requires a global volatile sig_atomic_t flag and some way to signal all threads that might be sleeping in event loops or what have you.
Agreed -- if signal handlers are too messy, `sem_post(3)`, `sigwait(2)`, or `signalfd(2)` will get that control flow where you want it. Then the problem is reduced to "my application needs to handle a graceful shutdown event", which, though possibly complex, isn't really that novel.
I feel this. Signal handling in Python code is especially complicated. I'm not even talking about multithreading here (not like you get anything out of it anyway).
Python registers POSIX signal handlers, that upon SIGTERM/SIGINT, set a flag. Next time the interpreter loop runs, the flag is checked before the next instruction is being run and the stack is unwinded.
When you call out to some C code, that C code may run for a long time. During that time, there is no interpreter loop actually looping. Therefore, all signals are ignored in principle while C code runs.
It's possible for Python code to become uninterruptible while it is calling something like pthread_join.
Definitely - it is a total chore to get a threaded Python program to handle Ctrl-C/SIGINT properly.
Single-threaded Python handles it well - as long as you don't register a custom signal handler, Ctrl-C raises a KeyboardInterrupt exception immediately. KeyboardInterrupt is not a subclass of Exception, (it inherits from SystemExit, which inherits from BaseException directly), so any "except Exception:" clauses don't catch it. Which is the intent. This is also a primary reason to never use bare "except:" clauses (it will prevent Ctrl-C from working!).
For multithreaded Python, the easiest thing to do is just mark all your threads as daemon=True, so they die if the main thread exits. When you can't do that, the best bet is some "threading.Event" and custom SIGINT handler that triggers the event. I kind of wish SIGNINT by default would raise the KeyboardInterrupt in all threads, but I'm sure there are good reasons not to.
> We don't want our ctrl-c to leak memory. […] If you allocate a piece of memory, you need to store a pointer to that memory from the object graph, and both of those operations need to occur inside of a critical section. Otherwise, if you get interrupted right after the allocation, there won't be any way to reach your memory and it will leak.
Maybe I'm missing something here but… so what? If at the end of your Ctrl+C signal handler you exit() as expected, then the OS will clean up your process's memory anyway.
That's the point: Ctrl-C shouldn't just gracefully kill the process, it should interrupt the current computation and let you resume your work without exiting the application. The use case here is interactive applications (think a REPL, for example), not commands you run, simply expect an output from and then they just exit (like, say, curl).
I'm having trouble judging what exactly the author wants here. My best reading is that he wants interactive programs to respond to SIGINT not by bailing out but by terminating the current task and returning to user input.
I'm having trouble, however, thinking of programs to which this applies. I just scrolled through my shell history, and the most common interactive program I've used in my history file is a debugger, which handles killing the active program correctly with no issues, followed by resource monitoring applications, shells, etc.
Can somebody tell me an example command-line application where there's a high degree of interactivity but is also multithreaded, has DB consistency guarantees, network requests in-flight, etc? I'm genuinely having trouble thinking of anything that's not a REPL or vim/emacs.
I'm not sure that the author is exclusively talking about command-line applications. The expected behaviors would make sense inside IDEs, particularly if they are blocked by a modal window.
Excellent advice, thanks for sharing. Would in turn recommend using CopyQ to store this tips (and other like it) as a pinned items in folder with explanations for use two years later, that's how I personally stay on top of terminal kung-fu without overloading the consciousness-in-meat*
Laugh all you want, but this is is precisely why I like "old-fashioned" asynchronous exceptions (the ones which unwind the stack), and ensure most programs are ready to handle a clean stack unwind at practically any point inside the program (e.g. asynchronous-unwind-tables).
The way exceptions are handled as a result of siglongjmp'ing out of a signal handler is currently platform-inconsistent and one of the many dark areas I alluded to. It isn't even consistent on Linux between compilers.
Even though it makes sense from the name, SIGINT, to interrupt, I've rarely seen console software "return control" to the user when the signal is received.
What I've mostly seen in programs is a clean exit from the running application, if live user input is not intended to be used. Clearing a line or something similar like redrawing the terminal (that's mostly Ctrl-L though) is what interactive programs do, let's say shells or ncurses UI programs.
Whenever I made some hobby scripts that exit cleanly when receiving a SIGINT, I've made a global counter of interrupts. When SIGINT is received, the counter is incremented, which tells the main loop to stop as soon as possible. But if this counter exceeds three signals, the application would exit immediatley. This may not be ideal, but CTRL-C CTRL-C CTRL-C is easier than kill -9 `pgrep a.out`.
Like the top comment says, expecting a concrete and general behaviour on different types of software for such a broad signal doesn't gain wide approval.
What "return of control" did the author mean, on what kinds software?
A lot of multi threaded server software handles ctr+c just fine. A lot of Java based server software have a shutdown hook, which is something that you can easily add to any jvm based program because it is part of the standard library. If you use Spring Boot, for example, it has one and it will start shutting down your Spring Context and call any destroy functions on DestroyingBean implementations, which is how you can add your own shut down logic in Spring.
As far as I can tell, this appears to be confusing Ctrl+C (SIGINT, which terminates a process, and is usually not restartable), with Ctrl+Z (SIGTSTP, which pauses a process, and is thus restartable).
The only software I can think of that could "restart" after a Ctrl+C is usually daemons or other long-lived processes (which already need to be able to "restart" after any kind of shutdown and thus have significant amounts of code dedicated to serializing and unserializing their internal state).
TFA even goes so far as to talk about memory leaks - which are completely irrelevant when your process is about to exit anyway!
This article is quite a roller coaster to get what it is about. As far as I understand it, the author wants SIGINT to become some sort of a universal “cancel” button which may or may not exit a process, because the idea is to stop and rollback to a nearest sensible restart point. E.g. an interactive disk formatting tool may stop lenghty formatting on SIGINT but wouldn’t just exit. It would clean up the mess and return to its menu where e.g. batch configuration happens, so a user doesn’t lose next steps. The author basically wants modern gui features in console via signals.
> this appears to be confusing Ctrl+C (SIGINT, which terminates a process, and is usually not restartable),
Respectfully, you seem to be confusing SIGINT, which is an interrupt signal and SIGTERM, which is a terminate signal. Many processes interpret SIGINT in a way which is indistinguishable from SIGTERM, but others do not (e.g. most REPLs).
I use Firebase emulator and man that likes to carry on running (or limping?) after Ctrl-C alot, hogging the port so you need to hunt it down and kill it before you can start it again. Both on Linux and Windows.
I think it is a Java (or more to the point JVM) program, not sure if that has anything to do with it. In addition I believe it is a lot of parallel programs running at once, or they could be different threads. As there are lots of Firebase services it needs to emulate.
Surprisingly, it is possible to do exactly what the author wants. I know because I've done it. However, it is as complicated as the author says it is.
The project in question is my `bc` [1].
Until version 3.0.0 [2], it used a "yield" architecture: every loop it could enter had a check for a signal. This got tedious, so I decided to make the jump to instant-ish reset.
I was lucky in several ways. First, `bc` is a really good program to reset; you just stop it executing, wipe all data away, and ask for more input with a blank slate. Second, it is single-threaded.
Nevertheless, it was still really difficult, especially to have no memory leaks.
First, I had to learn how to use `sigsetjmp()` and `siglongjmp()`. Yep, that was how I was going to do this. Once I learned, I implemented a stack of `sigjmp_buf`'s. Then, when a signal happens, each individual `sigjmp_buf` is used. This allowed me to properly free memory on the way.
In essence, if a function had allocated memory, then it would push a `sigjmp_buf` on the stack, and then when a `siglongjmp()` happened, execution would go to a label where that memory would be freed before continuing the jump series.
Then I implemented signal locks. It is safe to `siglongjmp()` out of signal handler, as long as it didn't interrupt code that was non-async-signal-safe. So I used signal locks for that, and when "unlocking" the lock, it would check for a signal and jump. And if the signal handler sees a lock, it just sets a flag and returns.
Then I had to go through my codebase and protect every bit of non-async-signal-safe code with locks. It was tedious, but the result is fantastic.
Edit: I forgot to add that there is more information at [3] and [4].
Nowadays, I'm working on a threaded build system, and when it gets SIGINT, it sends a message to threads to stop as soon as their children are done. If it receives a second, it just exits.
So yeah, every application is different, but it is possible.
Maybe I’m missing the point, but this (essentially random example) is multithreaded (asynchronous) and gracefully handles ctrl-c. Yes, it’s a high level language that makes it easy, I guess.
"More often than not I find myself having to kill the running process from an external app, such as the shell, after first figuring out what the process ID is."
Short cut here, ctrl-z to background the process, then kill -9 %1 to kill the first job (type jobs for the numbers)
[+] [-] xyzzy_plugh|3 years ago|reply
SIGINT is really designed for interactive applications. Most processes should simply treat it like a SIGTERM unless they have some sort of REPL. Unless you need graceful shutdown, most processes shouldn't mask either signal. If they do, the polite thing is to unmask after receiving the first signal so subsequent signals immediately terminate.
[+] [-] tolciho|3 years ago|reply
With an important productivity app like rogue(6) there is (probably) only one process in the foreground process group, and curses has (probably) set the terminal to have control+c either ignored or delivered to the rogue process as a key event. The player probably does not want to have their rogue process vanish because they hit control+c by habit, like trek(6) likes to do, but someone wrote blocksig(1) as an exec wrapper so that SIGINT can be ignored. With a complicated shell pipeline, the player probably does want the whole thing to go away, but that may be difficult depending on exactly how complicated the shell pipeline is and whether any processes in that pipeline are fiddling with the global terminal state. (Global to the process groups and their PIDs under that particular terminal, not the whole system or universe or whatever. But global enough to cause problems.)
Opinions also vary here, some want that LISP image to never go away (those emacs users, probably), others may want control+c to murder the LISP image so they can go back to hacking in vi. POSIX probably says something about shell pipelines and control+c and such, which various shells may or may not follow, exactly. Etc...
[+] [-] kcl|3 years ago|reply
> it's not rocket science to handle it in a sane fashion even in a multi-threaded application. Modern languages make this trivial. The author makes it sound like some dark art
Which language? I'll specify one so we can begin the process of picking each apart. Python? There is a sibling thread indicating Python issues. I don't know what the actual internal status is with Python signal handling but I am guessing the interpreter actually doesn't handle it correctly if I spent any time digging. Do you mean apps implemented in Python? They will almost certainly not be internally data-consistent. Exposing a signal handling wrapper means very little particularly when they frequently do this by ignoring all of the bad implications. I just checked Python's docs, and not surprisingly, Python guarantees you'll be stuck in tight loops: https://docs.python.org/3/library/signal.html That's just one gotcha of many that they probably aren't treating. This dialogue is going to play out the same way regardless of which language you choose.
Do you mean Postgres? I haven't used it recently but the last comment I read on HN seemed to indicate you needed to kill it in order to stop ongoing queries in at least some situations. If by a stroke of luck it does support SIGINT recovery (which would be great), what about the hundreds of other db applications that have appeared recently? You can't just call the signal handler wrapper and declare victory.
[+] [-] cryptonector|3 years ago|reply
It's not, though you need to be careful if you want to exit cleanly -- you can't just exit() or _exit(). You have to get all the threads to exit, and that requires a global volatile sig_atomic_t flag and some way to signal all threads that might be sleeping in event loops or what have you.
[+] [-] klez|3 years ago|reply
Which are the applications the article is talking about anyway.
[+] [-] colanderman|3 years ago|reply
[+] [-] intelVISA|3 years ago|reply
[+] [-] vonwoodson|3 years ago|reply
[+] [-] untitaker_|3 years ago|reply
Python registers POSIX signal handlers, that upon SIGTERM/SIGINT, set a flag. Next time the interpreter loop runs, the flag is checked before the next instruction is being run and the stack is unwinded.
When you call out to some C code, that C code may run for a long time. During that time, there is no interpreter loop actually looping. Therefore, all signals are ignored in principle while C code runs.
It's possible for Python code to become uninterruptible while it is calling something like pthread_join.
See https://stackoverflow.com/questions/39930722/how-do-i-catch-...
Then of course, you have that on top of all the other problems mentioned by the blogpost.
[+] [-] rzimmerman|3 years ago|reply
Single-threaded Python handles it well - as long as you don't register a custom signal handler, Ctrl-C raises a KeyboardInterrupt exception immediately. KeyboardInterrupt is not a subclass of Exception, (it inherits from SystemExit, which inherits from BaseException directly), so any "except Exception:" clauses don't catch it. Which is the intent. This is also a primary reason to never use bare "except:" clauses (it will prevent Ctrl-C from working!).
For multithreaded Python, the easiest thing to do is just mark all your threads as daemon=True, so they die if the main thread exits. When you can't do that, the best bet is some "threading.Event" and custom SIGINT handler that triggers the event. I kind of wish SIGNINT by default would raise the KeyboardInterrupt in all threads, but I'm sure there are good reasons not to.
[+] [-] actionfromafar|3 years ago|reply
[+] [-] codethief|3 years ago|reply
Maybe I'm missing something here but… so what? If at the end of your Ctrl+C signal handler you exit() as expected, then the OS will clean up your process's memory anyway.
[+] [-] klez|3 years ago|reply
[+] [-] ynik|3 years ago|reply
exit() is not signal-safe; signal handlers are expected to call quick_exit() or _Exit() instead.
[+] [-] e63f67dd-065b|3 years ago|reply
I'm having trouble, however, thinking of programs to which this applies. I just scrolled through my shell history, and the most common interactive program I've used in my history file is a debugger, which handles killing the active program correctly with no issues, followed by resource monitoring applications, shells, etc.
Can somebody tell me an example command-line application where there's a high degree of interactivity but is also multithreaded, has DB consistency guarantees, network requests in-flight, etc? I'm genuinely having trouble thinking of anything that's not a REPL or vim/emacs.
[+] [-] mike-the-mikado|3 years ago|reply
He would help his case by giving specific examples of problematic programs.
[+] [-] drdec|3 years ago|reply
[+] [-] dmarinus|3 years ago|reply
[+] [-] klez|3 years ago|reply
[+] [-] dingdingdang|3 years ago|reply
* https://www.mit.edu/people/dpolicar/writing/prose/text/think...
[+] [-] awild|3 years ago|reply
[+] [-] fmajid|3 years ago|reply
[+] [-] AshamedCaptain|3 years ago|reply
[+] [-] kcl|3 years ago|reply
[+] [-] teddyh|3 years ago|reply
Proper handling of SIGINT/SIGQUIT: https://www.cons.org/cracauer/sigint.html
[+] [-] jesprenj|3 years ago|reply
What I've mostly seen in programs is a clean exit from the running application, if live user input is not intended to be used. Clearing a line or something similar like redrawing the terminal (that's mostly Ctrl-L though) is what interactive programs do, let's say shells or ncurses UI programs.
Whenever I made some hobby scripts that exit cleanly when receiving a SIGINT, I've made a global counter of interrupts. When SIGINT is received, the counter is incremented, which tells the main loop to stop as soon as possible. But if this counter exceeds three signals, the application would exit immediatley. This may not be ideal, but CTRL-C CTRL-C CTRL-C is easier than kill -9 `pgrep a.out`.
Like the top comment says, expecting a concrete and general behaviour on different types of software for such a broad signal doesn't gain wide approval.
What "return of control" did the author mean, on what kinds software?
[+] [-] emmelaich|3 years ago|reply
[+] [-] jillesvangurp|3 years ago|reply
Good explanation here of shutdown hooks: https://www.baeldung.com/jvm-shutdown-hooks
[+] [-] M9HF8wwiaAdZKEZ|3 years ago|reply
The only software I can think of that could "restart" after a Ctrl+C is usually daemons or other long-lived processes (which already need to be able to "restart" after any kind of shutdown and thus have significant amounts of code dedicated to serializing and unserializing their internal state).
TFA even goes so far as to talk about memory leaks - which are completely irrelevant when your process is about to exit anyway!
[+] [-] wruza|3 years ago|reply
[+] [-] krallja|3 years ago|reply
Open a Ruby interpreter (`irb`). Type `i=0; loop { i += 1 }`. Press Ctrl+C.
* Irb is still running.
* Your infinite loop has been stopped.
Type `i`:
* The REPL state preserved as much progress as it could when you aborted the run.
Now do the same thing in `sh`. Now `python`. Now `psql`. All handle Ctrl+C in the way the article mentioned!
[+] [-] drdec|3 years ago|reply
Respectfully, you seem to be confusing SIGINT, which is an interrupt signal and SIGTERM, which is a terminate signal. Many processes interpret SIGINT in a way which is indistinguishable from SIGTERM, but others do not (e.g. most REPLs).
[+] [-] pdw|3 years ago|reply
[+] [-] quickthrower2|3 years ago|reply
I think it is a Java (or more to the point JVM) program, not sure if that has anything to do with it. In addition I believe it is a lot of parallel programs running at once, or they could be different threads. As there are lots of Firebase services it needs to emulate.
[+] [-] nrabulinski|3 years ago|reply
[+] [-] hprotagonist|3 years ago|reply
it shows the current line number in nano.
etc.
[+] [-] vladvasiliu|3 years ago|reply
[+] [-] krallja|3 years ago|reply
[+] [-] ghoward|3 years ago|reply
The project in question is my `bc` [1].
Until version 3.0.0 [2], it used a "yield" architecture: every loop it could enter had a check for a signal. This got tedious, so I decided to make the jump to instant-ish reset.
I was lucky in several ways. First, `bc` is a really good program to reset; you just stop it executing, wipe all data away, and ask for more input with a blank slate. Second, it is single-threaded.
Nevertheless, it was still really difficult, especially to have no memory leaks.
First, I had to learn how to use `sigsetjmp()` and `siglongjmp()`. Yep, that was how I was going to do this. Once I learned, I implemented a stack of `sigjmp_buf`'s. Then, when a signal happens, each individual `sigjmp_buf` is used. This allowed me to properly free memory on the way.
In essence, if a function had allocated memory, then it would push a `sigjmp_buf` on the stack, and then when a `siglongjmp()` happened, execution would go to a label where that memory would be freed before continuing the jump series.
Then I implemented signal locks. It is safe to `siglongjmp()` out of signal handler, as long as it didn't interrupt code that was non-async-signal-safe. So I used signal locks for that, and when "unlocking" the lock, it would check for a signal and jump. And if the signal handler sees a lock, it just sets a flag and returns.
Then I had to go through my codebase and protect every bit of non-async-signal-safe code with locks. It was tedious, but the result is fantastic.
Edit: I forgot to add that there is more information at [3] and [4].
Nowadays, I'm working on a threaded build system, and when it gets SIGINT, it sends a message to threads to stop as soon as their children are done. If it receives a second, it just exits.
So yeah, every application is different, but it is possible.
[1]: https://git.yzena.com/gavin/bc
[2]: https://git.yzena.com/gavin/bc/src/branch/master/NEWS.md#3-0...
[3]: https://git.yzena.com/gavin/bc/src/branch/master/manuals/dev...
[4]: https://git.yzena.com/gavin/bc/src/branch/master/manuals/dev...
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] mlhpdx|3 years ago|reply
https://github.com/mlhpdx/SimplestLoadBalancer
[+] [-] pixelbeat__|3 years ago|reply
I was calling this "responsive idempotence" when discussing how the GNU coreutils are tested:
https://www.pixelbeat.org/docs/coreutils-testing.html
[+] [-] rmetzler|3 years ago|reply
[+] [-] CoffeeCollector|3 years ago|reply
[+] [-] klez|3 years ago|reply
[+] [-] aumerle|3 years ago|reply
[+] [-] g5095|3 years ago|reply
Short cut here, ctrl-z to background the process, then kill -9 %1 to kill the first job (type jobs for the numbers)