I use vanilla gdb, the secret to the arrow-key problem the author had is they needed to switch window focus. The keybinding is Ctrl-X O, but if you don't want to remember that - and who could blame you - you can use the focus command, e.g. `focus cmd` will get you back to the command window, or `focus src` will get you back to the source window.
The display isn't really buggy, it's just the debugee output messed up the terminal. You can redraw it with ^L, or disable the tui while it's running. You can toggle it with Ctrl-X A, or just `tui enable` or `tui disable`.
I quite like the tui, the windows are configurable (use `show tui` and `show style`), and you can display more than just source, there's also registers and disassembly mode, e.g. `tui reg general`.
GDB is incredibly powerful but it really lacks some good tutorial, and some of its defaults are... questionable.
It's so heavily scriptable and configurable that you could argue that it's almost more of a debugger framework than just a debugger. I would argue that it's almost under-used really, simply because most devs don't realize what it can do besides setting a breakpoint and dumping a backtrace.
> The display isn't really buggy, it's just the debugee output messed up the terminal.
That's clearly a horrible UX bug. The point of a debugger is to debug something which is probably going horribly wrong - having it mess up the debug output is a massive pain.
Clearly GDB could prevent this by intercepting the debuggee and preventing it from overwriting the rest of the screen.
> You can redraw it with ^L, or disable the tui while it's running.
I often run into a problem where switching to TUI after a crash disables local echo for some reason. I can't see my typing and TUI's output is totally busted. CTRL-L doesn't fix this, I have to exit the debugger and run "reset" in my terminal. Does anyone know why this happens or how to fix it without exiting the debugger?
I've even tried suspending with CTRL-Z, running "reset", then resuming gdb. This fixes the shell, but TUI is still busted, even with CTRL-L.
This is a great tip. My workaround to the arrow-keys issue was to learn the (readline?) shortcuts. ie. C-b/C-f for back/forward, C-e/C-a for end of line / start of line, etc.
Its been years since I needed it...
but my favorite was always the DDD front-end (Data Display Debugger). Really nice X based front-end to GDB. The red hat TUI one wasn't bad either.
I recently discovered a Linux debugger & tool which allowed me to solve problems 10x faster than gdb: pernos.co (which is layered over Mozilla's rr time-tracking debugger).
Pernosco's tool is described pretty well on their website (which is full of minimal fluff and marketing -- another good sign). But basically it allows you to view a program inside and out, forwards /and/ backwards, with zero replay lag. Everything from stack traces to variable displays (at any point in time in your code execution) is extremely easy to view and understand. The best part is the lightning fast search functionality (again: zero lag).
On top of this: extraordinary customer service if anything breaks (in my experience, they fix bugs within 24 hours and are highly communicative).
I'm not a paid Pernosco shill. I'm just so blown away by this product that I wanted to word-of-mouth spread it around because it's saved so much time for me.
That looks cool, but $20/month for 5 submissions is absurd. That's practically a product demo you have to pay for. Seems like the type of tool you'd only use on a really tough and rare bug, but I am not running into those consistently enough to justify a monthly payment. GDB's CLI might be ugly/annoying, but it's still usable and perfectly adequate.
Plus it's a cloud service I need to trust with my code? This is a very tough sell.
I might pay an annual fee for a local/self-hosted version though, kind of like a Jetbrains subscription.
I feel like this article gave up way too fast on gdb's tui mode, especially given that gdb works and tui didn't crash like half the debuggers in the list, other than "the command tui disable crashes gdb." which I can't reproduce.
The complaints about rendering and arrow keys are very easy to overlook with just a few tui mode hotkeys. ctrl-x-a toggles tui, ctrl-x-o toggles arrow-key focus between code and the gdb command line, and ctrl-l repaints whenever the rendering gets borked. Those three things cover all the concerns listed here.
The single most important thing about gdb, however, is the .gdbinit file. Being able to script your debug session is hands down the best thing about gdb. Bugs that require several steps to repro can be completely automated in gdb. You have conditionals and looping available in the debug script, this is very powerful. I suspect a bunch of the debuggers listed don't really support such a workflow.
Since I work in a CUDA environment often, it's also nice that cuda-gdb mirrors gdb, I can have scripting & tui mode for both.
> I feel like this article gave up way too fast on gdb's tui mode, especially given that gdb works and tui didn't crash like half the debuggers in the list, other than "the command tui disable crashes gdb." which I can't reproduce.
I've had the same conundrum for a various categories of development tools on Linux:
For a particular category (e.g., remote debugging, Vim C++ autocomplete system, etc.), an initial search uncovers 3-10 software combinations that people have documented as working.
And when I work through the list, trying to find one that works for me, I often hit problem after problem. It's hard to know if (a) that particular recipe is no longer workable, (b) there's something about my particular setup that will forever damn that approach, or (c) I just need to keep working through the problems until I finally get things working.
So it's hard to know how long to bang away at a particular recipe to determine if it's (a), (b), or (c).
The unfortunate result is I often run out of time for the investigation, and fall back to using Vim + gdb + ctags for developing on remote servers.
Mostly I'm just surprised that Visual Studio in 1999 gave me a better debugging experience than I've been able to cobble together on Linux in 2020.
The article should be named "the state of C/C++ debuggers and gdb in particular on Linux". Because there are debuggers for virtually any software ecosystem (for instance Pdb for Python) which frequently work the same cross platform.
Considering gdb/Linux: I love the Kdevelop GUI front-end for gdb. It allows introspection of variables in source code at mouse hovering.
Of course it also can open coredumps. The Fortran gdb is also great for debugging arrays.
If you don't mind OSS, check out https://totalview.io/ -- it is an instance of a domain specific debugger for C/C++ in HPC (massively parallel computing).
Eh, more particularly "the state of Zig debuggers and gdb in particular on NixOS Linux" - the author isn't wrong to want things to work, of course, but the author isn't using the most boring environment either.
And if I go to that website it encourages me to "try it for free" which makes me think I am supposed to pay for it. But ok, let's give it a try. Then I need to enable a bunch of Javascript. Eh, ok. It was recommended on HN, probably not too malicious. And than it complains:
Uh oh... we couldn't load this form.
The following may help you to view this form:
* Are you using an ad blocker? Try whitelisting this page or disabling it
* Browsing in a private or incognito window may prevent this form from loading correctly. Try viewing this page in a normal window.
* If you’re using Firefox Tracking Protection may be enabled. This could prevent this form from loading. Learn how to disable it.
* Try clearing your cache (including cookies).
Ok, close tab. I will keep using other tool for HPC debugging... (Yes, I actually do HPC and I of course have heard of totalview. Last I heard it is also a total pain to install.)
I remember TotalView releasing a product years and years ago called Memscape - Like Netscape, but for Memory. It looked like a browser, but was a sort of graphical heap profiler/leak detector.
I think it wasn't as powerful as Valgrind, but you've got to give them credit for an interesting UX concept.
Yep, it's horrible. I use Vim 8's Termdebug. It just splits Vim in 3 windows - gdb console on top, program output in the middle, source code on the bottom. It does absolutely nothing fancy - you still type commands in the gdb window - but for me just having all 3 windows together is enough and it's super stable. In general any program that tries to do a fancy interface on Linux will be an absolutely miserable experience, there seems to be zero intersection between people who can write solid software and people who can design UI/UX in OSS and for some reason the two camps can't talk together. C'est la vie, I've just given up and use the console.
This is not really a criticism, just an observation. I suspect it stems from the fact that both software design and interface design are very complicated disciplines, but most OSS projects are done by a single person scratching an itch, so you either have programmers designing interface, or designers programming software, and both don't work out well. I suspect if we had some GUI libraries that were as easy as dumping text to console, the programmers would be able to churn out passable interfaces without having to do design. Quite possibly Tcl/Tk is that library, based on how rough, but usable gitk and git gui are.
"The State of Linux Debuggers". This is ... not that. Perhaps "One inexperienced user on an uncustomized system briefly trying several default-configuration front-ends (some of which are apparently malconfigured or missing dependencies) to GDB, without resorting to any documentation whatsoever, and gauging them solely on layout and visual appeal."
Pro-tip. When working with a new debugging environment "I haven't found a way to display the backtrace, list local variables or view memory" would be where most would look at documentation or seek advice, not move on to the next tool. That the author didn't is more of a comment on the reviewer than the product.
I feel this review is overly harsh. I felt the author was showing that many of the packages were not intuitive and were often buggy. He was showing me what I could expect when I might try them. When one expects one's users to RTFM, they've already lost.
Can't agree more. As a longtime user of Eclipse CDT on Ubuntu Desktop， I've not seen the kind of bugs mentioned in the article for more than five years.
After 10 years as a vim user, I switched to emacs in part for the gdb integration. (It turns out you can run emacs with vim keybindings and get the best of both worlds -- vi keybindings but a real programming language for scripting...)
I prefer using dap-mode. It has a much nicer interface that's consistent across multiple debuggers. I use it for C, C++, Rust (Native Debug + gdb) and Python (debugpy).
this has been a total mess for me (cue the xkcd about the spacebar and thermal throttling).
using emacs in terminal mode its pretty frustrating to have the gdb interface bring up windows that you aren't allowed to delete...that and I can I really do more than 2 or 3. so whatever the intent was has been well and truly lost.
this kind of modality is anti-emacs...at least why my hindbrain thinks of as emacs
working on compiler-introduced introspection annotations and writing an elisp front end. we'll see.
What's even more depressing to me is that this is the experience that most students get when they first learn debugging. It's no wonder that so many professional developers don't bother using the debugger when they've had such an awful experience using it.
I'm personally a big fan of vanilla GDB, at least for plain C debugging. Disclaimer: I spent >5years on a PhD work searching how to extend GDB for parallel and embedded programming.
I can't easily see the surrounding code
I have Emacs on the side window, so I use GDB to see the immediately surrounding code, and Emacs to see the rest of the function. I do exactly the same for Python debbuging with PDB
I have to manually request information rather than just glancing at the display
I just print the variables I want to see, there are not so many of them I'm actually interested in while debugging
I have to remember syntax rather than just clicking on things
...and in this case, it took me a few tries to correctly deref this pointer to an fixed-size array
those are the tradeoff between flexibility and ease. You can access any memory locations, and cast it in many different ways, examine the assembly to precisely understand the next steps, examine the bits to precisely understand the storage (only endianness issues remained tricky to follow)
> those are the tradeoff between flexibility and ease
It isn't a required tradeoff. Most of the gdb frontends I tried display the local variables at all time, but also allow writing arbitrary gdb expressions if I want to. Having both is clearly better.
It can't be a coincidence that literally every Linux C++ debugger majorly sucks. I think the blame has to lie ultimately with GDB itself and GDB/MI, because it's the only thing they all have in common (except for lldb I guess).
I've always found bitter irony in that the buggiest applications are the debuggers. Just like the author of this article I've been annoyed at GDB's teletype oriented interface and wanted something better, only to find a lot of applications that are even more clumsy to use and crash more than the application I'm trying to fix.
This goes back a long way too. xxgdb used to be reasonably stable but these days it seems to crash quite frequently. ddd seemed so awesome when I first discovered it, but it too failed spectacularly on the various occasions when I tried actually using it. I occasionally go back to see if they've fixed the bugs but thus far have not had any luck.
I think this approach by the programming profession towards tools that are critical to their occupation is somewhat ass-backwards.
Their idea of professionals depending on critical tools which are created by other professionals in their spare time is wrong. It is time software engineers created guilds that fund the development of tools they need and use. It may be that software development is a new profession which is failing to get properly organized on account of the involvement of big corporations. But that is a throwback to an era when computers were expensive resources only big companies (with lashings of DARPA and NSF funding) could afford and thus were able to influence the tools and the outlook.
Those days are long gone in an era where a powerful computer can be had for less than $200, and there is no need for professionals to allow corporate agendas to undermine the proper development of software tools, as it has been since the internet came around, and it is time software developers learned to see themselves as professionals and stop seeing their jobs as gigs as though they are part time semi-employed musicians or something.
I mean it is 2017 and GCC has been available since the 1990s, but where is the graphical C++ debugger rivaling Visual Studio that we would have by now if developers belonged to a well organized profession which saw the need to fund the development of their software tools through membership dues just like any of the professional bodies out there? By that I mean the software developers themselves, wholly independent of any corporations out there, ie no Linux Foundation type f&ckery where BigCo come in and shape the direction to their agenda.
The software could be Free GNU License software but the hard part would be done by paid professionals funded by guild membership fees.
It's an interesting idea, but like it or not, companies DO set the standards for corporate software development, and they unfortunately don't understand the value of wasted time. A company would rather tell their devs to use free tools like GCC than pay $x,xxx per head to give them something world-class like Visual Studio.
Visual Studio still has the best the debugger. It’s not perfect or even close. But it’s 50x better than cmdline gdb.
RemedyBG is a cool one-man side-project debugger. It’s crazy impressive for what it is. However it has never worked for me on any professional project. There’s too many little one-off issues that are hard blockers. I wish it was open source. https://remedybg.handmade.network/
I've used ddd for debugging statically compiled code, C C++ and Fortran. It is quirky. Its weirder with MPI, as you get lots of debug windows.
For dynamic languages, Perl Python and others, I use komodo from ActiveState. Been a customer of theirs since 2007 or so. Excellent tool, though with a few quirks as well.
Julia has/had a native debugger, and an atom plugin, but seems to be moving towards vscode. Vscode is many things, and is highly opinionated. I've gotten the debugger to "work", albeit not being helpful for the Julia I was working on. I tried the C/Fortran plugins for it and was disappointed.
I wasn't aware that totalview went OSS. I thought it was closed.
[+] [-] taviso|5 years ago|reply
The display isn't really buggy, it's just the debugee output messed up the terminal. You can redraw it with ^L, or disable the tui while it's running. You can toggle it with Ctrl-X A, or just `tui enable` or `tui disable`.
I quite like the tui, the windows are configurable (use `show tui` and `show style`), and you can display more than just source, there's also registers and disassembly mode, e.g. `tui reg general`.
[+] [-] simias|5 years ago|reply
It's so heavily scriptable and configurable that you could argue that it's almost more of a debugger framework than just a debugger. I would argue that it's almost under-used really, simply because most devs don't realize what it can do besides setting a breakpoint and dumping a backtrace.
[+] [-] coherentpony|5 years ago|reply
[+] [-] richardwhiuk|5 years ago|reply
That's clearly a horrible UX bug. The point of a debugger is to debug something which is probably going horribly wrong - having it mess up the debug output is a massive pain.
Clearly GDB could prevent this by intercepting the debuggee and preventing it from overwriting the rest of the screen.
[+] [-] haberman|5 years ago|reply
I often run into a problem where switching to TUI after a crash disables local echo for some reason. I can't see my typing and TUI's output is totally busted. CTRL-L doesn't fix this, I have to exit the debugger and run "reset" in my terminal. Does anyone know why this happens or how to fix it without exiting the debugger?
I've even tried suspending with CTRL-Z, running "reset", then resuming gdb. This fixes the shell, but TUI is still busted, even with CTRL-L.
[+] [-] ghostpepper|5 years ago|reply
[+] [-] tehnicaorg|5 years ago|reply
[+] [-] mech422|5 years ago|reply
[+] [-] jamii|5 years ago|reply
[+] [-] georgewsinger|5 years ago|reply
Pernosco's tool is described pretty well on their website (which is full of minimal fluff and marketing -- another good sign). But basically it allows you to view a program inside and out, forwards /and/ backwards, with zero replay lag. Everything from stack traces to variable displays (at any point in time in your code execution) is extremely easy to view and understand. The best part is the lightning fast search functionality (again: zero lag).
On top of this: extraordinary customer service if anything breaks (in my experience, they fix bugs within 24 hours and are highly communicative).
I'm not a paid Pernosco shill. I'm just so blown away by this product that I wanted to word-of-mouth spread it around because it's saved so much time for me.
[+] [-] bogwog|5 years ago|reply
Plus it's a cloud service I need to trust with my code? This is a very tough sell.
I might pay an annual fee for a local/self-hosted version though, kind of like a Jetbrains subscription.
[+] [-] tom_|5 years ago|reply
[+] [-] dahart|5 years ago|reply
I feel like this article gave up way too fast on gdb's tui mode, especially given that gdb works and tui didn't crash like half the debuggers in the list, other than "the command tui disable crashes gdb." which I can't reproduce.
The complaints about rendering and arrow keys are very easy to overlook with just a few tui mode hotkeys. ctrl-x-a toggles tui, ctrl-x-o toggles arrow-key focus between code and the gdb command line, and ctrl-l repaints whenever the rendering gets borked. Those three things cover all the concerns listed here.
The single most important thing about gdb, however, is the .gdbinit file. Being able to script your debug session is hands down the best thing about gdb. Bugs that require several steps to repro can be completely automated in gdb. You have conditionals and looping available in the debug script, this is very powerful. I suspect a bunch of the debuggers listed don't really support such a workflow.
Since I work in a CUDA environment often, it's also nice that cuda-gdb mirrors gdb, I can have scripting & tui mode for both.
[+] [-] DoofusOfDeath|5 years ago|reply
I've had the same conundrum for a various categories of development tools on Linux:
For a particular category (e.g., remote debugging, Vim C++ autocomplete system, etc.), an initial search uncovers 3-10 software combinations that people have documented as working.
And when I work through the list, trying to find one that works for me, I often hit problem after problem. It's hard to know if (a) that particular recipe is no longer workable, (b) there's something about my particular setup that will forever damn that approach, or (c) I just need to keep working through the problems until I finally get things working.
So it's hard to know how long to bang away at a particular recipe to determine if it's (a), (b), or (c).
The unfortunate result is I often run out of time for the investigation, and fall back to using Vim + gdb + ctags for developing on remote servers.
Mostly I'm just surprised that Visual Studio in 1999 gave me a better debugging experience than I've been able to cobble together on Linux in 2020.
[+] [-] jamii|5 years ago|reply
[+] [-] ktpsns|5 years ago|reply
Considering gdb/Linux: I love the Kdevelop GUI front-end for gdb. It allows introspection of variables in source code at mouse hovering.
Of course it also can open coredumps. The Fortran gdb is also great for debugging arrays.
If you don't mind OSS, check out https://totalview.io/ -- it is an instance of a domain specific debugger for C/C++ in HPC (massively parallel computing).
[+] [-] JoachimSchipper|5 years ago|reply
[+] [-] petschge|5 years ago|reply
The following may help you to view this form:
* Are you using an ad blocker? Try whitelisting this page or disabling it
* Browsing in a private or incognito window may prevent this form from loading correctly. Try viewing this page in a normal window.
* If you’re using Firefox Tracking Protection may be enabled. This could prevent this form from loading. Learn how to disable it.
* Try clearing your cache (including cookies).
Ok, close tab. I will keep using other tool for HPC debugging... (Yes, I actually do HPC and I of course have heard of totalview. Last I heard it is also a total pain to install.)
[+] [-] taviso|5 years ago|reply
I think it wasn't as powerful as Valgrind, but you've got to give them credit for an interesting UX concept.
[+] [-] nemetroid|5 years ago|reply
[+] [-] Asooka|5 years ago|reply
This is not really a criticism, just an observation. I suspect it stems from the fact that both software design and interface design are very complicated disciplines, but most OSS projects are done by a single person scratching an itch, so you either have programmers designing interface, or designers programming software, and both don't work out well. I suspect if we had some GUI libraries that were as easy as dumping text to console, the programmers would be able to churn out passable interfaces without having to do design. Quite possibly Tcl/Tk is that library, based on how rough, but usable gitk and git gui are.
[+] [-] ajarmst|5 years ago|reply
Pro-tip. When working with a new debugging environment "I haven't found a way to display the backtrace, list local variables or view memory" would be where most would look at documentation or seek advice, not move on to the next tool. That the author didn't is more of a comment on the reviewer than the product.
[+] [-] kdtop|5 years ago|reply
[+] [-] howtofly|5 years ago|reply
[+] [-] alexhutcheson|5 years ago|reply
Emacs has a much tighter integration with gdb than with other debuggers. The features and interface are documented here: https://www.gnu.org/software/emacs/manual/html_node/emacs/GD...
[+] [-] evmar|5 years ago|reply
[+] [-] pksadiq|5 years ago|reply
[+] [-] MetaDark|5 years ago|reply
[+] [-] convolvatron|5 years ago|reply
using emacs in terminal mode its pretty frustrating to have the gdb interface bring up windows that you aren't allowed to delete...that and I can I really do more than 2 or 3. so whatever the intent was has been well and truly lost.
this kind of modality is anti-emacs...at least why my hindbrain thinks of as emacs
working on compiler-introduced introspection annotations and writing an elisp front end. we'll see.
[+] [-] thewebcount|5 years ago|reply
[+] [-] unknown|5 years ago|reply
[deleted]
[+] [-] pjmlp|5 years ago|reply
https://www.gnu.org/software/ddd/
https://doc.qt.io/qtcreator/creator-debugging.html
https://docs.kde.org/trunk5/en/extragear-kdevelop/kdevelop/d...
Back in the old XEmacs days, I used the gdb integration, no idea how far Emacs has improved on it.
[+] [-] 2bluesc|5 years ago|reply
https://cgdb.github.io/
Works great for native and cross compiled apps
[+] [-] rightbyte|5 years ago|reply
Edit: Oh 'gdb -tui' ... I should read the docs more ofent.
[+] [-] mkhnews|5 years ago|reply
[+] [-] jiva|5 years ago|reply
https://github.com/cyrus-and/gdb-dashboard
It makes for a very pleasant debugging experience
https://i.imgur.com/iw21sbU.jpg
[+] [-] jamii|5 years ago|reply
[+] [-] ship_it|5 years ago|reply
[+] [-] wazari972|5 years ago|reply
[+] [-] FpUser|5 years ago|reply
This is not the exclusive domain of GDB. I have all those features in various Windows based IDEs all working in friendly manner.
[+] [-] jamii|5 years ago|reply
It isn't a required tradeoff. Most of the gdb frontends I tried display the local variables at all time, but also allow writing arbitrary gdb expressions if I want to. Having both is clearly better.
[+] [-] modeless|5 years ago|reply
[+] [-] jandrese|5 years ago|reply
This goes back a long way too. xxgdb used to be reasonably stable but these days it seems to crash quite frequently. ddd seemed so awesome when I first discovered it, but it too failed spectacularly on the various occasions when I tried actually using it. I occasionally go back to see if they've fixed the bugs but thus far have not had any luck.
[+] [-] vfclists|5 years ago|reply
Here it is for those who don't like reddit
========================
I think this approach by the programming profession towards tools that are critical to their occupation is somewhat ass-backwards.
Their idea of professionals depending on critical tools which are created by other professionals in their spare time is wrong. It is time software engineers created guilds that fund the development of tools they need and use. It may be that software development is a new profession which is failing to get properly organized on account of the involvement of big corporations. But that is a throwback to an era when computers were expensive resources only big companies (with lashings of DARPA and NSF funding) could afford and thus were able to influence the tools and the outlook.
Those days are long gone in an era where a powerful computer can be had for less than $200, and there is no need for professionals to allow corporate agendas to undermine the proper development of software tools, as it has been since the internet came around, and it is time software developers learned to see themselves as professionals and stop seeing their jobs as gigs as though they are part time semi-employed musicians or something.
I mean it is 2017 and GCC has been available since the 1990s, but where is the graphical C++ debugger rivaling Visual Studio that we would have by now if developers belonged to a well organized profession which saw the need to fund the development of their software tools through membership dues just like any of the professional bodies out there? By that I mean the software developers themselves, wholly independent of any corporations out there, ie no Linux Foundation type f&ckery where BigCo come in and shape the direction to their agenda.
The software could be Free GNU License software but the hard part would be done by paid professionals funded by guild membership fees.
[+] [-] phendrenad2|5 years ago|reply
[+] [-] forrestthewoods|5 years ago|reply
RemedyBG is a cool one-man side-project debugger. It’s crazy impressive for what it is. However it has never worked for me on any professional project. There’s too many little one-off issues that are hard blockers. I wish it was open source. https://remedybg.handmade.network/
[+] [-] billfruit|5 years ago|reply
[+] [-] nemetroid|5 years ago|reply
[+] [-] jart|5 years ago|reply
[+] [-] hpcjoe|5 years ago|reply
For dynamic languages, Perl Python and others, I use komodo from ActiveState. Been a customer of theirs since 2007 or so. Excellent tool, though with a few quirks as well.
Julia has/had a native debugger, and an atom plugin, but seems to be moving towards vscode. Vscode is many things, and is highly opinionated. I've gotten the debugger to "work", albeit not being helpful for the Julia I was working on. I tried the C/Fortran plugins for it and was disappointed.
I wasn't aware that totalview went OSS. I thought it was closed.