Anatomy of a Terminal Emulator

[dundarious]

Casey Muratori's refterm video series is a great example of how to do application design. It focuses on writing a terminal emulator for Windows, but almost all the concepts apply to nixes as well -- even down to the level of OS primitives like memory mappings to create a ring buffer, etc. He builds a nearly complete terminal that appears to better support unicode and escape sequences than windows-terminal, in drastically less code and with literally orders of magnitude better performance.

In particular, I like his focus on experimentation to figure out a reasonable upper bound on performance (so you can measure success/failure), "non-pessimisation" (don't do things extrinsic to the actual task), understanding the actual machine and its capabilities (he doesn't use the term, but sometimes called "mechanical sympathy"), and tactics for isolating "bad code" that you probably have to use (at least to get started).

https://www.youtube.com/watch?v=pgoetgxecw8

[leph]

While I was an economics student, I somehow landed a job to build out an elevator monitoring system at my college. Figured out that terminal emulators could connect to the elevator controller and that the output spec was called Wyse-60. My first "professional" program was written in Python using ncurses to parse the serial output [0].

The program is very cringe but it was my best attempt at the time as I couldn't find any literature on terminal emulators (what actually happened was that I didn't know what to search for)

Anyways, I've successfully pivoted my career away from economics and into software development which was the goal when I took the job in college.

[0]: https://github.com/lee-pham/Pyse-60

[doctor_eval]

I used to use a white phosphor Wyse 60 as my daily driver. It was connected to a DG AViiON running DG/UX - most of the tools were GNU.

Those were the days!

[hawski]

I recommend going through st source: https://git.suckless.org/st/file/st.c.html

I may have fallen out of love to suckless.org, but the code is usually simple so one can at least learn from it.

[dudik]

Why have you fallen out of love to suckless? I recently switched from dwm back to bspwm and I was also looking for a st replacement, but couldn't find one. Which is ok, but were you able to find something "better" in the terminal emulator or window manager space?

[matheusmoreira]

I recommend it as well. It's the simplest terminal emulator I've found, other projects I've explored were a lot more complex.

[timw4mail]

Why do I have to click a link to read the article? I don't understand why it's a thing to add a "fold" to a web article.

More on topic, this does seem like a decent overview.

[imsnif]

Sorry about that! The SVGs are a bit on the heavy side and the various social preview crawlers didn't appreciate it.

[Uberphallus]

Not a design/advertising guy, but often times I think it's to be able to display an ad at the bottom without the user to fully read the article, and without putting it ahead of the article (which is ugly IMO).

Here though I don't think it's the case.

[GekkePrutser]

Yeah a direct link would have been better: https://www.poor.dev/blog/terminal-anatomy/

[michaelsbradley]

For those interested in learning more about terminal emulators and character graphics, checkout Nick Black's book Hacking the Planet (with Notcurses): A Guide to TUIs and Character Graphics[1].

While the overall focus of the book is on programming with Notcurses[2], the author shares a wealth of related info and history throughout its pages.

[1] https://nick-black.com/htp-notcurses.pdf

[2] https://github.com/dankamongmen/notcurses#readme

[doodpants]

One thing I've been wondering for a long time (and my Google-fu is apparently too weak to find on my own): how do certain console applications change the output color without inserting ANSI escape sequences into the output stream? The specific case I have in mind is when writing console programs in C#/.NET, and using the System.Console.ForegroundColor property to vary output colors on the fly. The resulting output text does not have ANSI escape characters in it, yet the colors are displayed properly in the terminal.

[saulr]

On Linux, .NET does write ANSI escape sequences to stdout for you. ConsolePal.Unix.cs (Pal in this context referring to 'platform abstraction layer') does the work for you behind the scenes: https://github.com/dotnet/runtime/blob/10eb1a9ff1c09de4a2a1f...

[ensiferum]

Is this on Windows? If so Windows provides a typical Win32 type of HANDLE API for the console which can be used to change the properties of the console among other things.

[imsnif]

Personally I'm not a C#/.NET developer so wouldn't know where to look for the source code, but to check you can run the program in the examples of the post with the compiled binary on the other side (in place of the SHELL) and see what output you get.

I'm 99% sure it's inserting ANSI escape codes (I'm maintaining a terminal emulator myself, and really that's how everything works), but I could of course be wrong.

[davemp]

Great article. Terminal emulators / shells are an area where we haven't seen much improvement for years. I can't help but think there could be a much nicer command-line-esque interface other than a classic psuedoterminal and shell. At the same time, I doubt any improvements could really be large enough to gain adoption.

[vidarh]

There have been many attempts. Sixel and ReGIS being among the oldest attempt at augmenting terminals with graphics. But most modern apps that run in terminals don't even push the limits of what terminals can do, and that makes it kind-of hard to justify putting effort into adding more capabilities to terminals.

Part of the challenge is to find capabilities to add that are sufficiently simple and compelling to use in command line apps vs. as a web app or native GUI app that'd be more widely accessible than a new terminal capability.

[matheusmoreira]

Can terminals even be improved without breaking everything? Even maintenance work and fixing bugs has historically caused problems:

https://lwn.net/Articles/343828/

[0235005]

I think that for sure something line the Plan9 shell woukd be something cooler to have

[imsnif]

I totally agree. I'm actively working in that direction.

[uuddlrlr]

TempleOS comes to mind.

[0xdky]

Very well written article, thank you.

Especially when my daughter is taking a course on Unix and the instructor is superficially touching in these fundamental topics (due to their own limited understanding), I am going to use this to teach the underlying concepts.

[eatonphil]

Well done guide! Was visually enjoyable to read/watch as well as being well-written.

[chestervonwinch]

I realize naming things is hard, but they confuse me. Are these due to historical quirks, or am I missing something?

E.g., why is it called a terminal emulator and not a terminal user interface? Why is there a "y" in pty if it stands for pseudo terminal? Why is it call it a "pseudo terminal" if it's acting like a communication pipe?

[dredmorbius]

Expanding on what others have written:

- The original terminal was a teletypewriter, printing onto paper, connected to a computer, usually through a serial interface. A teleltype is abbreviated "TTY", and the original Unix terminal interfaces were given the device file names /dev/tty, /dev/tty1, /dev/tty2, ... Those were the hard-defined terminals. Yes, these were communication pipes, handling stdin (keyboard), and stdout and stderr (the printer).

- As psuedo terminals came to be used, for users connected via "glass TTYs" (CRT terminals such as the venerable VT-100 and VT-200), or by entiirely virtualised terminals through remote connections (telnet, rsh) or windowing systems (W and X11), pty came to be used for pesudoterminal. Again, these were communication pipes. I used a variant similar to this VT-320, notable for its amber phosphr display: https://yewtu.be/watch?v=RuZUPpmXfT0

Yes, historical quirks.

You can still hook Unix up to a teletype, by the way:

https://yewtu.be/watch?v=2XLZ4Z8LpEE

[cpuguy83]

Because a terminal (aka tty... or teletype) was a physical piece of hardware that we now emulate in software.

The y in pty is because of tty.

[jdougan]

Pty is a historical quirk: it stands for pseudo-tty, and tty is short for teletype. Actually there are a lot of things in old Unix that were influenced by the primary user interaction device being a slow (110 baud) printing terminal with no lowercase.

[dekhn]

I absolutely love terminals. But, just about the time I could have learned curses, I switched to GUIs. Now I've switched back and uses curses to make UIs (so I can ssh into remote computers with low bandwidth and CPU). It's so funny how terminals are controlled and all the edge cases of each implementation.

[GekkePrutser]

Interesting article. It's too late now for me to focus on it but I'll check it out in the morning! :D Thanks for posting.

[nothrowaways]

The domain name is hilarious