The Art of Command Line

[scrollaway]

> To disable slow i18n routines and use traditional byte-based sort order, use export LC_ALL=C (in fact, consider putting this in your ~/.bashrc).

Do not. Having a non-utf8 locale means you won't be able to handle utf-8 sanely ("that's why it's faster") and it will break at the most inexplicable times. Any non-latin1 character appearing in your prompt or command line with this will mess its spacing up for example. Do not do not do not.

Hell I even check for it in my .zshrc: https://github.com/jleclanche/dotfiles/blob/master/.zshrc#L3...

Good post otherwise.

[zalzal]

The discussion here has tended to focus on either one or the other of two different facts:

You absolutely need to be aware of and manage language and locale settings diligently. This is a big topic, but generally, whenever possible, use UTF8. And you generally don't want to change your desktop OS or applications' locale or language settings other than that.

However, if you are processing data files and are aware of the implications, and can get away with treating data as binary instead of character streams -- for example, any of the command sequences that use sort/uniq for uniqueness or set union/intersection/difference -- then using C locale is way faster. As in, the difference between doing something in an hour on a big machine and rewriting a whole pipeline in Hadoop. A good fact to know.

It's obvious I (the author) didn't make the simultaneous truth of both these points clear. I'll rewrite.

[binarycrusader]

I have to reiterate this; don't do it -- sort order changes and you'll run into other mysterious issues.

It's not worth the "performance improvement".

[stass]

The question is, do you need to handle UTF-8? In most cases of sever management having LATIN1 would suffice, or appropriate localized 8-byte encoding.

You can apply the same argument to any other obscure encoding scheme, not just UTF-8. The answer is really to pick the encoding which will suite the most usecases and won't be painful to use.

[xenophonf]

I have the following set in my .cshrc:

  setenv LC_ALL en_US.UTF-8
  setenv LC_COLLATE C             # use the ASCII sort order

What have I broken, and how badly?

[thristian]

Some environments support a locale named "C.UTF-8" - with the same familiar sorting and formatting rules as the C locale, but with proper UTF-8 support.

[voltagex_]

Raised https://github.com/jlevy/the-art-of-command-line/issues/22

[cmrx64]

> StrictHostKeyChecking=no

And welcome to MITM haven. This is awful advice if you care about the first S in SSH.

[Karunamon]

Depends on your circumstances, really. If you're in an environment with lots of VM's floating around, being created, destroyed, and so on, dealing with SSH's paranoia (why can't I just dismiss a warning about a host/key mismatch instead of having to edit .known_hosts?) quickly becomes an exercise in frustration for dubious security benefit. (If the enemy is on your LAN and able to manipulate your DNS, you've already lost)

On your home box, sure. But let's not pretend there's not a reason for the option to exist.

[AYBABTME]

What would be good is an interactive confirmation when a host fingerprint has changed, instead of a flatout abortion and having to edit your `known_host` file. I have to SSH into hundreds/thousands of VM that reuse the same IPs/FQDN sometimes, and the strict host checking is a major PITA. I haven't turned it off tho, I prefer the pain to MITM.

[bdamm]

If you learn bash, and you learn vi, then the next most glorious addition is:

set -o vi

Then you have vi keys in your shell. And it is marvelous.

[a3n]

That's nice, but it only affects the command line.

Instead of that, create the following file:

~/.inputrc (tilde slash dot-inputrc)

And in that file put at least this:

set editing-mode vi

Now every program that you run that a) has its own command line, and b) uses the readline library (there are a lot) has vi commandline editing. psql, mysql, telnet, lftp, and many more.

man bash has a section on READLINE. For example, in man bash search for

editing-mode

[baby]

or if you learn emacs you have emacs shortcuts by default in shells.

which goes against his childish comment on emacs:

> Learn Vim (vi). There's really no competition for random Linux editing (even if you use Emacs, a big IDE, or a modern hipster editor most of the time).

[javajosh]

Can you share a little more info about this? I tried it but found it rather confusing.

Also, for more readline maneuvers: http://unix.stackexchange.com/questions/21788/how-to-delete-...

[barbs]

Agreed. I made a pull-request for this.

https://github.com/jlevy/the-art-of-command-line/pull/9

[jonahx]

Learning about this was a revelation for me. I can't live without it now.

[jwcrux]

> Fluency on the command line is a skill that is in some ways archaic...

I don't see this to be the case at all. Plus, having this be the opening line may cause people to equate archaic=I don't need this.

I'd suggest (would pull request but afk) that you remove "is a skill that is in some ways archaic".

[_jal]

Agree. I've been hearing people saying the command line is archaic since Windows 95, and probably before that. Yet somehow I (any everyone I work with, and have worked with over the past 25 years) still works in a shell daily. I don't understand how someone on the technical end of running large internet services could avoid it.

Sure, GUI tools have crept in here and there. And I understand that some toolchains mandate all-singing, all-dancing IDEs. I don't work in those areas (the closest I've come was doing Java work for about five years and I still lived in vi, because Eclipse makes me homicidal). And I still see our iOS developers using `find' and `sed'.

Which isn't to say GUIs can't improve on the command line for some things. For example, the MacOS tool "A Better Finder Rename" (despite its inability to rename itself something less clunky) is a great tool for mass-renaming, e.g., photos. I have written such critters before as shell/perl/python scripts, but pulling EXIF data and renaming files programmatically is ticky and potentially destructive enough that I tend to write it once and then not modify it unless it breaks. The GUI preview and regex capture display makes this much less annoying.

[mbillie1]

https://github.com/jlevy/the-art-of-command-line/pull/12

[agumonkey]

Just throwing this http://mywiki.wooledge.org/BashGuide (it's listed 2 indirections deep so will probably be missed)

Every time you have an issue with the shell, go there.

[pella]

similar for data science: http://datascienceatthecommandline.com/

https://github.com/jeroenjanssens/data-science-at-the-comman...

[MachinaX]

The Linux Command Line (free pdf): http://linuxcommand.org/tlcl.php

[tomaac]

Hosted on Sourceforge which is now blocked almost everywhere.

[hamburglar]

The author should prepare to catch some hell for advocating 'ForwardAgent=yes' in ssh configs.

[xenophonf]

I had meant to take another look at ProxyCommand (http://heipei.github.io/2015/02/26/SSH-Agent-Forwarding-cons...), so thanks for the reminder!

[webnrrd2k]

If you're into the command line, I'd recommend getting a physical copy of Unix Power Tools [1] and spending some time with it. This is a nice article, but Unix Power Tools is better in almost every way for learning the basics (and more) of the Unix command line. This article mentions a few more modern tools, but Unix Power Tools has a far better explanation of what's going on.

[1] http://www.amazon.com/Unix-Power-Tools-Third-Edition/dp/0596...

[gitaarik]

One really useful shortcut I learned in bash is Ctrl-/ which is a sort of undo for bash when composing a command. Can't really explain how it works, you should try it out yourself.

Also really handy is Ctrl-Y, which pastes the last cut characters by Ctrl-W, Ctrl-U or Ctrl-K.

[nanny]

C-/ and C-y are from readline which mimicked them (and many more) from Emacs.

[seri]

Unix tools are very powerful, but they are not as intuitive as I would like. It would be nice if we can have an httpie for process management, one for text manipulation, another for system stats, and so on.

[AdieuToLogic]

Powerful tools are rarely instinctive. Quite often, one person's obvious is another's opaque.

[karmicthreat]

So this is tangentially related by why do people seem to push Vi? Nano is a perfectly serviceable editor. If I need to do extensive editing I always end up loading it into Atom, Sublime or some other editor anyway.

I've never really had to use a console editor for more than 10s of lines.

[girvo]

Nano is fine for 10s of lines, but Vim is great for much much more than that. It's also nice to have the same keybindings for every console app (gosh I love readline), and it's repeat commands make editing text an absolute breeze. Why open another editor on my host if I can edit the file in my console on a remote server?

[mbillie1]

vi keybindings are ubiquitous, so familiarity with the editor is beneficial outside the editor. Just offhand, both ncdu and tig (which I have used in the past hour) respond to vi keystrokes.

It's not as hard as it first seems to learn, either. Well worth the time spent, vi is worlds more powerful than nano.

[taco_emoji]

Nano is not as ubiquitous as vi. Every single *nix box will have vi on it, so even if you prefer something else, it's still in your best interest to learn the basics of vi.

[hobarrera]

> If I need to do extensive editing I always end up loading it into Atom, Sublime or some other editor anyway.

So you're saying "why would you use X, when you could avoid that by using Y"?

Well, because people prefer X, for a variety of reasons. Not to mention vi is installed everywhere, while sublime and atom are not.

[elchief]

nano is super easy to use, I agree. I tried vim and gave up. Will try again.

[AdieuToLogic]

In modern Bourne shell derivatives (Bash, ksh, some POSIX shells), the command:

set -o emacs

Will give command line editing bindings compatible with Emacs (which is also the default editing keys in MS-Windows and many IDE's BTW).

HTH

[kjs3]

"I don't understand why everyone doesn't do things exactly like I do using only tools I like" is a really stupid way to look at the world.

[stinos]

Fluency on the command line is a skill now often neglected or considered archaic

By whom? (honest question: even the most GUI oriented people I know reckon the power of command line skills)

[weavie]

I work in a .Net house. When we had to move to front end development which required using grunt shock from the command line there was plenty of dissent at how we were being forced back into the "dark ages".

[barely_stubbell]

I'm glad the author took time to mention sl - every sysadmin should make sure that its installed on all the boxes they manage.

[craneca0]

The new csysdig curses CLI [1] unifies and replaces a bunch of the system debugging tools listed here.

[1] https://github.com/draios/sysdig/wiki/Csysdig%20Overview

[MarcScott]

I'd never even thought about commenting a line I was half way through writing, when suddenly realising I'd forgotten the correct arguments.

I normally end up opening another terminal and using man there.

[AdieuToLogic]

Much, much more can be found at:

http://www.commandlinefu.com/commands/browse

[hobarrera]

Fun fact: alt+# is impossible on a Spanish keyboard, because alt+3 = #. So you actually need to hold alt to type the character in the first place.

[mediocrejoker]

> To locate a file by name in the current directory, find -iname something . (or similar). To find a file anywhere by name, use locate something (but bear in mind updatedb my not have indexed recently created files).

I think this will glob unless you escape the asterisks. Also on my system (debian 8) I need to put the directory to search first, or not at all:

find . -iname \something\

find -iname \something\

edit: hackernews ate all my asterisks

[AdieuToLogic]

> I think this will glob unless you escape the asterisks.

The majority of times, this is the case. At least one exception is when the shell pattern does not match one or more files in the directory which find(1) is executed. However, this can be "surprising" so it's best practice to either use escapes or ensure interpolation is disabled via single quotes (in Bourne shell syntax).

> Also on my system (debian 8) I need to put the directory to search first...

FWIW, you can specify more than one directory if desired. The primaries will be applied to the results of each.

[hobarrera]

This depends a lot on the terminal, and my advise is to always suround the glob with quotes:

    find . -iname "*file*"

zsh will cry that nothing matches the glob if you don't. bash will expand it if something matches, or pass it as-is to the client if something does - which will give you unexpected results.

[ryenus]

  cat hosts | xargs -I{} ssh root@{} hostname

glad to know this one :-)

  echo y | xargs -Ix echo x

how tricky!

[jnpatel]

I agree, this is one of my favorite tricks. I like to think of xargs as a `map` function over stdin lines.

As the article mentions, especially useful when combined with the -P flag for paralellism.

    cat urlpaths.txt | xargs -I{} -P wget domain.com/{}

Another handy trick, is to use `sh -c` or `bash -c` to include multiple commands with xargs.

    cat dirs.txt | xargs -I{} sh -c 'cd {}; touch file;'