Ask HN: What are your best shell scripts that you use?

[JadoJodo]

I saw a note[0] about using a script to activate Redshift in Linux and wondered what other cool, useful, or otherwise interesting scripts you might be using.

[0] https://news.ycombinator.com/item?id=21916949

[jdblair]

I've been using my "dots" script for over 10 years to give me feedback that a long task (like un-tarring a big source tree) is still running, without flooding the terminal with text. I pipe the "verbose" output of a command into the script: tar xvfz kernel.tar.gz | dots

#!/usr/bin/perl

$| = 1; $i = 0;

if ($#ARGV < 0) { $number = 1000; } else { $number = shift(@ARGV); }

while (<STDIN>) { print '.' unless ($i % $number); $i++; }

print "\n";

[wruza]

See also pv.

https://linux.die.net/man/1/pv

[captn3m0]

I use snippy as my snippet manager. Keep them version controlled in ~/.snippy and activate it with dmenu: https://gist.github.com/coderofsalvation/46549e3788ade2f3a93...

I have another one that splits AAX files from Audible by chapters: https://github.com/captn3m0/Scripts/blob/master/split-audio-... (After stripping the DRM)

ansi2html converts terminal output to HTML: https://github.com/captn3m0/Scripts/blob/master/ansi2html.sh

dydns.sh sets up a dynamic DNS entry on CloudFlare against my current local IP address. https://github.com/captn3m0/Scripts/blob/master/dydns.sh

emojify converts emoji shortcodes to emojies on the command line: https://github.com/captn3m0/Scripts/blob/master/emojify

vtt2srt: convert vtt subtitle files to SRT https://github.com/captn3m0/Scripts/blob/master/vtt2srt

[dmoreno]

Mine watches for changes in a directory and runs a command on any save change. I use it a lot for Test Driven Development. Modify source code, save and automatically on a terminal it runs `make test` or similar.

https://gist.github.com/davidmoreno/c049e922e41aaa94e18955b9...

[0xdeadb00f]

Have you heard of wendy[1]?

Might be useful, I'm not entirely sure if it suits your case because my experience with inotify is limited to playing around with it and nothing substantial coming from it.

1: http://z3bra.org/wendy/ (Author's blog post: http://blog.z3bra.org/2015/03/under-wendys-dress.html)

[nl5887]

I'm using a script that will request an AWS spot instance for 6 hours, reporting the address when fulfilled:

https://gist.github.com/nl5887/56912b70b782baa4bd580ae22bde6...

[hoffs]

What do you typically use these instances for?

[blackrock]

I use this in Ubuntu to take screenshots. Ubuntu comes built in with gnome-screenshot, but the file saving mechanism is lacking.

So, this will automatically file away your screenshots to your $HOME/Documents/screenshot/ folder, organized by year/year_month/file.png.

Where file.png is in the format yyyy_mmdd_hhmmss.png.

I use it to take an area screenshot of all my research notes, useful comments, gold nuggets, etc. The automatic folder organization files it away nicely, and keeps it organized as the years go by.

Create it and set the execute bit:

  sudo vi /usr/bin/area_screenshot
  chmod ugo+x /usr/bin/area_screenshot

Then copy the contents below:

  #!/bin/bash
  
  screenshot_dir="$HOME/Documents/screenshot"
  current_year_dir="$screenshot_dir/$(date +%Y)"
  current_month_dir="$current_year_dir/$(date +%Y_%m)"
  fileout="$current_month_dir/$(date +%Y_%m%d_%H%M%S).png"
  
  # Step 1: Check for screenshot directory 
  [ -d "$screenshot_dir" ] || mkdir "$screenshot_dir"
  
  # Step 2: Check year and month directory
  [ -d "$current_year_dir" ] || mkdir "$current_year_dir"
  [ -d "$current_month_dir" ] || mkdir "$current_month_dir"
  
  # Step 3: Take area screenshot to the current month 
  [ -d "$current_month_dir" ] && /usr/bin/gnome-screenshot -a -f "$fileout" $@

Then map it to the printscreen key.

[Daviey]

Uhm, by default the 'Print Screen' key in Ubuntu Gnome saves the screenshot in ~/Pictures/Screenshot from $(date "+%Y-%m-%d %H-%M-%S").png

[emilburzo]

Maybe helpful: scrot

https://en.m.wikipedia.org/wiki/Scrot

[lcall]

Basically, I write a script (or use an alias) for everything that I do repeatedly which would otherwise require typing more than 2-3 unique characters before using tab completion. The habit just makes life easier (and is fun and relaxing sometimes). I try to use names that are quick to type (like, not the same finger repeatedly) and which are memorable, or grouping things by first characters (if there is a theme).

(This goes with the idea, which I also try to encourage, that any repeated process should be first documented in some rough form at least (like a personal note-base or team wiki), then improved over time, via improving the doc, scripting it, and moving toward full automation based on balancing considerations of cost/benefit over time, YAGNI, and avoiding debt, and ideas from the "checklist manifesto", such as the realization that even smart people can forget important things, drop the ball sometimes, or leave.)

Edit: This also lets me script away differences between platforms, so I can just remember my usual little 1-3 letter command and it takes care of the rest, while the script records the details for reference.

[lcall]

To clarify this slightly (I guess I have an urge to make a speech): it has really seemed to me, that, for the benefit of an organization (medium- and long-term), and the peacefulness & less pain for everyone involved: if anything is going to be repeated much, it seems worth making the process reliable and improving/automating it, at least a little bit each time it is performed (again, documenting, and indicating openness to peer feedback, on the path toward automating).

This, to me, seems like part of the Golden Rule of working together (treat others the way one would want to be treated), as are things like not leaving useful but hard-to-maintain messes behind for others without some clarification, or having important processes that only one person knows how to do, so when they leave it is a crisis (or no one really knows how to do it so it is a crisis every time).</speech>

Feedback welcome. :)

[iudqnolq]

I've been holding off making scripts for a lot of cases so that I memorize the commands. Do you think that's pointless? I'm a college student now and I've been thinking that I'd be better off not customizing my shell too much so that I can use other computers I end up in front of. (I have nothing against customization in general, as I think my emacs would show)

[brimstedt]

The largest reason I don't do this is because I ssh into a lot of different servers and then won't have my unique aliases or scripts readily available.

Is there an easy way other than copy paste to rectify this?

[jmstfv]

When building a new website, I default to handwriting the HTML, instead of going with a static site generator. More pages lead to more duplication. I wrote a script that recursively goes through the directory, and replaces all occurrences of a string:

    find . -name '*.html' -print0 | xargs -0 sed -i "" "s/replaceme/withthis/g"

The same but with fd:

    fd -e html --print0 | xargs -0 sed -i "" "s/replaceme/withthis/g"

I have a tiny Sinatra app that I use in testing some parts of my Rails app. I either start it in the background or in a separate terminal window (that I close after some time). I have a script that kills a process given its port number (4567):

    kill -9 $(lsof -ti tcp:4567)

[shakna]

I've been writing a lot recently.

There's a bit of scaffolding around things going into a docs folder, some JS and CSS tweaks for the various output formats (ePub, HTML & PDF), but there is a very simple script I use to reduce some friction whilst writing.

It's nice to have a lot of small files. However, you may often want to:

1. Rearrange those files

2. Insert a new section between two other files

So, I hacked together this incredibly tiny script that means anytime I want to add anything, I can immediately get to wherever I'm up to:

    next="$(ag TODO docs | awk -F':' '{print $1}' | uniq | sort | head -n1)"

    if [ -z "$next" ]
      n="$(basename "$(find docs -type f | sort | tail -n1)" .md)"
      n2="$(echo "$n + 10" | bc -l)"
      next=docs/"$(printf "%05d" "$n2")".md
    end

    echo "$next"
    nano -s 'aspell -c' "$next"

Bonus: It outputs the current filename to the console, so that if I want to add stuff after where I'm currently at, I have the starting number.

The entire scaffold for my books is three scripts, some standard CSS, JS and YAML. Which makes setting up a new one to match my sensibilities, quick and simple.

[iudqnolq]

A hacky python wrapper around bluetoothctl that lets me use aliases as addresses. So I can write `connect {niss}` to connect my phone instead of `connect 12:6a:78:c2:74:d5 (ref. to one of childhood favorite books, Startide Rising).

The script then runs "devices", looks for an alias "niss", and substitutes in the corresponding address. I use expect in Python to script it all together.

[iudqnolq]

Got back to my laptop, here it is (extremely hacky): https://gist.github.com/danielzfranklin/bb4848ffda1d44b437e2...

All commands you enter pass directly to bluetoothctl except aliases in curly braces are replaced. You can use it interactively or pipe to it.

Complex piped stuff may or may not actually work. If stdin isn't a tty the program exits once bluetoothctl reports a success/failure after it gets an EOF on stdin. This means you can write `btctl <<< "connect {my_device_alias}"` and it will exit once it's done connecting or couldn't connect.

If you're using it interactively it only waits 0.01 seconds for results before displaying them and moving on to the next input() (whereas bluetoothctl will asynchronously display more results even while you're typing in a new input) so you may need to spam enter to see results.

[ivanmaeder]

I wrote this to make renaming multiple files quicker/easier:

https://github.com/ivanmaeder/vimv

Basically it loads up the output of `ls` into an editor, then runs a `mv` command for each line.

Not something I use daily, but still handy.

[gaving]

Pretty cool with the editor interaction.

Also worth bearing in mind most shells can do this sort of thing easily enough too.

The example from your video with zsh's zmv (bundled: https://github.com/zsh-users/zsh/blob/master/Functions/Misc/... )

  > touch btn_google_{signin,signup}_dark_normal_web{,@2x}.png # create files
  > ls -l
  .rw-r--r-- 0 gaving 31 Dec 10:34 btn_google_signin_dark_normal_web.png
  .rw-r--r-- 0 gaving 31 Dec 10:34 [email protected]
  .rw-r--r-- 0 gaving 31 Dec 10:34 btn_google_signup_dark_normal_web.png
  .rw-r--r-- 0 gaving 31 Dec 10:34 [email protected]

  > zmv -n -W btn_*_sign*_dark_normal_web* *_sign_** # dry run
  mv -- btn_google_signin_dark_normal_web.png google_sign_in.png
  mv -- [email protected] [email protected]
  mv -- btn_google_signup_dark_normal_web.png google_sign_up.png
  mv -- [email protected] [email protected]

  > zmv -W btn_*_sign*_dark_normal_web* *_sign_** # change files
  > ls -l
  .rw-r--r-- 0 gaving 31 Dec 10:34 google_sign_in.png
  .rw-r--r-- 0 gaving 31 Dec 10:34 [email protected]
  .rw-r--r-- 0 gaving 31 Dec 10:34 google_sign_up.png
  .rw-r--r-- 0 gaving 31 Dec 10:34 [email protected]

[pheug]

There's vidir in debian's moreutils package which does pretty much exactly that.

Check all also all the other tools from that package: https://manpages.debian.org/buster/moreutils/index.html

The package's tagline is "collection of the Unix tools that nobody thought to write long ago, when Unix was young", so pretty relevant to the whole topic.

[globular-toast]

Emacs has something built in just like that called dired (directory edit). Shows you ls-like output, you edit it like a text file then save it. It means you can use all your familiar text editing tools to do what you need to do.

[zzo38computer]

Here is one:

  #!/bin/bash --
  f0() {
    echo 'select moz_bookmarks.title || '"'"' = '"'"' || url from moz_places, moz_bookmarks on moz_places.id = moz_bookmarks.fk where parent = 2;' | sqlite3 /home/user/.mozilla/firefox/twht79zd.default/places.sqlite
  }
  f1() {
    firefox `echo 'select url from moz_places, moz_bookmarks on moz_places.id = moz_bookmarks.fk where moz_bookmarks.title = '"'$1'"';' | sqlite3 /home/user/.mozilla/firefox/twht79zd.default/places.sqlite`
  }
  f$# $1

Another one:

  #!/bin/bash --
  curl 'http://icanhazip.com/'

[stevewodil]

I wrote a script the other day that starts my dev environment. It opens terminal and executes the commands to start the backend and CDN services, then opens another tab for the git directory. After that it opens the frontend and backend codebases in my IDE. I then added the script to my PATH so I can launch it with a command from any directory.

This is for a side project and being able to launch the dev environment so quickly has allowed me to start working more easily as opposed to going on YouTube or Reddit instead. Your brain will crave the easiest source of dopamine so anything you can do to make the habits you want to build (like working on a side project) easier will help you immensely!

[zaphodias]

it's probably worth checking tmux + tmuxinator, definitely pays off in the long run :)

I'm using it for a few months now and I'm in love with it

[robgibbons]

Back when I still used Xubuntu, I wrote little script called Highlander that manages the launching and switching between programs in my launcher tray. It gives your app launcher icons the same functionality as OSX dock icons: launch an instance of a program, then for subsequent clicks, bring that instance to focus. That way you don't get redundant copies of a given program running (and thus the joke of calling it Highlander.)

https://github.com/robgibbons/highlander/

These days I use Ubuntu MATE which has a Mac-like dock with the same functionality for free.

[vkaku]

Here's a few useful ones:

- SSH wrappers/eval-ed Aliases, which do: exec ssh -l user host "$@"

- AWS wrappers, that allow me to: exec aws --profile blah "$@"

- DB wrappers, that allow me to hit a target DB

- cred: Allows me to pick creds from a password manager / keychain independent of OS; Usually a polyfill.

- subtract, intersect, union, sample: for handling one liner data, incl. CSV files

- csvcut: Python wrapper though that does -f, -d, -c but on CSV files

- j2y, y2j: Python wrappers for JSON to YAML, YAML to JSON

- envsubst, watch: polyfills for some environments

- vdimount, imgmount: Allow loopback mounting for partitions / images for VirtualBox/Qemu

- nanny: Nanny for starting processes, passing listen sockets on restart, change NS properties; Usually a polyfill

[Carpetsmoker]

I got various scripts here: https://github.com/arp242/dotfiles/tree/master/local

[LinuxBender]

There are many useful commands on CommandLineFu [1] for what it's worth.

[1] - https://www.commandlinefu.com/commands/browse

[duelingjello]

Automatically re-ssh if a host key changed and run monkeysphere s (ssh pk’s stored in gpg keychain) if not already loaded in the gpg agent (which also supports ssh-agent functions).

lw which does ls -l “$(which -a “$1”)”

ew which does ”$EDITOR” “$(which “$1”)”

newsh which does touch ~/bin/“$1” && chmod +x ~/bin/“$1” && “$EDITOR” ~/bin/“$1”

And habitat (hab) and arch pkgbuild, which use shell scripts as their package DSL... the former I’ve hacked up to screen-scrape package versions (due to the dearth of RDF/metalink usage in release artifact publication) and check gpg keys.

[miccah]

I have one similar to your "newsh" called "new" which does `mkdir "$1" && cd "$1"`

I also have another one called "store" which does `mkdir "${@: -1}" && mv $@`. I basically wanted an easy way to move everything into a new directory ("store * backup").

[sirtoffski]

I’ve got a VPN server on a machine which gets a public IPv4 address via DHCP. Once in a while it changes. Instead of using something like DDNS, I just have a script running via chron to get my public address and send it to a GitHub gist.

If VPN stops working, I can always check to see if the address changed. May not be the best solution, but it works for me.

Script itself: https://sirtoffski.github.io/docs/my-pub-ip

[thrwaway69]

curious, what purpose is vpn for assuming you host it for yourself only?

[gesman]

...or ZeroTier

[clauderoux]

In my lab, we use more and more a programming language such as tamgu, which has been implemented to benefit from pipes with an integrated editor. (https://github.com/naver/tamgu/blob/master/docs/English/tamg...)