Sed – An Introduction and Tutorial

[narrator]

I just use | perl -pi -e 's/foo/bar/g' , etc for this kind of stuff. Is there anything I can't do with perl on a line that sed will do? I can see how perl is a lot more complex than sed, but I went through the whole perl learning curve back in the late 90's so it doesn't bother me that much..

[dogweather]

For me, this is the whole point of perl: You can learn one tool which replaces three, each with their own syntax: Sed, Awk, and Grep.

[scdlbx]

It's a lot easier to delete specific lines using sed. Also you can have sed do replacements to the n'th instance of something. Doing that in Perl is a bit more complicated and a lot less succint.

$ echo "foo foo foo foo" | sed 's/foo/bar/3'

foo foo bar foo

[sbi]

Probably not, but in some cases sed and AWK might be faster. I am a big fan of AWK. It is limited in some ways that make it impractical to use for really serious programs, but it is very expressive. Look at [1, 2].

[1]: http://c2.com/doc/expense/

[2]: http://www.pement.org/awk/awk1line.txt

[kerkeslager]

I'm a firm believer that there are many ways to work effectively. If Perl fills the same needs for you that sed does for some people, then by all means stick to Perl.

However, for someone who knows neither, here are some reasons you might want to choose sed over Perl:

1. sed syntax is pervasive in other tools. For example, to run a substitution from early on in the tutorial in vim, type :%s/abc/(&)/<enter> from insert mode.

2. sed is simpler than Perl. It used to be that Perl filled a unique role as a scripting language but now there are a bunch of languages in that space (most notably Python and Ruby). Python + sed for example would fulfill most of the same functions that Perl does (and there are reasons to choose Python over Perl as scripting languages, although that's a much more complicated domain to discuss).

3. sed is more performant (or so I hear). This has never been a real concern for me, but some people cite this is a concern.

4. sed usually is a bit more terse. For the length of expressions you'll typically be writing with sed, this isn't a big concern.

Disclaimer: There are probably good reasons to choose Perl over sed, too. Not being a Perl guy, I don't know those reasons. I'll leave that to someone who knows more about Perl.

When I learned sed it was because I was taking a *nix class in college, and a professor pointed me at sed and not Perl. I learned sed instead of Perl because it was put in front of me, not because of any weighing of pros and cons. That's how a lot of learning happens. Sometimes simply learning what's put in front of you leads turns out to be an obvious mistake in retrospect (I was stuck writing VBA for a little while). I don't know whether learning Perl or sed is better for what sed does, but I do know that after maybe 6ish years using sed quite frequently it hasn't turned out to be an obvious mistake.

[JeremyMorgan]

Sed is one of those tools that once you learn it, you'll start to wonder how you ever got by without it.

This is a great set of tutorials, he also wrote one about Awk: http://www.grymoire.com/Unix/Awk.html

Get to know these two tools and you'll be amazed at the hours you can save and what you can do, especially with text files.

[Fede_V]

I am always amazed by how much people can get done using piped unix commands - but, personally, I find it much easier to just use sed when I need to quickly edit a few files, and if I need to do anything slightly more elaborate, to do it in Python with a script.

It's obviously much slower - but I've never been in a position where I needed insane speed to quickly fix a bunch of files.

[sqrt17]

Maybe it's because Perl was really popular at the time I discovered Unix and its tool, but why would you use sed and awk instead of a Perl one-liner? (Or even :s// in vim or M-x query-replace-regexp in emacs, if it's just regex munging)

[sidcool]

I tried posting this to /r/programming. It said already submitted 8 years ago. It's a very good tutorial and I was oblivious of its existence for so long.

[c3RlcGhlbnI_]

To be honest I don't know how much he has improved on the manual. It is such a small language that you could easily read up to the the examples very quickly even if you aren't particularly interested in learning.

I would suggest just giving it a look directly at https://www.gnu.org/software/sed/manual/sed.html

Though be forewarned, something that neither document explains well is the actual syntax. As in how addresses and expressions can be used and how to read a script. The syntax is relatively simple to understand looking at some examples, but the lack of clear delimiters between the address, command, and command parameters can confuse beginners.

[josteink]

But that only covers the GNU bits.

If you stick (too closely) to that one, sometimes you might use features only found in GNU Sed and think you're writing portable scripts.

I think this tutorial helps clarify what is and isn't portable.

[senorsmile]

probably the best reference besides the o'reilly books.

[sea6ear]

Peter Krumins also has decent a walkthrough of sed that essentially goes through and explains it via detailed explanations of sed one-lines (The explanations are original, but the list of one-liners was already popular on the internet).

Book: http://www.catonmat.net/blog/sed-book/

Free online articles: http://www.catonmat.net/blog/sed-one-liners-explained-part-o...

[mbubb]

For context of 'why sed and why not x?'

This was written in 1984 (I think) and still works with a few syntax adjustments. I think it is not bad discipline to return to these tools from time to time and remember core UNIX principles.

http://web.stanford.edu/class/cs124/kwc-unix-for-poets.pdf

I am not so sure anything that I currently am writing would/ could be relevant in 30 years. Very humbling.

[Jacky800]

I have always used grymoire.com for regex and sed tutorials since probably around 2009. Thanks grymoire

[jakeogh]

Is there a command to automatically escape a string for use in sed?

I got frusturated escaping for simple replacement: https://github.com/jakeogh/replace-text

[klshxsh]

If you find yourself having to escape a lot of slashes, then you can do something like this:

echo "/foo/bar" | sed -e 's|/foo|/tmp|'

(The article mentions this)

[Tobu]

Python has re.escape; awk can take variable definitions from the command line:

  echo b |awk '/b/{print a}' a=4

[nkangoh]

A friend of mine has been trying to get me to learn awk, sed, perl or grep. Honestly I only have the patience for one at the moment, which do you think is the best (taking the ease of learning into account)?

[seanp2k2]

IMO ack is a useful complement to grep if you do development: http://beyondgrep.com

Also, for general bash stuff, these are great, and IMO much better than the TLDP site that always ranks high on Google:

http://mywiki.wooledge.org/BashGuide

http://wiki.bash-hackers.org/doku.php

[jibberia]

Learn grep. If you're familiar with using a shell, it'll be the most immediately useful.

# search current and all child directories for files containing "bananas", case insensitive $ grep -ir "bananas" .

[stygiansonic]

Great reference; with the awk reference on that site, this formed a great resource for learning text manipulation/searching in a Unix environment.

[michaelsbradley]

There's also an excellent (non-free) book:

Definitive Guide to sed

http://www.sed-book.com/

I found it to be well worth the money, though I wish it were available as a PDF.

[la6470]

Awesome

[misiti3780]

was interested in reading this until i saw the yellow background and couldnt stomach it - looks like a lot of great information just displayed in a horrible way

[mdaniel]

If it's available on your platform, you could print it to a local file since the print version doesn't use the same CSS as the web (OS X = print to PDF, Win = print to XPS, etc).

Or, presuming you're on a modern browser and care that much about the content, you can just inspect the dom, find that <link type="text/css"...> in the head, and delete it.

[msoad]

If you're using Chrome, you can install Clearify[1] and read it with better readable CSS

[1] https://chrome.google.com/webstore/detail/clearly/iooicodkii...

[alganet]

I had the same opinion about the official sed page[1], until I noticed the footnote. It got my attention and I decided to learn. One of the best, portable, elegant tools I use every day.

[1] http://sed.sourceforge.net/

[a3n]

In Firefox:

View/Page Style/No Style

It also looks pretty good in the lynx browser, and probably other text browsers. https://en.wikipedia.org/wiki/Lynx_%28web_browser%29

[LukeShu]

It's weird, it didn't always look like that. I'd been recommending it for years while it was a very Spartan HTML page with no CSS or inter-page navigation. The horrid new look came early last year.

[tthayer]

I use the Clearly browser plugin for pages like this. Works like a charm.

[chrisan]

The site is so garish I actually love it for some reason.

[bewuethr]

Use Instapaper?