Ask HN: Should I learn Awk or Perl? Or is that a bad question?

[AnthOlei]

I find myself being totally lost on the command line when I need to do more complex stuff - a series of pipes between greps and cuts and the works. Recently, I finally took some time to try to learn some awk and it was life changing. I’ve been doing some awk puzzles, but it occurred to me that if I’ll be deliberately spending time learning a command line text processing tool, I may as well go the extra mile and learn perl, which I’ve heard is far more expressive than gawk.

I’m also noticing a lack of literature on Perl on the command line, I.e “perl -nae <some script>”.

Is awk just better for this use case?

[_xk9i]

The case for Perl is just that it has no limits. You will never run out of tool if you go with Perl. That is what, I believe, makes the investment worthwhile.

Why Perl gets such a bad rap, then? I'll tell you.

I've used Perl for almost 30 years now, and I've never found any limits in the tool. This means every limit I found was _mine_. Many programmers feel bad when they find their own limits, but it doesn't have to be the case.

You should be OK with finding your own limits, and be willing to overcome them, and strive to become a Perl power user, just like the founder, Larry Wall.

If you choose this, to grow and improve and be better, and better, and better, then Perl is the best option.

It's up to you.

[muzani]

This is good insight. Limits are often built into programming languages. Java is all about the limits. TS's selling point is literally the limits. As well as every opinionated language.

I wonder if AI will make limitless languages easier to work with and maintain.

[wruza]

My reason to break up with Perl was the lack of garbage collection and then later the lack of types and then later 6.

I don’t think investing in it is wise when you can invest in something more relevant. Although there’s barely anything that covers the subj as much as Perl, which is a shame on us all. A programming language that requires an import or two to dig through a file is degenerate. Old languages understood well what a programmer needs.

[fargle]

AWK is better suited to command line ad-hoc and shell script use. it is simple, straight-forward, and surprisingly powerful. most of the time sh+awk can replace sh+sed+cut+grep constructs. GNU awk (gawk) flavor is ubiquitous and it has a great manual [1]. and you have the AWK book [2] (not "free", but easily found).

perl is more of a completely replacement for sh+awk for a standalone script/program. it was the first scripting language i know of that could do sockets networking and other complex things you'd previously need to write in c. it has extremely powerful text processing capabilities, like awk. but it's a big complicated, unusual scripting language. if you want a big complex full-featured scripting language to make a complete tool these days, why not use Python? if you only need really fancy text manipulation, maybe Perl, but i don't see that use a lot.

[1] https://www.gnu.org/software/gawk/manual/html_node/index.htm... [2] https://awk.dev/

[nextos]

IMHO, Ruby is a much better Perl replacement than Python.

Ruby is heavily inspired by Perl and lets you do a lot of work with simple one liners embedded in Bash scripts.

It's also great for longer standalone programs and there is a larger community.

Lots of interesting Perl-isms are also doable in Ruby.

[asicsp]

>noticing a lack of literature on Perl on the command line

I have written a book on Perl one-liners with plenty of examples and exercises [0]. I've also written books on CLI text processing tools like grep, sed, awk and coreutils [1]. If you prefer just solving exercises, check out my interactive TUI apps [2].

>Is awk just better for this use case?

It depends on the kind of tasks you'd need to solve. I generally prefer grep, sed and awk first. Perl helps if I need powerful regexp and other niceties like the huge number of built-in functions and access to third-party libraries.

[0] https://learnbyexample.github.io/learn_perl_oneliners/

[1] https://github.com/learnbyexample/scripting_course#ebooks

[2] https://github.com/learnbyexample/TUI-apps

[AnthOlei]

Oh wow. I went though some of the perl book you linked and I was noticing the examples were really familiar; I then realized you were the same guy who wrote the awk book I have been going though.

Your work is excellent! Thank you, I’ll buy a copy soon.

[thesuperbigfrog]

>> I’m also noticing a lack of literature on Perl on the command line, I.e “perl -nae <some script>”.

Searching for 'perl one liners' and the related term 'perl golf' gives many articles and books:

https://www.perl.com/article/perl-one-liners-part-1/

https://www.perl.com/article/perl-one-liners-part-2/

https://catonmat.net/perl-one-liners-explained-part-one

https://catonmat.net/perl-one-liners-explained-part-two

https://catonmat.net/perl-one-liners-explained-part-three

https://catonmat.net/perl-one-liners-explained-part-four

https://catonmat.net/perl-one-liners-explained-part-five

https://catonmat.net/perl-one-liners-explained-part-six

https://catonmat.net/perl-one-liners-explained-part-seven

http://novosial.org/perl/one-liner/

https://www.oreilly.com/library/view/perl-one-liners/9781457...

[AnthOlei]

this is amazing! I guess I was searing the wrong keywords. Thanks, I’ll evaluate these.

[thiht]

I highly recommend awk because you can learn it in 20 minutes [1]

So you can start by spending 20 minutes to learn awk, and then spend 20 years to learn Perl (and use awk in the mean time)

[1]: https://ferd.ca/awk-in-20-minutes.html

[gregw2]

Having learned both at an intermediate level, I use awk for structured data handling, Perl for textual pattern matching.

Perl was inspired by awk, not the other way around. Perl is more general purpose.

[red-iron-pine]

this was my understanding as well. perl grew out of sed/awk/bash, so might as well learn perl.

that said, i found it gives a lot of 'foot-guns'.

[thesuperbigfrog]

Perl is far more capable than awk and more portable.

The issue with awk is that there are multiple non-interoperable implementations:

https://en.wikipedia.org/wiki/AWK#Versions_and_implementatio...

https://stackoverflow.com/questions/40409632/what-is-the-dif...

This makes awk scripts less portable than other text processing tools.

awk is also not extensible--it can be awkward (heh) to adapt it to some problems and big scripts get difficult to wrangle.

awk is great if your text processing problem is small.

Sometimes small problems grow in which case Perl tends to be a better choice:

https://stackoverflow.com/questions/366980/what-are-the-diff...

People love to hate perl, but there is a reason why it was installed on all Unix and Linux systems by default and was so popular for web servers on the early Internet (e.g. Apache mod_perl https://perl.apache.org/ and perl mason https://www.perl.com/pub/2002/12/11/mason.html/ ).

Perl also popularized regular expressions as a standard component of programming languages so much that "Perl-compatible regular expressions" are probably the most widely-used regex flavor versus POSIX or other regex variants:

http://www.pcre.org/

https://en.wikipedia.org/wiki/Regular_expression

[srean]

> but there is a reason why it was installed on all Unix and Linux systems by default

AWK has that one covered fair and square, even BusyBox has AWk.

[scrapheap]

If you're looking at learning Perl then you'll find the https://perldoc.perl.org/ site very useful (you'll probably also have its content available on the command line via the perldoc command).

If you want more information about running Perl on the command line then start with https://perldoc.perl.org/perlrun (or `perldoc perlrun` in your shell)

[pvg]

Maybe not so much a question of 'better' but perl is definitely bigger and multiheaded and does many, many more things (perl scientists were so preoccupied with whether or not they could, they didn't stop to think if they should). So you'll spend a lot more time (compared to awk) orienting yourself, beside the learning of the thing you want to know about. That could be a fun extra hobby or a waste of time, depending on your goals and inclinations.

[znpy]

Awk for sure! The pdf for the original awk book is somewhere around the web, i learned awk from that and it’s mostly still good. I heard they made a new edition some time ago? Really worth reading!

For perl… not sure. I learned a bit of perl in high school and never had a chance to use it, even at work where i occasionally see it being used from other people.

[srean]

How much time do you want to invest ?

Perl is more powerful but learning and remembering all (or a sufficiently more powerful superset of awk's capabilities) of it is going to take time. The peculiarities of its syntax is certainly not a small set.

All of awk on the other hand needs about 2 hours, if you have done some programming before.

[thesuperbigfrog]

>> Perl is more powerful but learning and remembering all (or a sufficiently more powerful superset of awk's capabilities) of it is going to take time. The peculiarities of its syntax is certainly not a small set.

I disagree.

Perl's syntax is easy to learn for anyone who has used C-style syntax as found in C, C++, Java, Javascript, or C#:

https://perldoc.perl.org/perlintro

Perl's syntax is more similar to AWK and C-style languages than Python is.

That said, there is nothing wrong with Python or AWK. They work great and have their places. But it is silly how many people treat Perl like it is impossible to learn or read without ever having used it or seeing where it is used.

[rurban]

Perl for sure, because it's much better than awk. Also perl has a huge set of libraries, awk has nothing.

[gregjor]

The tools have a lot of overlap. No need to choose one over the other, you can learn both. I don't write Perl but knowing how to read it comes in handy. AWK usually does what I need.

[Woshiwuja]

i feel like awk is better for simple pipings and the likes, so i guess it depends on what you need to do. I find myself using awk a lot expecially with other programs

[fiedzia]

use Python, so that other people can read it

[langcss]

But can they install the depencies and actually run it? :-)

[pestatije]

i wouldnt go for perl...id suggest the better question to be awk vs python, in which case id go for python

[Kim_Bruning]

This is a saner answer than you might think.

In theory Python is the more verbose of these languages. But when I switched from Perl to Python way back when, my short scripts actually got shorter, were quicker and easier to write, and you could actually read them back and understand what they said!

Of course both Perl and Python have changed a lot since I last compared them, so YMMV these days somewhat. I'd be interested to hear from a modern perl-er!

(And: I do still use PCRE libraries sometimes :-P )