[noamteyssier]

I've added awk into the benchmarks also!

[pabs3]

Also perl, should be faster than awk IIRC:

perl -ne 'print if ! $a{$_}++'