Linked by pkrumins on Thu 19th Feb 2009 12:17 UTC
General Development If you have ever been interested in awk and sed Unix utilities, then you probably know about the awk1line.txt and sed1line.txt files that are floating around the Internet. Each file contains around 80 idiomatic sed and awk one-liners for performing various text modification tasks.
Thread beginning with comment 349794
To read all comments associated with this story, please click here.
Michael
Member since:
2005-07-01

My dad is always singing the praises of Sed and Awk but that's not the only reason I don't like them ;)

They encourage these one-liners, which just means removing the formatting from code. They use regular expressions, something I consider should be avoided at all costs on account of their impenetrable syntax. My general feeling is that in the time it takes to figure out how to do anything with these tools, you could have just written a Python script to do it.

I'm sure they were great tools in their day but I really think Python trumps them, replacing all their functionality and throwing in maintainability to boot.

All that said, these are excellent, useful and well written articles. If I ever find I'm forced to use these things, I shall be eternally greatful that something like this exists.

Reply Score: 1

Kroc Member since:
2005-11-10

Depends on the load you have to lift. If I wanted to perform a quick bucket-sort, then I’d use PHP, VBScript or any other non-compact scripting language. But if I had to make a syntax processor (like I have: http://camendesign.com/code/remarkable ), then there’s no way I’d do it without regex. I’d have to practically reimplement a hard-coded regex engine in the process of handling the byte-by-byte matching for all the use-cases.

Sure the likes of sed and awk are hard to use, but there’s wizards out there who can, and those of us who can’t -- it’s not ours to say that one tool is better than the other, when in the right hands.

Reminds me of this T-Shirt - http://www.thinkgeek.com/tshirts-apparel/unisex/frustrations/374d/?... - "Go away or I will replace you with a very small shell script."

Edited 2009-02-19 13:08 UTC

Reply Parent Score: 1

massysett Member since:
2007-12-04

I've never considered awk hard to use. You can learn the basics in about thirty pages. Have you ever sat down and read "Effective GAWK Programming"?

http://www.gnu.org/software/gawk/manual/gawk.html

sed is not hard either. Python is great too, but for little quick jobs awk is great. awk can be just as maintainable if you write it cleanly (and I've seen python code that is a mess, because you can write messy Python too.) But for a one liner, who cares if it is maintainable? You use it once then throw it out!

Reply Parent Score: 3

spiderman Member since:
2008-10-23

First off, Python is not available anywhere, but awk and sed are (almost). Maybe perl is better than python, but anyway, that is not the point.
awk and sed scripts are not there to be maintained. They are not developer tools. They are administrator tools and they are there to make it easy for you to edit or search files quickly. sed and awk are used for one-shot commands in 95% of the cases. Once you have the result, there is no need to maintain the command at all.
How many lines do you have to write in python, just to open a file and read it? In sed or awk, that's 0. The file is open and parsed. There is no way you can make it faster in python.

Edited 2009-02-19 14:22 UTC

Reply Parent Score: 6

vivainio Member since:
2008-12-26

How many lines do you have to write in python, just to open a file and read it?


One. cont = open("file.txt").read().

Or

for line open("file.txt"): do_stuff_with(line)

If that means significant extra work for you, you need a harder problem domain ;-)

Reply Parent Score: 3

JMcCarthy Member since:
2005-08-12

Chinese (traditional/simplified) looks pretty incomprehensible to me but evidently a 1,000,000,000+ people manage to get by.

Without a doubt it's unwelcoming to those who don't understand it, but for those who do...

Reply Parent Score: 4

pkrumins Member since:
2008-05-07

That's why I wrote the articles, to explain them.

It is unfortunate that only the first paragraph shows on hacker news website.

Click on http://www.osnews.com/story/21004/Awk_and_Sed_One-Liners_Explained... to find links to my articles!

Reply Parent Score: 2

Googol Member since:
2006-11-24

no, there is not a billion people who can read Chinese, unlike popular believe, and there is a reason for that. Guess why they had to 'invent' simplified on top of traditional? The same applies to all other fancy writing systems.

However, the point is that different tools serve different purposes and you may get along with it without knowing it inside out. I wished I knew any which one of these to some extent, but absent an actual need, I cannot justify putting the required time into it. You will know better...

Reply Parent Score: 1

appel Member since:
2007-12-29

Are you serious?!

Regular expressions are awesome, and definately a core technology, even when using Python.

You are either a troll or completely clueless. soz.

Reply Parent Score: 5

poundsmack Member since:
2005-07-13

http://xkcd.com/208/

and of course:

http://xkcd.com/353/

and for what it's worth; Python > Perl

:)

Reply Parent Score: 4

Michael Member since:
2005-07-01

Regular expressions are very powerful but they cover a limited range of problems between the the trivial (the title is the example given to trim whitespace) and the very complex.

These old UNIX tools were terific in their day when they were the only way of doing things. But I think the fact that they continue to get so much air time has more to do with the fact that it's fun play with them. They're like crossword clues.

Don't underestimate what you can do with Python. It's trivial to read in a text file and split it into an array using whatever seperator you want. Modern scripting languages have tremendously powerful string handling functions and they do all this using real words. RE's are there if you need them but you very rarely do.

My work rate isn't limited by the speed at which I type, it's limited by the speed at which I think (that is, severely limited). I think better if I'm not having to translate everything via these arcane hieroglyphics.

I wish people were better at distinguishing between a genuinely held (and valid) opinion and a troll. It is only my opinion.

Reply Parent Score: 1

whartung Member since:
2005-07-06

I wouldn't really say that AWK and SED encourage these one liners, rather I would say they enable one liners.

When you work with these tools to the comfort level that you can just spit out these one liners, that's where this facility becomes a powerful command line tool rather than a just a scripting language.

I don't think twice about just pounding out long pipelines in the shell, or short scripts. Similarly, if it can fit on one line, I'll consider doing the same with AWK or SED.

These are one off events that never last beyond perhaps the shell history.

Reply Parent Score: 3

Delgarde Member since:
2008-08-19

They use regular expressions, something I consider should be avoided at all costs on account of their impenetrable syntax.


You obviously subscribe to the school of thought that says someone who uses regular expressions to solve a problem now has two problems. ;)

True to a degree - they tend to be overused, or made overly complicated by people who think that because they can read it, so will the next person who comes along. But they're also an extremely powerful tool, for which there really isn't any practical alternative - e.g if you want to validate that a string matches a pattern, you can use a simple regex to do it, or you can write your own parser. And writing your own parser is almost never the right answer.

Reply Parent Score: 4