Unix Shell Programming, Third Edition 292
| Unix Shell Programming, Third Edition | |
| author | Stephen G. Kochan and Patrick Wood |
| pages | 406 |
| publisher | SAMS |
| rating | 8 - Well written, good topic coverage, some small flaws |
| reviewer | Tony Williams |
| ISBN | 0672324903 |
| summary | Good introduction to shell programming and using the shell |
Now that we have languages such as Perl and Python, much of shell scripting has been forgotten. The need still arises for the times and places where running Perl would be just that little bit too much overhead; cron jobs, process start and stop scripts, even machine start and stop scripts. For these we could best go back to the old ways. Combining the power of the common Unix tools, pipes and scripts in a fairly obscure and slightly arcane syntax is not easy to pick up, though the language's simplicity does, in some ways, make it easier than more complex ones such as Perl. This book does a good job at introducing shell programming and I found it an excellent book when I needed a refresher.
I don't want to sell this volume short: you won't just learn about shell programming. The first ninety or so pages provide an excellent guide to getting the best out of the shell, and the last chapter is devoted to the features specific to an interactive shell such as command-line editing and using the history.
The authors have chosen to use the POSIX standard Bourne shell ('bash', available on many *nix systems, is a superset of the POSIX standard). That seems the right decision, given that it is so universally available and usually the default shell.
The book is well structured, starting out with a brief look at *nix operating systems before introducing the shell followed by some basic tools; cut, paste, sed, tr, grep, sort and uniq. One minor quibble, the book explains how to redirect STDOUT to a file and STDERR to a file, but not how to redirect both to the same file. That aside, these few chapters provide a good introduction to the shell.
The text goes on to systematically explore shell programming starting with variables and arithmetic. The chapters are kept short, in a good order and have a number of exercises at the end of each. The structure of the book and the order each new concept is introduced is well thought out; at each stage small examples are given that only use material already introduced and are complete in performing a task. In early chapters they are fairly trivial but by the end there is a fairly complete rolodex program written in shell script that would be a good model for anything you wished to do.
There is also a good summary of the shell syntax and common commands in Appendix A and good 'Further Information' in Appendix B. Kudos must go to the authors for a list of books for further reading that is not ashamed of mentioning other publishers, indeed they say "One of the best sources of books on Unix-related topics is O'Reilly and Associates" and list volumes from them before mentioning their own publishers.
There are some small typographic errors in the text but I did not find any in the script examples I tried. I found it to be well written and readable throughout, perhaps an advantage of a third edition in a slow moving technology.
You can visit the Sams web page devoted to the book which has the Table of Contents and the third chapter available for download. It has no errata or source code, I looked to see if the authors maintained a site for the book but could not find one.
I would recommend everyone read this book once or twice, it provides a comprehensive, well written tutorial on one of the most basic (and often overlooked) tools at your disposal. Even Windows users could install Cygwin and gain the benefit of a good POSIX compliant shell and this book. It also has the advantage that once purchased it will be useful for many, many years to come - the language has not changed noticeably in twenty five years and should not change in another twenty five.
You can purchase Unix Shell Programming, Third Edition from bn.com. Slashdot welcomes readers' book reviews -- to submit a review for consideration, read the book review guidelines, then visit the submission page.
Shell scripting is a Lost Art (Score:3, Insightful)
Re:Shell scripting is a Lost Art (Score:4, Interesting)
I find it to be a very nice tool to do simple jobs, without having to write a program to do it.
If I'm not mistaken Longhorn (next Windows) will even have a shell like interface (something other than the command prompt), so perhaps shell scripting will have some sort of a ressurection in the following years.
Yes, and... (Score:2)
Re:Missing the point. (Score:2)
I know what scripting is, my point was that KDE, Gnome etc, hell, even Windows have programs or "stuff" in the GUI that will let you set up what you want to run, when, and what you want it to do. You do not NEED to write a script to do it!
Re:Shell scripting is a Lost Art (Score:2, Interesting)
Throwing in a bit of sed handles most of the rest.
The power inherent in these simple tools is really quite amazing and knowing how to use them means I've never had to upgrade from my 486 laptop.
The only real downside is a need for at least a practical understanding of formal logic and set theory and some people are agin that.
Ta
Re:Shell scripting is a Lost Art (Score:2)
Those can usually be done on a single pipeline. Some people insist on using multiline shell scripts to do the trick.
Re:Shell scripting is a Lost Art (Score:2)
BTW, you can't honestly tell me you ENJOY programming in straight bash?!?!?
Re:Shell scripting is a Lost Art (Score:3, Interesting)
Hold on a damn minute... It's quite possible to write a virus in Bourne shell script, also. A virus that not only infects other shell scripts, but binaries as well. I know this because I did it as a project for a course in college.
Just goes to show you can do damn near anything in shell script.
No, I won't explain how it's done -- if you can't figure it out on your own, you're too stupid to handle the knowledge responsibly :-)
Re:Shell scripting is a Lost Art (Score:5, Informative)
Shell scripting usually trumps all other programatic languages in that the engine is so small. That allows it to start up and shutdown really quick, not like the bloated Perls, Pythons, Java's, VBs, etc. A typical shell script engine might use about a Meg or less. Shell scripting is not programming in the strict sense of the word. You are not working in a well structured and cohesive language to be used for a variety of application development. No, shell scripting was made primarily for process and file system management. Making shell scripting work much beyond those means is an art, and not something that should ordinarily be done because other better languages should be used to fulfill broader purposes.
In our Windows environment we use CMD.EXE, or the command shell engine for just about all of our network control scripting. It comes with Windows, and is the only standardized default shell console. The command shell of NT/2k/XP is NOT your fathers BATCH language. Microsoft made some extensive professional changes to the command shell engine starting way back with Windows NT 3.51. The command shell is now very functional and has many Bourne shell syntax features.
Microsoft hasn't promoted the use of the command shell until very recently. This is because early on, they wanted to push system programmers and admins into using their GUI tools and VB scripting languages so that you would have to buy other products. Microsoft really pushed people away from using command shell scripting for the purpose of their own gain. Recently, with the gowing popularity of Linux and Unix again, they are now pushing to get people to use the command shell. They've finally documented most of the many changes they've made to the shell over the years, those changes that were kept in secret to all but the most diehard shell users.
The most popular changes come with the following commands...
CALL, GOTO, EXIT, FOR, and SET.
These commands have been expanded for the development of subroutines, string manipulation, math functions, tokenizing, file system recursion and enumeration. The command processor is fairly powerful in its own right because you don't have to exit very much for externals like FINDSTR, or SED/AWK type executables.
Under NT/2k/XP if you are using the ".BAT" extension for your scripts, or the "COMMAND.COM" processor, then don't. These are now outdated. If you fire up command.com, the the 2k kernel must startup the 16 bit virtual dos machine (VDM). This is S-L-O-W, and the BATCH language is indeed very poor. You should now use the ".CMD" extension for your command shell scripting.
See the following for more command shell (cmd.exe) documentation...
Command Shell Docs [microsoft.com]
Re:Shell scripting is a Lost Art (Score:2)
Re:Shell scripting is a Lost Art (Score:2)
Re:LOL! (Score:2)
Your reaction however in my experience is typical of most Perl programmers. But her
Re:Shell scripting is a Lost Art (Score:2)
This raises an interesting question... (Score:2, Insightful)
Now that machines are getting much faster, can we expect shell scripting to be totally forgotten and unused? Are there movements for this already?
Why? What does speed have to do with it (Score:2)
All of these I can do by hand and on machines where the scripts are borked I do. I then replace the IF WHEN WHILE bits of the scripting language with my brain. In fact that is how I write scripts. I write down i
Re:This raises an interesting question... (Score:2)
As always.... (Score:2)
Amazon Link [amazon.com]
Cheaper at bookpool . . . (Score:2)
SCO (Score:2, Funny)
Default shell? (Score:5, Informative)
Re:Default shell? (Score:5, Funny)
When bash is ultimately so much more feature ridden than tcsh/ksh. Ok, just my observation/opinion.
But since bash is the default shell in Linux, and there are probably more linux systems than any other unix-like platform on planet earth, I vote that bash become earth's default shell.
Windows' default shell can probably be considered either Symantec or McAfee Anti Virus.
Re:Default shell? (Score:2)
Re:Default shell? (Score:3, Informative)
Re:Default shell? (Score:2)
Re:Default shell? (Score:2)
I didn't realize that because I don't use tcsh at all but thanks for pointing out the licensing thing here. And agreed - the GPL is less desirable - really the BSD license is the way to go and I'll continue using it in the future (tropism [freshmeat.net])
Does tcsh/csh have more "interactive" features than bourne? On the command line? I've never gotten this... what, for example? Thanks
Re:Default shell? (Score:2)
Well sure, you got history, tab completion, improved job control, etc. TCSH is just as interactive as BASH. Bash and TCSH have many things incommon in regards as the users point of view, and ease of use. However, both shells are not suitable for portable shell scripting, like bourne shell is. One difference is that TCSH is utterly unsuitable for shell script, as it is designe
Re:Default shell? (Score:2)
In bash and some (most?) sh variants, and ksh, and I believe the C shell families, which have substitution and string handling ability, you can do things like:
foo=sharkattack
echo ${foo/ark/---}
or
for file in $(ls -1 *.pls); do
mv $file $(echo ${file%%.pls}); done
So, with a little creativity, and not all that much effort, you can do some
Re:Default shell? (Score:2)
To put things in perspective, the *-DOS 1.x could be fit in 16 to 20 kB of RAM which included both COMMAND.COM and a couple of kB for the FAT. Not surprising that it didn't do much. OTOH, M$ was kinda slow updating the utilities, e.g. EDLIN didn't go away until DOS 5.0 (Tim Patterson was planning to have a successor ready sometime in 1981).
Re:Default shell? (Score:2)
I think a pretty strong case can be made for explorer over cmd as the default windows shell. CMD.exe is really nothing more than a command line interface to explorer. It implements only a narrow subset of explorers capabilities, relies on explorer much like any other called program for process control, filesystem access, etc. etc. etc.
As for WSH, forget it, same paradigm, different implementatio
Re:Default shell? (Score:2)
He said Bourne (sh), not Bash. I'm don't agree that his statement is overly broad, Bourne is the default shell on the greatest number of installed systems. Most admins work on systems where Bourne is the default shell.
/bin/bash as /bin/sh considered harmful (Score:2)
It's fine to have bash as your everyday user shell. It's even ok to have it as your default shell for root, though having a second uid 0 login with some more minimal shell (like ash, for example) is often a good idea. However, I wouldn't make it as absolutely essential for the system as it is when it's
Re:Default shell? (Score:3, Informative)
Re:Default shell? (Score:2)
(There used to be occasional flame threads on debian-devel about this)
Re:Default shell? (Score:2)
Re:Default shell? (Score:2)
Huh? I think you mean "Linux", not "modern systems."
Re:Default shell? (Score:2, Informative)
Appears we interpret this differently ..
I read that as stating they chose the original Bourne shell, eg /bin/sh, with an aside (for Linux folks?) that bash is a superset of sh.
Re:Default shell? (Score:2)
Mother of All Shells (Score:2)
Basically, if you have a *NIX like system, you are pretty much guaranteed to have a /bin/sh, and it will always work the same. Bash is common, but not universal. Ksh is proprietary, though there are open source versions (pdksh), but you won't find it everywhere. Csh
Re:Default shell? (Score:2)
root@server: > echo $SHELL
root@server: > grep jrashaad
jrashaad:!:206:1::/home/jrashaad:/us
root@server: > uname -a
AIX server 3 4 000XXXXXXxX
So, yeah. Unless the default shell in the 5 series has changed.
But /bin/sh _is_ installed (Score:3, Interesting)
And every unix system has some sort of
Re:HPUX (Score:2)
FWIW, the Korn shell was available before BASH and came with most commercial unices from about 1990 on. Korn's intent was to have a shell that was useful for programming as the original Bourne shell and was as nice for interactive use as the C shell.
BASH is common on Linux as the source code for KSH was not widely available until relatively recently. The other problem is
Shell scripting is great, even in Windows! (Score:4, Informative)
Shell Scripts are really The Cat's Pajamas.
Re:Shell scripting is great, even in Windows! (Score:2)
You gotta say: download things thing and run this.
Not so nice.
Re:Shell scripting is great, even in Windows! (Score:2)
Hmm.....
Re:Shell scripting is great, even in Windows! (Score:2, Informative)
Re:Shell scripting is great, even in Windows! (Score:2)
Sh is a little spartan also... not to argue the relative merits of the various shells (my work requires us to use tcsh of all things) but the posters point about portability was, well, ridiculous.
Re:Shell scripting is great, even in Windows! (Score:2)
Oldie, but goodie differences ... (Score:5, Interesting)
One of the first things I remember reading was the FAQ describing the shell differences. [faqs.org]
Imagine my surprise that the command line could hold so much control and there were other aspects of controlling a PC. I haven't looked back since, but that page really brings back some memories.
It's funny how something you read can cause one of those lightbulb moments.
how touching (Score:2)
GET http://www.perfidious.org/award|\
sed '1!G;h;$!d'|sed '/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//'|\
writ
The Land of the Lost (Score:3)
Back when dinosaurs roamed the earth and NCR made Unix computers
I used to work on an AT&T 3b2. I can also script pretty well in DCL. Does that make me a sleestak?
Extra points for anyone younger than 30 who can prove they actually know DCL. Demonstrate something good - maybe set up IPC using mailboxes or something.
Ah, DCL (Score:2)
Unfortunately, it's almost all left me, though I suppose I could still puzzle out the quoting rules to pass quoted arguments down three levels of @-signing. (Doubling the quotes each time...)
And hey, _I'm_ still under thirty (for two more years...)
Re:The Land of the Lost (Score:2)
10 years ago I've created document-oriented transactioned RPC based on Sendmail, Procmail, PGP, and Shell scripting (and OraTCL). It was secure and reliable on very unreliable phone lines (ever connected with 1200 bod ?). Does it give me any points? :)
Or, for the free speech *and* beer people (Score:5, Informative)
Re:Or, for the free speech *and* beer people (Score:2)
Can anyone point me to something (other than the reviewed book) I can read to learn about shell scripting?
Re:Or, for the free speech *and* beer people (Score:2)
Not too closely it appears, since as it says in bold on the front page "the only way to really learn scripting is to write scripts". It's right. There seems to be a view amongst people who "can't" write shell script that it needs to be taught which is utterly wrong. It is not an arcane programming language like C for crying out loud; it's the commands you use at the command line *every* day, unless you are entirely reliant on a GUI.
The only major diffe
Re:Or, for the free speech *and* beer people (Score:2)
Yes, shell scripting seems obvious, I would have just liked something less intimidating than an A-Z reference (maybe a limited subset that just covers the bit you need to get started).
Advanced Bash Shell Scripting conveniently glosses over somethings like how variables work in a Bash script and so on. (or at least, if it's there it isn't well explained).
I did some work for him on this book (Score:3, Interesting)
Patrick's a pretty sharp guy but he outsources a lot of his research to proffesionals (makes sense) and has several staff people help him put the pieces together, as it were.
I offered my services as part of the FTEST (final tech editing service team) but Patrick didn't want a publishing pundit as much as he was looking for computer pundits. Ah well, at least now I'm in his rolodex and hopefully I'll get more chances to work with him.
Warmest regards,
--Jack
I also worked on his book... (Score:2)
Re:I did some work for him on this book (Score:2)
Download scripts and errata here (Score:5, Informative)
The one Amazon.com [amazon.com] review gave this book 4 stars.
Helevius
Shell scripts (Score:5, Insightful)
I make my living as a Unix sysadmin, to support my hobby as a Real Programmer :-p.
Without shell scripts, I'd be lost.
Shell scripts provide the ability to leverage the work of every well-behaved perl (etc.) script, binary in /bin, and other shell script ever written. While the same can be said for perl scripts (which I happily use or write when needed), with shell scripting this is the intended paradigm.
Scripts are intrinsically open source, even if copyrighted and under a closed license. The techniques are visible, and thus instructive.
Given Unix's design attribute of easy process creation, shell scripting is often the best way for me to handle a task.
See some examples here [healconsulting.com].
Re:Shell scripts (Score:2)
Ok, slowly: notice the use of lower case in "open source". Notice the next sentence, which talks about techniques being instructive. Copyright does not disallow copying verbatim into your head. Once in your head, you can examine it any way you wish. You can even write it down. You just can't let someone else have a copy.
But the point about techniques being ins
what a coincidence (Score:2)
"It is easier to port a shell than a shell script." -- Larry Wall
Re:what a coincidence (Score:2)
Damned reality...
Really the better one is this one: (Score:3, Informative)
Of the same name, not related. Go figure. I got both and the one above is the better of the two, IMHO. Chapter 5 of this one is golden, lots of simple tables covering built in operaters, varibles, and control structures.
I really did not GET programming until the book above, it made it click for me. Perl came right after, as this got the concepts in my thick head.
A shell-less admin is a useless as a mermaid, and not as pretty to look at.
Re:Really the better one is this one: (Score:2)
Best thing about mermaids is they breath with gills. No coming up for air.
Automatic quote rotation? (Score:2, Interesting)
"It is easier to port a shell than a shell script." -- Larry Wall
?
Not such a great option these days (Score:3, Interesting)
But these days as soon as I need a conditional or a loop I start thinking about doing the task in Perl or Python. These languages are so much easier to read and let you fork a process if you need to dip down into the shell.
Another good book (Score:2, Informative)
And free as in beer:
Advanced Bash-Scripting Guide [tldp.org]
Aaaauuugh! NCR! (Score:3, Funny)
True, but how long? (Score:2)
Is this a good thing? I mean don't we sometimes have to abandon the ideas of old in order to progress? How has English evolved over the past 50 years? Besides the branch of American English and the ones the UK speaks and writes. Maybe not much but how about 300 years? As technology progresses further than a Moore breaking speed should the language not evolve as well?
I agree that Unix language is so
Shell procedures -- not shell scripts (Score:2)
smartadmin a scripting example using dialog (Score:2)
ftp://ftp.crashrecovery.org/pub/linux/smartadmin/ [crashrecovery.org]
smartadmin - Create/Edit/Remove a UNIX/Linux user and/or a Samba user.
smartadmin was created out of sheer misgrief about total contra produc-
tive effects of packages like webmin, requiring rediculous resources to
perform basic user administration on a UNIX/Linux and Samba server. The
requirements for smartadm
My mantra (Score:2, Insightful)
When the Junior Unix Sysadmin comes to me, as they do, and asks "what language should I write this tool in?", here is my decision sequence:
Re:Shell scripting is deprecated (Score:2, Interesting)
*) is available.
It's new years eve, wine made me do the grammatcal/semantic error.
And thanks, anonymour moderator, for the "troll" moderation. Want to silence any discussion that touches too close to the home, right?
Re:Shell scripting is deprecated (Score:3, Informative)
The shell is universally available. On some low-end systems (yes they still exist), Python and even Perl might not be installed.
Re:Shell scripting is deprecated (Score:2)
Yes, of course. Yet most of the scripts that sysadmins write are for normal systems with no such limitations.
I'm not saying shell scripts have no purpose. I'm saying that in 95% of of the cases they are used a more structured solution would have been more appropriate.
BZZZT. Name one distro that doesn't include them (Score:2)
BZZZT. You are a newbie (Score:2)
Re:BZZZT. You are a newbie (Score:2)
I'M A NEWBIE??? Gee - like i couldn't compile perl to reside in /bin??????
Make it your goal in 2004 to understand the --prefix feature of the gnu build toolset. And to think you are calling me a newbie.
Okay (Score:2, Informative)
Here's a clue: there's more to *nix than Linux.
Re:Shell scripting is deprecated (Score:2)
Of course, I know where to draw the line. Another sysadmin here wrote a set of backup scripts (thankfully now lost) that were over 10000 lines of sh, with embedded awk and sed. It maintained a tape database, etc. It was truly insane to debug. Anything more than about 300 lines ends up either as a set of small, easily maintained scri
Re:Shell scripting is deprecated (Score:3, Informative)
Re:Shell scripting is deprecated (Score:2)
Name three major modern unices that does not include a decent scripting language by default. Perl and Python typically do show up in most self-respecting distros or are trivially available as precompiled packages.
It is the same language you use in an interactive shell
This is basically the "legacy" argument. You can use perl or python as interactive shells, there are tools to aid in this.
It is gener
Re:What do you mean by default? (Score:2)
Re:Shell scripting is deprecated (Score:2)
Techically it's not 'written', it's an autoconf script.
Knowing shell scripting also helps when working in interactive mode, a loop or case/esac is also useful in one-liners.
This is true. Especially for loop comes in handy in interactive sessions.
Knowing shell scripts is good, even necessary for sysadmins. Writing shell scripts is what should be discouraged.
You can source shell scripts, hence you can write a shell
Moderators are clearly not sysadmins (Score:2)
I typically discard shell for anything more complex than a while/do sh loop. For anything moderately complex, sh is baroque, syntactically retarded, and feature-deficient.
Re:Shell scripting is deprecated (Score:2)
os.system("mycommand a.txt")
Python will make you do incredibly stupid and redundant things like open files.
How? Could you elaborate? You don't need to do anything you don't need with shell scripts.
Secondly, injecting a filter between pipes in Python means either reading in the input entirely (waste of memory), or reading it piecemeal (verbo
Re:Shell scripting is deprecated (Score:2)
Well, the situation is not buch better with shell scripts on windows - cygwin just doesn't work like it should.
On Python I have to write a wrapper function just to make things usable.
I always have a wrapper fn for os.system. Several, in fact. Typically I print out the command and then execute it. Sometimes w/ logging, sometimes w/o.
In Python, I would either have to fork explicitly (yuck), or read in from the input piecemeal, and then write my tran
Re:Shell scripting is deprecated (Score:2)
Of course it does. That's why you can do most of the stuff you use shell for in Python.
I'm obviously not saying that the shell is evil incarnate that should be removed from the system. I'm saying that programming straight for it, instead of invoking it from Python, is futile.
Also, how is os.system() supposed to be less verbose?
We developers then to use these things called "functions" to make stuff less verbose. And nobody prevents you from d
Classic "young programmer" misconceptions (Score:2)
If memory concerns are truly an issue, you are not going to get relief by simply switching environments. You probably need to rethink and redesign.
Re:Shell scripting is deprecated (Score:2)
I think that says more about Perl than shell scripts
Re:Shell scripting is deprecated (Score:2)
Do you have an interactive debugger for sh? Thought not. Do you have language-level features for dumping and evaluating data structures in sh at run-time as inspectable structures? Thought not.
Re:Shell scripting is deprecated (Score:2)
What you are describing is a syntax check, not a debugger.
Re:Shell scripting is deprecated (Score:3, Interesting)
If you are serious, you need to freshen up your computing skills a little bit. Shell scripts tend to fail in unpredictable ways, often with no clue what went wrong. Python script raises an exception that immediately tells where the problem is.
Programming languages are for writing programs to do things, sh scripts are for tying things together in an automated way.
You don't realize that things have changed from the
Re:Shell scripting is deprecated (Score:2, Informative)
Re:Shell scripting is deprecated (Score:2)
os.system("mount
i'm not even a python user and i know you can do that
Re:Please kill sh and its children (Score:2)
This book is irrelevant because anyone sane already knows the futility in trying to do anything non-trivial in sh.
I disagree and refer you to an example of a significant shell application that is complex and useful: Enhanced Network Scanjet 5 [madole.net] (I'm the author).
This is about 2,400 lines of Bourne shell (runs under BSD sh or Bash) that reimplements the HP Network Scanjet 5 functionality, with many improvements, particularly the fact that is is completely standalone.
This application was originally writt
Re:Please kill sh and its children (Score:2)
For fun I downloaded your script. You are truly a glutton for punishment. 80% of this code could have been more sanely expressed in perl. Your script demonstrates extreme proficiency in sh scripting (hats off to you) but in no way invalidates the well-established idea that perl or python are superior for long scripts. I suspect th
Re:wtf is up with this RACISM? (Score:2)