INDEX | SITEMAP | GOOGLE | LINKS | UPDATES | BLOG | EMAIL | DONATE | HOME

A Shell Programming Primer

v1.0.7 / 01 feb 13 / greg goebel / public domain

* The UNIX operating system provides a flexible set of simple tools to perform a wide variety of system-management, text-processing, and general-purpose tasks. These simple tools can be used in very powerful ways by tying them together programmatically, using "shell scripts" or "shell programs".

The UNIX "shell" itself is a user-interface program that accepts commands from the user and executes them. It can also accept the same commands written as a list in a file, along with various other statements that the shell can interpret to provide input, output, decision-making, looping, variable storage, option specification, and so on. This file is a shell program.

Shell programs are, like any other programming language, useful for some things but not for others. They are excellent for system-management tasks but not for general-purpose programming of any sophistication. Shell programs, though generally simple to write, are also tricky to debug and slow in operation.

There are three versions of the UNIX shell: the original "Bourne shell (sh)", the "C shell (csh)" that was derived from it, and the third-generation "Korn shell (ksh)". The Bourne shell is in popular use as the freeware "Bourne-again shell" AKA "bash".

This document focuses on the Bourne shell. The C shell is more powerful but has various limitations, and while the Korn shell is clean and more powerful than the other two shells, it is a superset of the Bourne shell: anything that runs on a Bourne shell runs on a Korn shell, though the reverse is not true. Since the Bourne shell's capabilities are probably more than most people require, there's no reason to elaborate much beyond them in an introductory document, and the rest of the discussion will assume use of the Bourne shell unless otherwise stated.


[1] GETTING STARTED
[2] SHELL VARIABLES
[3] COMMAND SUBSTITUTION
[4] COMMAND-LINE ARGUMENTS
[5] DECISION-MAKING & LOOP CONSTRUCTS
[6] OTHER SHELL FEATURES
[7] USEFUL TOOLS
[8] REGULAR EXPRESSIONS
[9] GREP, SED, & AWK
[10] SHELL PROGRAM EXAMPLES
[11] QUICK REFERENCE
[12] REVISION HISTORY

[1] GETTING STARTED

* The first thing to do in understanding shell programs is to understand the elementary system commands that can be used in them. A list of fundamental UNIX system commands follows:

  ls         # Give a simple listing of files.
  cp         # Copy files.
  mv         # Move or rename files.
  rm         # Remove files.  
  rm -r      # Remove entire directory subtree.
  cd         # Change directories.
  pwd        # Print working directory.
  cat        # Lists a file or files sequentially.
  more       # Displays a file a screenfull at a time.
  pg         # Variant on "more".
  mkdir      # Make a directory.
  rmdir      # Remove a directory.

The shell executes such commands when they are typed in from the command prompt with their appropriate parameters, which are normally options and file names.

* The shell also allows files to be defined in terms of "wildcard characters" that define a range of files. The "*" wildcard character substitutes for any string of characters, so:

   rm *.txt

-- deletes all files that end with ".txt". The "?" wildcard character substitutes for any single character, so:

   rm book?.txt

-- deletes "book1.txt", "book2.txt", and so on. More than one wildcard character can be used at a time, for example:

   rm *book?.txt

-- deletes "book1.txt", "mybook1.txt", "bigbook2.txt", and so on.

* Another shell capability is "input and output redirection". The shell, like other UNIX utilities, accepts input by default from what is called "standard input", and generates output by default to what is called "standard output". These are normally defined as the keyboard and display, respectively, or what is referred to as the "console" in UNIX terms. However, standard input or output can be "redirected" to a file or another program if needed. Consider the "sort" command. This command sorts a list of words into alphabetic order; typing in:

   sort
   PORKY
   ELMER
   FOGHORN
   DAFFY
   WILE
   BUGS
   <CTL-D>

-- spits back:

   BUGS
   DAFFY
   ELMER
   FOGHORN
   PORKY
   WILE

Note that the CTL-D key input terminates direct keyboard input. It is also possible to store the same words in a file and then "redirect" the contents of that file to standard input with the "<" operator:

   sort < names.txt

This would list the sorted names to the display as before. They can be redirected to a file with the ">" operator:

   sort < names.txt > output.txt

They can also be appended to an existing file using the ">>" operator:

   sort < names.txt >> output.txt

In these cases, there's no visible output, since the command just executes and ends. However, if that's a problem, it can be fixed by connecting the "tee" command to the output through a "pipe", designated by "|". This allows the standard output of one command to be chained into the standard input of another command. In the case of "tee", it accepts text into its standard input and then dumps it both to a file and to standard output:

   sort < names.txt | tee output.txt

So this both displays the names and puts them in the output file. Many commands can be chained together to "filter" information through several processing steps. This ability to combine the effects of commands is one of the beauties of shell programming. By the way, "sort" has some handy additional options:

   sort -u    # Eliminate redundant lines in output.
   sort -r    # Sort in reverse order.
   sort -n    # Sort numbers. 
   sort -k 2  # Skip first field in sorting.

* If a command generates an error, it is displayed to what is called "standard error", instead of standard output, which defaults to the console. It will not be redirected by ">". However, the operator "2>" can be used to redirect the error message. For example:

   ls xyzzy 2> /dev/null

-- will give an error message if the file "xyzzy" doesn't exist, but the error will be redirected to the file "/dev/null". This is actually a "special file" that exists under UNIX where everything sent to it is simply discarded.

* The shell permits the execution of multiple commands sequentially on one line by chaining them with a ";":

   rm *.txt ; ls

A time-consuming program can also be run in a "parallel" fashion by following it with a "&":

   sort < bigfile.txt > output.txt &

* These commands and operations are essential elements for creating shell programs. They can be stored in a file and then executed by the shell. To tell the shell that the file contains commands, just mark it as "executable" with the "chmod" command. Each file under UNIX has a set of "permission" bits, listed by an "ls -l" -- the option providing file details -- as:

   rwxrwxrwx

The "r" gives "read" permission, the "w" gives "write" permission, and the "x" gives "execute" permission. There are three sets of these permission bits, one for the user, one for other members of a local group of users on a system, and one for everyone who can access the system -- remember that UNIX was designed as a multiuser environment.

The "chmod" command can be used to set these permissions, with the permissions specified as an octal code. For example:

   chmod 644 myfile.txt

This sets both read and write permission on the file for the user, but everybody else on the system only gets read permission. The same octal scheme can be used to set execute permission, though it's simpler just to use chmod "+x" option:

   chmod +x mypgm

This done, if the name "mypgm" is entered at the prompt, the shell reads the commands out of "mypgm" and executes them. The execute permission can be removed with the "-x" option.

For example, suppose we want to be able to inspect the contents of a set of archive files stored in the directory "/users/group/archives". We could create a file named "ckarc" and store the following command string in it:

  ls /users/group/archives | pg

This is a very simple shell program. As noted, the shell has control constructs, supports storage variables, and has several options that can be set to allow much more sophisticated programs. The following sections describe these features in a quick outline fashion.

Incidentally, this scheme for creating executable files is for the UNIX environment. Under the Windows environment, the procedure is to end shell program file names in a distinctive extension -- ".sh" is a good choice, though any unique extension will do -- and then configure Windows to run all files with that extension with a UNIX-type shell, usually bash.

BACK_TO_TOP

[2] SHELL VARIABLES

* The first useful command to know about in building shell programs is "echo", which can be used to produce output from a shell program:

   echo "This is a test!"

This sends the string "This is a test!" to standard output. It is recommended to write shell programs that generate some output to inform the user of what they are doing.

The shell allows variables to be defined to store values. It's simple, just declare a variable is assign a value to it:

   shvar="This is a test!"

The string is enclosed in double-quotes to ensure that the variable swallows the entire string (more on this later), and there are no spaces around the "=". The value of the shell variable can be obtained by preceding it with a "$":

   echo $shvar

This displays "This is a test!". If no value had been stored in that shell variable, the result would have simply been a blank line. Values stored in shell variables can be used as parameters to other programs as well:

   ls $lastdir

The value stored in a shell variable can be erased by assigning the "null string" to the variable:

   shvar=""

There are some subtleties in using shell variables. For example, suppose a shell program performed the assignment:

   allfiles=*

-- and then performed:

   echo $allfiles

This would echo a list of all the files in the directory. However, only the string "*" would be stored in "allfiles". The expansion of "*" only occurs when the "echo" command is executed.

Another subtlety is in modifying the values of shell variables. Suppose we have a file name in a shell variable named "myfile" and want to copy that file to another with the same name, but with "2" tacked on to the end. We might think to try:

   mv $myfile $myfile2

-- but the problem is that the shell will think that "myfile2" is a different shell variable, and this won't work. Fortunately, there is a way around this; the change can be made as follows:

   mv $myfile ${myfile}2

A UNIX installation will have some variables installed by default, most importantly $HOME, which gives the location of a particular user's home directory.

As a final comment on shell variables, if one shell program calls another and the two shell programs have the same variable names, the two sets of variables will be treated as entirely different variables. To call other shell programs from a shell program and have them use the same shell variables as the calling program requires use of the "export" command:

   shvar="This is a test!"
   export shvar
   echo "Calling program two."
   shpgm2
   echo "Done!"

If "shpgm2" simply contains:

   echo $shvar

-- then it will echo "This is a test!".

BACK_TO_TOP

[3] COMMAND SUBSTITUTION

* The next step is to consider shell command substitution. Like any programming language, the shell does exactly what it is told to do, and so it is important to be very specific when telling it to do something. As an example, consider the "fgrep" command, which searches a file for a string. For example, to search a file named "source.txt" for the string "Coyote", enter:

   fgrep Coyote source.txt

-- and it would print out the matching lines. However, suppose we wanted to search for "Wile E. Coyote". If we did this as:

   fgrep Wile E. Coyote source.txt

-- we'd get an error message that "fgrep" couldn't open "E.". The string has to be enclosed in double-quotes (""):

   fgrep "Wile E. Coyote" source.txt

If a string has a special character in it, such as "*" or "?", that must be interpreted as a "literal" and not a wildcard, the shell can get a little confused. To ensure that the wildcards are not interpreted, the wildcard can either be "escaped" with a backslash ("\*" or "\?") or the string can be enclosed in single quotes, which prevents the shell from interpreting any of the characters within the string. For example, if:

   echo "$shvar"

-- is executed from a shell program, it would output the value of the shell variable "$shvar". In contrast, executing:

   echo '$shvar'

-- the output is the string "$shvar".

* Having considered "double-quoting" and "single-quoting", let's now consider "back-quoting". This is a little tricky to explain. As a useful tool, consider the "expr" command, which can be used to perform simple math from the command line:

   expr 2 + 4

This displays the value "6". There must be spaces between the parameters; in addition, to perform a multiplication the "*" has to be "escaped" so the shell doesn't interpret it:

   expr 3 \* 7 

Now suppose the string "expr 12 / 3" has been stored in a shell variable named "shcmd"; then executing:

   echo $shcmd

-- or:

   echo "$shcmd"

-- would simply produce the text "expr 12 / 3". If single-quotes were used:

   echo '$shcmd'

-- the result would be the string "$shcmd". However, if back-quotes, the reverse form of a single quote, were used:

   echo `$shcmd`

-- the result would be the value "4", since the string inside "shcmd" is executed. This is an extremely powerful technique that can be very confusing to use in practice.

BACK_TO_TOP

[4] COMMAND-LINE ARGUMENTS

* In general, shell programs operate in a "batch" mode, that is, without interaction from the user, and so most of their parameters are obtained on the command line. Each argument on the command line can be seen inside the shell program as a shell variable of the form "$1", "$2", "$3", and so on, with "$1" corresponding to the first argument, "$2" the second, "$3" the third, and so on.

There is also a "special" argument variable, "$0", that gives the name of the shell program itself. Other special variables include "$#", which gives the number of arguments supplied, and "$*", which gives a string with all the arguments supplied.

Since the argument variables are in the range "$1" to "$9", so what happens there's more than 9 arguments? No problem, the "shift" command can be used to move the arguments down through the argument list. That is, when "shift" is executed, then the second argument becomes "$1", the third argument becomes "$2", and so on; and if a "shift" is performed again, the third argument becomes "$1"; and so on. A count can be specified to cause a multiple shift:

   shift 3

-- shifts the arguments three times, so that the fourth argument ends up in "$1".

BACK_TO_TOP

[5] DECISION-MAKING & LOOP CONSTRUCTS

* Shell programs can perform conditional tests on their arguments and variables and execute different commands based on the results. For example:

   if [ "$1" = "hyena" ]
   then
     echo "Sorry, hyenas not allowed."
     exit
   elif [ "$1" = "jackal" ]
   then
     echo "Jackals not welcome."
     exit
   else
     echo "Welcome to Bongo Congo."
   fi 
   echo "Do you have anything to declare?"

-- checks the command line to see if the first argument is "hyena" or "jackal" and bails out, using the "exit" command, if they are. Other arguments allow the rest of the file to be executed. Note how "$1" is enclosed in double quotes, so the test will not generate an error message if it yields a null result.

There are a wide variety of such test conditions:

   [ "$shvar" = "fox" ]    String comparison, true if match.
   [ "$shvar" != "fox" ]   String comparison, true if no match.
   [ "$shvar" = "" ]       True if null variable.
   [ "$shvar" != "" ]      True if not null variable.

   [ "$nval" -eq 0 ]       Integer test; true if equal to 0.
   [ "$nval" -ge 0 ]       Integer test; true if greater than or equal to 0.
   [ "$nval" -gt 0 ]       Integer test; true if greater than 0.
   [ "$nval" -le 0 ]       Integer test; true if less than or equal to 0.
   [ "$nval" -lt 0 ]       Integer test; true if less than to 0.
   [ "$nval" -ne 0 ]       Integer test; true if not equal to 0.

   [ -d tmp ]              True if "tmp" is a directory.
   [ -f tmp ]              True if "tmp" is an ordinary file.
   [ -r tmp ]              True if "tmp" can be read.
   [ -s tmp ]              True if "tmp" is nonzero length.
   [ -w tmp ]              True if "tmp" can be written.
   [ -x tmp ]              True if "tmp" is executable.

Incidentally, in the example above:

   if [ "$1" = "hyena" ]

-- there is a potential pitfall in that a user might enter, say, "-d" as a command-line parameter, which would cause an error when the program was run. Now there is only so much that can be done to save users from their own clumsiness, and "bullet-proofing" simple example programs tends to make them not so simple any more, but there is a simple if a bit cluttered fix for such a potential pitfall. It is left as an exercise for the reader.

There is also a "case" control construct that checks for equality with a list of items. It can be used with the example at the beginning of this section:

   case "$1" 
   in
     "gorilla")  echo "Sorry, gorillas not allowed."
                 exit;;
     "hyena")    echo "Hyenas not welcome."
                 exit;;
     *)          echo "Welcome to Bongo Congo.";;
   esac

The string ";;" is used to terminate each "case" clause.

* The fundamental loop construct in the shell is based on the "for" command. For example:

   for nvar in 1 2 3 4 5
   do
     echo $nvar
   done

-- echoes the numbers 1 through 5. The names of all the files in the current directory could be displayed with:

   for file in *
   do
     echo $file
   done

One nice little feature of the shell is that if the "in" parameters are not specified for the "for" command, it just cycles through the command-line arguments.

* There is a "break" command to exit a loop if necessary:

   for file
   do
     if [ "$file" = punchout ]
     then 
       break
     else
       echo $file
     fi
   done

There is also a "continue" command that starts the next iteration of the loop immediately. There must be a command in the "then" or "else" clauses, or the result is an error message. If it's not convenient to actually do anything in the "then" clause, a ":" can be used as a "no-op" command:

   then
     :
   else

* There are two other looping constructs available as well, "while" and "until". For an example of "while":

   n=10
   while [ "$n" -ne 0 ]
   do
     echo $n
     n=`expr $n - 1`
   done

-- counts down from 10 to 1. The "until" loop has similar syntax but tests for a false condition:

   n=10
   until [ "$n" -eq 0 ]
   do
   ...
BACK_TO_TOP

[6] OTHER SHELL FEATURES

* There are other useful features available for writing shell programs. For example, comments can be placed in shell programs by preceding the comments with a "#":

   # This is an example shell program.
   cat /users/group/grouplog.txt | pg    # Read group log file.

It is strongly recommended to comment all shell programs. If they are just one-liners, a simple comment line at the top of the file will do. If they are complicated shell programs, they should have a title, revision number, revision date, and revision history along with descriptive comments. That will prevent confusion if multiple versions of the same shell program are found, or if the program is modified later. Shell programs can be obscure, even by the standards of programming languages, and it is useful to provide a few hints.

* Standard input can be read into a shell program using the "read" command. For example:

   echo "What is your name?"
   read myname
   echo $myname

-- echoes the user's own name. The "read" command will read each item of standard input into a list of shell variables until it runs out of shell variables, and then it will read all the rest of standard input into the last shell variable. As a result, in the example above, the user's name is stored into "myname".

* If a command is too long to fit on one line, the line continuation character "\" can be used to put it on more than one line:

   echo "This is a test of \
         the line continuation character."

* There is a somewhat cryptic command designated by "." that executes a file of commands within a shell program. For example:

  . mycmds

-- will execute the commands stored in the file "mycmds". It's something like an "include" command in other languages.

* For debugging, the execution of a shell program can be traced using the "-x" option with the shell:

   sh -x mypgm *

This traces out the steps "mypgm" takes during the course of its operation.

* One last comment on shell programs before proceeding: What happens if with a shell program that just performs, say:

  cd /users/coyote

-- to change to another directory? Well ... nothing happens. After the shell program runs and exits, the directory remains unchanged. The reason is that the shell creates a new shell, or "subshell", to run the shell program, and when the shell program is finished, the subshell vanishes, along with any changes made in that subshell's environment. It is easier, at least in this simple case, to define a command alias in the UNIX "login" shell instead of struggling with the problem in shell programs.

BACK_TO_TOP

[7] USEFUL TOOLS

* Before we go on to practical shell programs, let's consider a few more useful tools.

The "paste" utility takes a list of text files and concatenates them on a line-by-line basis. For example:

   paste names.txt phone.txt > data.txt

-- takes a file containing names and a file containing corresponding phone numbers and generates a file with each name and number "pasted" together on the same line.

* The "head" and "tail" utilities list the first 10 or last 10 lines in a file respectively. The number of lines to be listed can be specified if needed:

   head -n -5 source.txt      # List first 5 lines.
   tail -n -5 source.txt      # List last 5 lines.
   tail -n +5 source.txt      # List all lines from line 5.

* The "tr" utility translates from one set of characters to another. For example, to translate uppercase characters to lowercase characters:

   tr '[A-Z]' '[a-z]' < file1.txt > file2.txt

The reverse conversion can of course be made using:

   tr '[a-z]' '[A-Z]' < file1.txt > file2.txt

The "-d" option deletes a character. For example:

   tr -d '*'

-- deletes all asterisks from the input stream. Note that "tr" only works on single characters.

* The "uniq" utility removes duplicate consecutive lines from a file. It has the syntax:

   uniq source.txt output.txt

A "-c" option provides an additional count of the number of times a line was duplicated, while a "-d" option displays only the duplicated lines in a file.

* The "wc (word count)" utility tallies up the characters, words, and lines of text in a text file. It can be invoked with the following options:

    wc -c        # Character count only.
    wc -w        # Word count only.
    wc -l        # Line count only.

* The "find" utility is extremely useful, if a little hard to figure out. Essentially, it traverses a directory subtree and performs whatever action specified on every match. For example:

   find / -name findtest.txt -print

This searches from the root directory ("/") for "findtest.txt", as designated by the "-name" option, and then prints the full pathname of the file, as designated by the "-print" option. Incidentally, "find" must be told what to do on a match; it will not by default say or do anything, it will just keep right on searching.

There are a wide variety of selection criteria. Simply printing out the names of directories in a search can be done with:

   find . -type d -print

Files can also be found based on their username, date of last modification, size, and so on.

BACK_TO_TOP

[8] REGULAR EXPRESSIONS

* An advanced set of tools can be used to perform searches on text strings in files and, in some cases, manipulate the strings found. These tools are known as "grep", "sed", and "awk" and are based on the concept of a "regular expression", which is a scheme by which specific text patterns can be specified by a set of special or "magic" characters.

The simplest regular expression is just the string being searched for. For example:

   grep Taz *.txt

-- finds every example of the string "Taz" in all files ending in ".txt", then displays the name of the file and the line of text containing the string.

But using the magic characters provides much more flexibility. For example:

   grep ^Taz *.txt

-- finds the string "Taz" only if it is at the beginning of the line. Similarly:

   grep Taz$ *.txt

-- matches it only if it is at the end of the line.

Now suppose we want to match both "Taz" and "taz". This can be done with:

  [Tt]az

The square brackets ("[]") can be used to specify a range of characters. For example:

   group_[abcdef]

-- matches the strings "group_a", "group_b", and so on up to "group_f". This range specification can be simplified to:

   group_[a-f]

Similarly:

   set[0123456789]

-- can be simplified to:

   set[0-9]

It is also possible to match to all characters except a specific range. For example:

   unit_[^xyz]

-- matches "unit_a" or "unit_b", but not "unit_x" or "unit_y" or "unit_z".

Other magic characters provide a wildcard capability. The "." character can substitute for any single character, while the "*" substitutes for zero or more repetitions of the preceding regular expression. For example:

   __*$

-- matches any line that is padded with spaces to the right margin (for clarity the space character is represented here by a "_"). If a magic character is to be matched as a real item of text, it has to be "escaped" with a "\":

   test\.txt

This matches "test.txt".

BACK_TO_TOP

[9] GREP, SED, & AWK

* Now that we understand regular expressions, we can consider "grep", "sed", and "awk" in more detail.

The name of "grep" seems distinctly non-user-friendly, but it can be regarded as standing for "general regular expression processor"; as noted it searches a file for matches to a regular expression like "^Taz" or "_*$". It has a few useful options as well. For example:

   grep -v <regular_expression> <file_list>

-- lists all lines that don't match the regular expression. Other options include:

   grep -n      # List line numbers of matches.
   grep -i      # Ignore case.
   grep -l      # Only list file names for a match.

If there's no need to go through the bother of using regular expressions in a particular search, there is a variation on "grep" called "fgrep" (meaning "fixed grep" or "fast grep") that searches for matches on strings and runs faster; it was used in an earlier example. It uses the same options as described for "grep" above.

* The name "sed" stands for "stream editor" and it provides, in general, a search-and-replace capability. Its syntax for this task is as follows:

   sed 's/<regular_expression>/<replacement_string>/[g]' source.txt

The optional "g" parameter specifies a "global" replacement. That is, if there are have multiple matches on the same line, "sed" will replace them all. Without the "g" option, it will only replace the first match on that line. For example, "sed" can be used to replace the string "flack" with "flak" as follows:

   sed 's/flack/flak/g' source.txt > output.txt

It can also delete strings:

   sed 's/bozo//'

-- or perform substitutions and deletions from a list of such specifications stored in a file:

   sed -f sedcmds.txt source.txt > output.txt

Another useful feature stops output on a pattern match:

   sed '/^Target/q' source.txt > output.txt

It is also possible to append a file to the output after a pattern match:

   sed '/^Target/ r newtext.txt' source.txt > output.txt

The "sed" utility has a wide variety of other options, but a full discussion of its capabilities is beyond the scope of this document.

* Finally, "awk" is a full-blown text processing language that looks something like a mad cross between "grep" and "C". In operation, "awk" takes each line of input and performs text processing on it. It recognizes the current line as "$0", with each word in the line recognized as "$1", "$2", "$3", and so on. This means that:

   awk '{ print $0,$0 }' source.txt

-- prints each line with duplicate text. A regular expression can be specified to identify a pattern match. For example, "awk" could tally the lines with the word "Taz" on them with:

   awk '/Taz/  { taz++ }; END { print taz }' source.txt

The END clause used in this example allows execution of "awk" statements after the line-scanning has been completed. There is also a BEGIN clause that allows execution of "awk" statements before line-scanning begins. "Awk" can be used to do very simple or very complicated things. Its syntax is much like that of "C", though it is much less finicky to deal with. Details of "awk" are discussed in a companion document.

BACK_TO_TOP

[10] SHELL PROGRAM EXAMPLES

* The most elementary use of shell programs is to reduce complicated command strings to simpler commands and to provide handy utilities. For example, if we always use the same options for compiling an ANSI C program, it would be simple to just store them in a script program named, say, "compile":

   cc $1.c -Aa -o $1

Similarly, if we like to timestamp documents in a particular format, we could have a shell program named "td" ("timedate") that invokes "date" as follows:

   date +"date: %A, %d %B %Y %H%M %Z"

This gives, for example:

   date: Friday, 24 November 1995 1340 MST

Another simple example is a shell script to convert file names from uppercase to lowercase:

   for file
   do
     mv $file `echo $file | tr "[A-Z]" "[a-z]"`
   done

In this example, "for" is used to sequence through the file arguments, and of "tr" and back-quoting are used to establish the lower-case name for the file.

BACK_TO_TOP

[11] QUICK REFERENCE

* This final section provides a fast lookup reference for the materials in this document.

* Useful commands:

   cat                  # Lists a file or files sequentially.
   cd                   # Change directories.
   chmod +x             # Set execute permissions.
   chmod 666            # Set universal read-write permissions.
   cp                   # Copy files.
   expr 2 + 2           # Add 2 + 2.
   fgrep                # Search for string match.
   grep                 # Search for string pattern matches.
   grep -v              # Search for no match.
   grep -n              # List line numbers of matches.
   grep -i              # Ignore case.
   grep -l              # Only list file names for a match.
   head -5 source.txt   # List first 5 lines.
   ls                   # Give a simple listing of files.
   mkdir                # Make a directory.
   more                 # Displays a file a screenfull at a time.
   mv                   # Move or rename files.
   paste f1 f2          # Paste files by columns.
   pg                   # Variant on "more".
   pwd                  # Print working directory.
   rm                   # Remove files.  
   rm -r                # Remove entire directory subtree.
   rmdir                # Remove a directory.
   sed 's/txt/TXT/g'    # Scan and replace text.
   sed 's/txt//'        # Scan and delete text.
   sed '/txt/q'         # Scan and then quit.
   sort                 # Sort input.
   sort +1              # Skip first field in sorting. 
   sort -n              # Sort numbers. 
   sort -r              # Sort in reverse order.
   sort -u              # Eliminate redundant lines in output.
   tail -5 source.txt   # List last 5 lines.
   tail +5 source.txt   # List all lines after line 5.
   tr '[A-Z]' '[a-z]'   # Translate to lowercase.
   tr '[a-z]' '[A-Z]'   # Translate to uppercase.
   tr -d '_'            # Delete underscores.
   uniq                 # Find unique lines. 
   wc                   # Word count (characters, words, lines).
   wc -w                # Word count only.
   wc -l                # Line count.

* Elementary shell capabilities:

   shvar="Test 1"       # Initialize a shell variable.
   echo $shvar          # Display a shell variable.
   export shvar         # Allow subshells to use shell variable.
   mv $f ${f}2          # Append "2" to file name in shell variable.
   $1, $2, $3, ...      # Command-line arguments.
   $0                   # Shell-program name.
   $#                   # Number of arguments.
   $*                   # Complete argument list.
   shift 2              # Shift argument variables by 2.
   read v               # Read input into variable "v".
   . mycmds             # Execute commands in file.

* IF statement:

   if [ "$1" = "red" ]
   then
     echo "Illegal code."
     exit
   elif [ "$1" = "blue" ]
   then
     echo "Illegal code."
     exit
   else
     echo "Access granted."
   fi 

   [ "$shvar" = "red" ]     String comparison, true if match.
   [ "$shvar" != "red" ]    String comparison, true if no match.
   [ "$shvar" = "" ]        True if null variable.
   [ "$shvar" != "" ]       True if not null variable.

   [ "$nval" -eq 0 ]        Integer test; true if equal to 0.
   [ "$nval" -ge 0 ]        Integer test; true if greater than or equal to 0.
   [ "$nval" -gt 0 ]        Integer test; true if greater than 0.
   [ "$nval" -le 0 ]        Integer test; true if less than or equal to 0.
   [ "$nval" -lt 0 ]        Integer test; true if less than to 0.
   [ "$nval" -ne 0 ]        Integer test; true if not equal to 0.

   [ -d tmp ]               True if "tmp" is a directory.
   [ -f tmp ]               True if "tmp" is an ordinary file.
   [ -r tmp ]               True if "tmp" can be read.
   [ -s tmp ]               True if "tmp" is nonzero length.
   [ -w tmp ]               True if "tmp" can be written.
   [ -x tmp ]               True if "tmp" is executable.

* CASE statement:

   case "$1" 
   in
     "red")      echo "Illegal code."
                 exit;;
     "blue")     echo "Illegal code."
                 exit;;
     *)          echo "Access granted.";;
   esac

* Loop statements:

   for nvar in 1 2 3 4 5
   do
     echo $nvar
   done 

   for file            # Cycle through command-line arguments.
   do
     echo $file
   done

   while [ "$n" != "Joe" ]     # Or:   until [ "$n" = "Joe" ]
   do
     echo "What's your name?"
     read n
     echo $n
   done

There are "break" and "continue" commands that exit or skip to the end of loops as the need arises.

BACK_TO_TOP

[12] REVISION HISTORY

* This document was originally written during the 1990s, but I yanked it in 2001 since it didn't seem to be attracting much attention. In the spring of 2003 I retrieved it and put it back up, since I realized it offered some value and it made no sense just to keep it archived, gathering dust.

Unfortunately, by that time I had lost track of its revision history. For want of anything better to do, I simply gave the resurrected document the initial revcode of "v1.0.0". I believe it is unlikely that any earlier versions of this document are available on the Internet, but since I had switched from a two-digit revcode format ("v1.0") to a three-digit format ("v1.0.0") in the interim, any earlier copies will have a two-digit revcode.

* One last thing to keep the copyright cops happy: UNIX was originally a trademark of AT&T Bell Labs; the trademark has changed hands a number of times and is now, as best I can tell, a trademark of the Open Group. Once upon a time the trademark issue was so stuffy that most folks wrote UNIX as "UN*X" just to avoid trouble, but those days are over.

* Revision history:

   v1.0.0 / 01 may 03 / Posted as INTRODUCTION TO SHELL PROGRAMMING.
   v1.0.1 / 01 nov 04 / Minor corrections.
   v1.0.2 / 01 mar 06 / Minor fixes.
   v1.0.3 / 01 feb 08 / Minor fixes.
   v1.0.4 / 01 may 09 / General cleanup, changed to PRIMER.
   v1.0.5 / 01 jun 09 / Yet more little tweaks.
   v1.0.6 / 01 apr 11 / Review & polish.
   v1.0.7 / 01 feb 13 / Review & polish.
BACK_TO_TOP
INDEX | SITEMAP | GOOGLE | LINKS | UPDATES | BLOG | EMAIL | DONATE | HOME