Pantek Expert IT Services Software Library
Hosting provided by:
CybrHost

Visit our blog, Open Sores: Tips From the Trenches

 Enterprise Linux Hosting
Miscellaneous: The Linux Documentation Project > LDP > abs > htmlabs-guide.html (Request Expert The Linux Documentation Project Support)

Advanced Bash-Scripting Guide

An in-depth exploration of the art of shell scripting

Mendel Cooper


6.1

30 September 2009

Revision History
Revision 5.626 Jan 2009Revised by: mc
'WORCESTERBERRY' release: Minor Update.
Revision 6.023 Mar 2009Revised by: mc
'THIMBLEBERRY' release: Major Update.
Revision 6.130 Sep 2009Revised by: mc
'BUFFALOBERRY' release: Minor Update.

This tutorial assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction . . . all the while sneaking in little nuggets of UNIX® wisdom and lore. It serves as a textbook, a manual for self-study, and a reference and source of knowledge on shell scripting techniques. The exercises and heavily-commented examples invite active reader participation, under the premise that the only way to really learn scripting is to write scripts.

This book is suitable for classroom use as a general introduction to programming concepts.

Dedication

For Anita, the source of all the magic

Table of Contents
Part 1. Introduction
1. Shell Programming!
2. Starting Off With a Sha-Bang
Part 2. Basics
3. Special Characters
4. Introduction to Variables and Parameters
5. Quoting
6. Exit and Exit Status
7. Tests
8. Operations and Related Topics
Part 3. Beyond the Basics
9. Variables Revisited
10. Loops and Branches
11. Command Substitution
12. Arithmetic Expansion
13. Recess Time
Part 4. Commands
14. Internal Commands and Builtins
15. External Filters, Programs and Commands
16. System and Administrative Commands
Part 5. Advanced Topics
17. Regular Expressions
18. Here Documents
19. I/O Redirection
20. Subshells
21. Restricted Shells
22. Process Substitution
23. Functions
24. Aliases
25. List Constructs
26. Arrays
27. /dev and /proc
28. Of Zeros and Nulls
29. Debugging
30. Options
31. Gotchas
32. Scripting With Style
33. Miscellany
34. Bash, versions 2, 3, and 4
35. Endnotes
35.1. Author's Note
35.2. About the Author
35.3. Where to Go For Help
35.4. Tools Used to Produce This Book
35.5. Credits
35.6. Disclaimer
Bibliography
A. Contributed Scripts
B. Reference Cards
C. A Sed and Awk Micro-Primer
C.1. Sed
C.2. Awk
D. Exit Codes With Special Meanings
E. A Detailed Introduction to I/O and I/O Redirection
F. Command-Line Options
F.1. Standard Command-Line Options
F.2. Bash Command-Line Options
G. Important Files
H. Important System Directories
I. An Introduction to Programmable Completion
J. Localization
K. History Commands
L. A Sample .bashrc File
M. Converting DOS Batch Files to Shell Scripts
N. Exercises
N.1. Analyzing Scripts
N.2. Writing Scripts
O. Revision History
P. Download and Mirror Sites
Q. To Do List
R. Copyright
S. ASCII Table
Index
List of Tables
14-1. Job identifiers
30-1. Bash options
33-1. Operator Precedence
33-2. Numbers representing colors in Escape Sequences
B-1. Special Shell Variables
B-2. TEST Operators: Binary Comparison
B-3. TEST Operators: Files
B-4. Parameter Substitution and Expansion
B-5. String Operations
B-6. Miscellaneous Constructs
C-1. Basic sed operators
C-2. Examples of sed operators
D-1. Reserved Exit Codes
M-1. Batch file keywords / variables / operators, and their shell equivalents
M-2. DOS commands and their UNIX equivalents
O-1. Revision History
List of Examples
2-1. cleanup: A script to clean up the log files in /var/log
2-2. cleanup: An improved clean-up script
2-3. cleanup: An enhanced and generalized version of above scripts.
3-1. Code blocks and I/O redirection
3-2. Saving the output of a code block to a file
3-3. Running a loop in the background
3-4. Backup of all files changed in last day
4-1. Variable assignment and substitution
4-2. Plain Variable Assignment
4-3. Variable Assignment, plain and fancy
4-4. Integer or string?
4-5. Positional Parameters
4-6. wh, whois domain name lookup
4-7. Using shift
5-1. Echoing Weird Variables
5-2. Escaped Characters
6-1. exit / exit status
6-2. Negating a condition using !
7-1. What is truth?
7-2. Equivalence of test, /usr/bin/test, [ ], and /usr/bin/[
7-3. Arithmetic Tests using (( ))
7-4. Testing for broken links
7-5. Arithmetic and string comparisons
7-6. Testing whether a string is null
7-7. zmore
8-1. Greatest common divisor
8-2. Using Arithmetic Operations
8-3. Compound Condition Tests Using && and ||
8-4. Representation of numerical constants
9-1. $IFS and whitespace
9-2. Timed Input
9-3. Once more, timed input
9-4. Timed read
9-5. Am I root?
9-6. arglist: Listing arguments with $* and $@
9-7. Inconsistent $* and $@ behavior
9-8. $* and $@ when $IFS is empty
9-9. Underscore variable
9-10. Inserting a blank line between paragraphs in a text file
9-11. Generating an 8-character "random" string
9-12. Converting graphic file formats, with filename change
9-13. Converting streaming audio files to ogg
9-14. Emulating getopt
9-15. Alternate ways of extracting and locating substrings
9-16. Using parameter substitution and error messages
9-17. Parameter substitution and "usage" messages
9-18. Length of a variable
9-19. Pattern matching in parameter substitution
9-20. Renaming file extensions:
9-21. Using pattern matching to parse arbitrary strings
9-22. Matching patterns at prefix or suffix of string
9-23. Using declare to type variables
9-24. Indirect Variable References
9-25. Passing an indirect reference to awk
9-26. Generating random numbers
9-27. Picking a random card from a deck
9-28. Brownian Motion Simulation
9-29. Random between values
9-30. Rolling a single die with RANDOM
9-31. Reseeding RANDOM
9-32. Pseudorandom numbers, using awk
9-33. C-style manipulation of variables
10-1. Simple for loops
10-2. for loop with two parameters in each [list] element
10-3. Fileinfo: operating on a file list contained in a variable
10-4. Operating on files with a for loop
10-5. Missing in [list] in a for loop
10-6. Generating the [list] in a for loop with command substitution
10-7. A grep replacement for binary files
10-8. Listing all users on the system
10-9. Checking all the binaries in a directory for authorship
10-10. Listing the symbolic links in a directory
10-11. Symbolic links in a directory, saved to a file
10-12. A C-style for loop
10-13. Using efax in batch mode
10-14. Simple while loop
10-15. Another while loop
10-16. while loop with multiple conditions
10-17. C-style syntax in a while loop
10-18. until loop
10-19. Nested Loop
10-20. Effects of break and continue in a loop
10-21. Breaking out of multiple loop levels
10-22. Continuing at a higher loop level
10-23. Using continue N in an actual task
10-24. Using case
10-25. Creating menus using case
10-26. Using command substitution to generate the case variable
10-27. Simple string matching
10-28. Checking for alphabetic input
10-29. Creating menus using select
10-30. Creating menus using select in a function
11-1. Stupid script tricks
11-2. Generating a variable from a loop
11-3. Finding anagrams
14-1. A script that spawns multiple instances of itself
14-2. printf in action
14-3. Variable assignment, using read
14-4. What happens when read has no variable
14-5. Multi-line input to read
14-6. Detecting the arrow keys
14-7. Using read with file redirection
14-8. Problems reading from a pipe
14-9. Changing the current working directory
14-10. Letting let do arithmetic.
14-11. Showing the effect of eval
14-12. Using eval to select among variables
14-13. Echoing the command-line parameters
14-14. Forcing a log-off
14-15. A version of rot13
14-16. Using set with positional parameters
14-17. Reversing the positional parameters
14-18. Reassigning the positional parameters
14-19. "Unsetting" a variable
14-20. Using export to pass a variable to an embedded awk script
14-21. Using getopts to read the options/arguments passed to a script
14-22. "Including" a data file
14-23. A (useless) script that sources itself
14-24. Effects of exec
14-25. A script that exec's itself
14-26. Waiting for a process to finish before proceeding
14-27. A script that kills itself
15-1. Using ls to create a table of contents for burning a CDR disk
15-2. Hello or Good-bye
15-3. Badname, eliminate file names in current directory containing bad characters and whitespace.
15-4. Deleting a file by its inode number
15-5. Logfile: Using xargs to monitor system log
15-6. Copying files in current directory to another
15-7. Killing processes by name
15-8. Word frequency analysis using xargs
15-9. Using expr
15-10. Using date
15-11. Date calculations
15-12. Word Frequency Analysis
15-13. Which files are scripts?
15-14. Generating 10-digit random numbers
15-15. Using tail to monitor the system log
15-16. Printing out the From lines in stored e-mail messages
15-17. Emulating grep in a script
15-18. Crossword puzzle solver
15-19. Looking up definitions in Webster's 1913 Dictionary
15-20. Checking words in a list for validity
15-21. toupper: Transforms a file to all uppercase.
15-22. lowercase: Changes all filenames in working directory to lowercase.
15-23. du: DOS to UNIX text file conversion.
15-24. rot13: ultra-weak encryption.
15-25. Generating "Crypto-Quote" Puzzles
15-26. Formatted file listing.
15-27. Using column to format a directory listing
15-28. nl: A self-numbering script.
15-29. manview: Viewing formatted manpages
15-30. Using cpio to move a directory tree
15-31. Unpacking an rpm archive
15-32. Stripping comments from C program files
15-33. Exploring /usr/X11R6/bin
15-34. An "improved" strings command
15-35. Using cmp to compare two files within a script.
15-36. basename and dirname
15-37. A script that copies itself in sections
15-38. Checking file integrity
15-39. Uudecoding encoded files
15-40. Finding out where to report a spammer
15-41. Analyzing a spam domain
15-42. Getting a stock quote
15-43. Updating FC4
15-44. Using ssh
15-45. A script that mails itself
15-46. Generating prime numbers
15-47. Monthly Payment on a Mortgage
15-48. Base Conversion
15-49. Invoking bc using a here document
15-50. Calculating PI
15-51. Converting a decimal number to hexadecimal
15-52. Factoring
15-53. Calculating the hypotenuse of a triangle
15-54. Using seq to generate loop arguments
15-55. Letter Count"
15-56. Using getopt to parse command-line options
15-57. A script that copies itself
15-58. Exercising dd
15-59. Capturing Keystrokes
15-60. Securely deleting a file
15-61. Filename generator
15-62. Converting meters to miles
15-63. Using m4
16-1. Setting a new password
16-2. Setting an erase character
16-3. secret password: Turning off terminal echoing
16-4. Keypress detection
16-5. Checking a remote server for identd
16-6. pidof helps kill a process
16-7. Checking a CD image
16-8. Creating a filesystem in a file
16-9. Adding a new hard drive
16-10. Using umask to hide an output file from prying eyes
16-11. killall, from /etc/rc.d/init.d
18-1. broadcast: Sends message to everyone logged in
18-2. dummyfile: Creates a 2-line dummy file
18-3. Multi-line message using cat
18-4. Multi-line message, with tabs suppressed
18-5. Here document with replaceable parameters
18-6. Upload a file pair to Sunsite incoming directory
18-7. Parameter substitution turned off
18-8. A script that generates another script
18-9. Here documents and functions
18-10. "Anonymous" Here Document
18-11. Commenting out a block of code
18-12. A self-documenting script
18-13. Prepending a line to a file
18-14. Parsing a mailbox
19-1. Redirecting stdin using exec
19-2. Redirecting stdout using exec
19-3. Redirecting both stdin and stdout in the same script with exec
19-4. Avoiding a subshell
19-5. Redirected while loop
19-6. Alternate form of redirected while loop
19-7. Redirected until loop
19-8. Redirected for loop
19-9. Redirected for loop (both stdin and stdout redirected)
19-10. Redirected if/then test
19-11. Data file names.data for above examples
19-12. Logging events
20-1. Variable scope in a subshell
20-2. List User Profiles
20-3. Running parallel processes in subshells
21-1. Running a script in restricted mode
22-1. Code block redirection without forking
23-1. Simple functions
23-2. Function Taking Parameters
23-3. Functions and command-line args passed to the script
23-4. Passing an indirect reference to a function
23-5. Dereferencing a parameter passed to a function
23-6. Again, dereferencing a parameter passed to a function
23-7. Maximum of two numbers
23-8. Converting numbers to Roman numerals
23-9. Testing large return values in a function
23-10. Comparing two large integers
23-11. Real name from username
23-12. Local variable visibility
23-13. Demonstration of a simple recursive function
23-14. Another simple demonstration
23-15. Recursion, using a local variable
23-16. The Fibonacci Sequence
23-17. The Towers of Hanoi
24-1. Aliases within a script
24-2. unalias: Setting and unsetting an alias
25-1. Using an and list to test for command-line arguments
25-2. Another command-line arg test using an and list
25-3. Using or lists in combination with an and list
26-1. Simple array usage
26-2. Formatting a poem
26-3. Various array operations
26-4. String operations on arrays
26-5. Loading the contents of a script into an array
26-6. Some special properties of arrays
26-7. Of empty arrays and empty elements
26-8. Initializing arrays
26-9. Copying and concatenating arrays
26-10. More on concatenating arrays
26-11. The Bubble Sort
26-12. Embedded arrays and indirect references
26-13. The Sieve of Eratosthenes
26-14. The Sieve of Eratosthenes, Optimized
26-15. Emulating a push-down stack
26-16. Complex array application: Exploring a weird mathematical series
26-17. Simulating a two-dimensional array, then tilting it
27-1. Using /dev/tcp for troubleshooting
27-2. Playing music
27-3. Finding the process associated with a PID
27-4. On-line connect status
28-1. Hiding the cookie jar
28-2. Setting up a swapfile using /dev/zero
28-3. Creating a ramdisk
29-1. A buggy script
29-2. Missing keyword
29-3. test24: another buggy script
29-4. Testing a condition with an assert
29-5. Trapping at exit
29-6. Cleaning up after Control-C
29-7. Tracing a variable
29-8. Running multiple processes (on an SMP box)
31-1. Numerical and string comparison are not equivalent
31-2. Subshell Pitfalls
31-3. Piping the output of echo to a read
33-1. shell wrapper
33-2. A slightly more complex shell wrapper
33-3. A generic shell wrapper that writes to a logfile
33-4. A shell wrapper around an awk script
33-5. A shell wrapper around another awk script
33-6. Perl embedded in a Bash script
33-7. Bash and Perl scripts combined
33-8. A (useless) script that recursively calls itself
33-9. A (useful) script that recursively calls itself
33-10. Another (useful) script that recursively calls itself
33-11. A "colorized" address database
33-12. Drawing a box
33-13. Echoing colored text
33-14. A "horserace" game
33-15. A Progress Bar
33-16. Return value trickery
33-17. Even more return value trickery
33-18. Passing and returning arrays
33-19. Fun with anagrams
33-20. Widgets invoked from a shell script
33-21. Test Suite
34-1. String expansion
34-2. Indirect variable references - the new way
34-3. Simple database application, using indirect variable referencing
34-4. Using arrays and other miscellaneous trickery to deal four random hands from a deck of cards
34-5. A simple address database
34-6. A somewhat more elaborate address database
34-7. Testing characters
A-1. mailformat: Formatting an e-mail message
A-2. rn: A simple-minded file renaming utility
A-3. blank-rename: Renames filenames containing blanks
A-4. encryptedpw: Uploading to an ftp site, using a locally encrypted password
A-5. copy-cd: Copying a data CD
A-6. Collatz series
A-7. days-between: Days between two dates
A-8. Making a dictionary
A-9. Soundex conversion
A-10. Game of Life
A-11. Data file for Game of Life
A-12. behead: Removing mail and news message headers
A-13. password: Generating random 8-character passwords
A-14. fifo: Making daily backups, using named pipes
A-15. Generating prime numbers using the modulo operator
A-16. tree: Displaying a directory tree
A-17. tree2: Alternate directory tree script
A-18. string functions: C-style string functions
A-19. Directory information
A-20. Library of hash functions
A-21. Colorizing text using hash functions
A-22. More on hash functions
A-23. Mounting USB keychain storage devices
A-24. Converting to HTML
A-25. Preserving weblogs
A-26. Protecting literal strings
A-27. Unprotecting literal strings
A-28. Spammer Identification
A-29. Spammer Hunt
A-30. Making wget easier to use
A-31. A podcasting script
A-32. Nightly backup to a firewire HD
A-33. An expanded cd command
A-34. A soundcard setup script
A-35. Locating split paragraphs in a text file
A-36. Insertion sort
A-37. Standard Deviation
A-38. A pad file generator for shareware authors
A-39. A man page editor
A-40. Petals Around the Rose
A-41. Quacky: a Perquackey-type word game
A-42. Nim
A-43. A command-line stopwatch
A-44. An all-purpose shell scripting homework assignment solution
A-45. The Knight's Tour
A-46. Magic Squares
A-47. Fifteen Puzzle
A-48. The Towers of Hanoi, graphic version
A-49. The Towers of Hanoi, alternate graphic version
A-50. An alternate version of the getopt-simple.sh script
A-51. The version of the UseGetOpt.sh example used in the Tab Expansion appendix
A-52. Cycling through all the possible color backgrounds
A-53. Basics Reviewed
C-1. Counting Letter Occurrences
I-1. Completion script for UseGetOpt.sh
L-1. Sample .bashrc file
M-1. VIEWDATA.BAT: DOS Batch File
M-2. viewdata.sh: Shell Script Conversion of VIEWDATA.BAT
Q-1. Print the server environment
S-1. A script that generates an ASCII table

Part 1. Introduction

 

Script: A writing; a written document. [Obs.]

Do you need help?X

--Webster's Dictionary, 1913 ed.

The shell is a command interpreter. More than just the insulating layer between the operating system kernel and the user, it's also a fairly powerful programming language. A shell program, called a script, is an easy-to-use tool for building applications by "gluing together" system calls, tools, utilities, and compiled binaries. Virtually the entire repertoire of UNIX commands, utilities, and tools is available for invocation by a shell script. If that were not enough, internal shell commands, such as testing and loop constructs, lend additional power and flexibility to scripts. Shell scripts are especially well suited for administrative system tasks and other routine repetitive tasks not requiring the bells and whistles of a full-blown tightly structured programming language.

Table of Contents
1. Shell Programming!
2. Starting Off With a Sha-Bang
2.1. Invoking the script
2.2. Preliminary Exercises

Chapter 1. Shell Programming!

 

No programming language is perfect. There is not even a single best language; there are only languages well suited or perhaps poorly suited for particular purposes.

--Herbert Mayer

A working knowledge of shell scripting is essential to anyone wishing to become reasonably proficient at system administration, even if they do not anticipate ever having to actually write a script. Consider that as a Linux machine boots up, it executes the shell scripts in /etc/rc.d to restore the system configuration and set up services. A detailed understanding of these startup scripts is important for analyzing the behavior of a system, and possibly modifying it.

The craft of scripting is not hard to master, since the scripts can be built in bite-sized sections and there is only a fairly small set of shell-specific operators and options [1] to learn. The syntax is simple and straightforward, similar to that of invoking and chaining together utilities at the command line, and there are only a few "rules" governing their use. Most short scripts work right the first time, and debugging even the longer ones is straightforward.

    
       In the 1970s, the BASIC language enabled anyone reasonably computer proficient
       to write programs on an early generation of microcomputers. Decades later, the Bash
       scripting language enables anyone with a rudimentary knowledge of Linux or UNIX to do the same
       on much more powerful machines.
            

Do you need more help?X

A shell script is a quick-and-dirty method of prototyping a complex application. Getting even a limited subset of the functionality to work in a script is often a useful first stage in project development. This way, the structure of the application can be tested and played with, and the major pitfalls found before proceeding to the final coding in C, C++, Java, Perl, or Python.

Shell scripting hearkens back to the classic UNIX philosophy of breaking complex projects into simpler subtasks, of chaining together components and utilities. Many consider this a better, or at least more esthetically pleasing approach to problem solving than using one of the new generation of high powered all-in-one languages, such as Perl, which attempt to be all things to all people, but at the cost of forcing you to alter your thinking processes to fit the tool.

According to Herbert Mayer, "a useful language needs arrays, pointers, and a generic mechanism for building data structures." By these criteria, shell scripting falls somewhat short of being "useful." Or, perhaps not. . . .

When not to use shell scripts

  • Resource-intensive tasks, especially where speed is a factor (sorting, hashing, recursion [2] ...)

    Can we help you?X

  • Procedures involving heavy-duty math operations, especially floating point arithmetic, arbitrary precision calculations, or complex numbers (use C++ or FORTRAN instead)

  • Cross-platform portability required (use C or Java instead)

  • Complex applications, where structured programming is a necessity (type-checking of variables, function prototypes, etc.)

  • Mission-critical applications upon which you are betting the future of the company

  • Situations where security is important, where you need to guarantee the integrity of your system and protect against intrusion, cracking, and vandalism

  • Project consists of subcomponents with interlocking dependencies

  • Extensive file operations required (Bash is limited to serial file access, and that only in a particularly clumsy and inefficient line-by-line fashion.)

  • Need native support for multi-dimensional arrays

    Can't find what you're looking for?X

  • Need data structures, such as linked lists or trees

  • Need to generate / manipulate graphics or GUIs

  • Need direct access to system hardware

  • Need port or socket I/O

  • Need to use libraries or interface with legacy code

  • Proprietary, closed-source applications (Shell scripts put the source code right out in the open for all the world to see.)

If any of the above applies, consider a more powerful scripting language -- perhaps Perl, Tcl, Python, Ruby -- or possibly a compiled language such as C, C++, or Java. Even then, prototyping the application as a shell script might still be a useful development step.

Don't know where to look next?X

We will be using Bash, an acronym for "Bourne-Again shell" and a pun on Stephen Bourne's now classic Bourne shell. Bash has become a de facto standard for shell scripting on most flavors of UNIX. Most of the principles this book covers apply equally well to scripting with other shells, such as the Korn Shell, from which Bash derives some of its features, [3] and the C Shell and its variants. (Note that C Shell programming is not recommended due to certain inherent problems, as pointed out in an October, 1993 Usenet post by Tom Christiansen.)

What follows is a tutorial on shell scripting. It relies heavily on examples to illustrate various features of the shell. The example scripts work -- they've been tested, insofar as was possible -- and some of them are even useful in real life. The reader can play with the actual working code of the examples in the source archive (scriptname.sh or scriptname.bash), [4] give them execute permission (chmod u+rx scriptname), then run them to see what happens. Should the source archive not be available, then cut-and-paste from the HTML or pdf rendered versions. Be aware that some of the scripts presented here introduce features before they are explained, and this may require the reader to temporarily skip ahead for enlightenment.

Unless otherwise noted, the author of this book wrote the example scripts that follow.
 

His countenance was bold and bashed not.

--Edmund Spenser

Chapter 2. Starting Off With a Sha-Bang

 

Shell programming is a 1950s juke box . . .

--Larry Wall

Confused? Frustrated?X

In the simplest case, a script is nothing more than a list of system commands stored in a file. At the very least, this saves the effort of retyping that particular sequence of commands each time it is invoked.

Example 2-1. cleanup: A script to clean up the log files in /var/log 
# Cleanup
# Run as root, of course.

cd /var/log
cat /dev/null > messages
cat /dev/null > wtmp
echo "Logs cleaned up."

There is nothing unusual here, only a set of commands that could just as easily have been invoked one by one from the command-line on the console or in a terminal window. The advantages of placing the commands in a script go far beyond not having to retype them time and again. The script becomes a program -- a tool -- and it can easily be modified or customized for a particular application.

Example 2-2. cleanup: An improved clean-up script
#!/bin/bash
# Proper header for a Bash script.

# Cleanup, version 2

# Run as root, of course.
# Insert code here to print error message and exit if not root.

LOG_DIR=/var/log
# Variables are better than hard-coded values.
cd $LOG_DIR

cat /dev/null > messages
cat /dev/null > wtmp


echo "Logs cleaned up."

exit # The right and proper method of "exiting" from a script.

Now that's beginning to look like a real script. But we can go even farther . . .

Example 2-3. cleanup: An enhanced and generalized version of above scripts.
#!/bin/bash
# Cleanup, version 3

#  Warning:
#  -------
#  This script uses quite a number of features that will be explained
#+ later on.
#  By the time you've finished the first half of the book,
#+ there should be nothing mysterious about it.



LOG_DIR=/var/log
ROOT_UID=0     # Only users with $UID 0 have root privileges.
LINES=50       # Default number of lines saved.
E_XCD=86       # Can't change directory?
E_NOTROOT=87   # Non-root exit error.


# Run as root, of course.
if [ "$UID" -ne "$ROOT_UID" ]
then
  echo "Must be root to run this script."
  exit $E_NOTROOT
fi  

if [ -n "$1" ]
# Test whether command-line argument is present (non-empty).
then
  lines=$1
else  
  lines=$LINES # Default, if not specified on command-line.
fi  


#  Stephane Chazelas suggests the following,
#+ as a better way of checking command-line arguments,
#+ but this is still a bit advanced for this stage of the tutorial.
#
#    E_WRONGARGS=85  # Non-numerical argument (bad argument format).
#
#    case "$1" in
#    ""      ) lines=50;;
#    *[!0-9]*) echo "Usage: `basename $0` file-to-cleanup"; exit $E_WRONGARGS;;
#    *       ) lines=$1;;
#    esac
#
#* Skip ahead to "Loops" chapter to decipher all this.


cd $LOG_DIR

if [ `pwd` != "$LOG_DIR" ]  # or   if [ "$PWD" != "$LOG_DIR" ]
                            # Not in /var/log?

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

then
  echo "Can't change to $LOG_DIR."
  exit $E_XCD
fi  # Doublecheck if in right directory before messing with log file.

# Far more efficient is:
#
# cd /var/log || {
#   echo "Cannot change to necessary directory." >&2
#   exit $E_XCD;
# }




tail -n $lines messages > mesg.temp # Save last section of message log file.
mv mesg.temp messages               # Becomes new log directory.


#  cat /dev/null > messages
#* No longer needed, as the above method is safer.

cat /dev/null > wtmp  #  ': > wtmp' and '> wtmp'  have the same effect.
echo "Logs cleaned up."

exit 0
#  A zero return value from the script upon exit indicates success
#+ to the shell.

Since you may not wish to wipe out the entire system log, this version of the script keeps the last section of the message log intact. You will constantly discover ways of fine-tuning previously written scripts for increased effectiveness.

* * *

Do you need help?X

The sha-bang ( #!) [5] at the head of a script tells your system that this file is a set of commands to be fed to the command interpreter indicated. The #! is actually a two-byte [6] magic number, a special marker that designates a file type, or in this case an executable shell script (type man magic for more details on this fascinating topic). Immediately following the sha-bang is a path name. This is the path to the program that interprets the commands in the script, whether it be a shell, a programming language, or a utility. This command interpreter then executes the commands in the script, starting at the top (the line following the sha-bang line), and ignoring comments. [7]

#!/bin/sh
#!/bin/bash
#!/usr/bin/perl
#!/usr/bin/tcl
#!/bin/sed -f
#!/usr/awk -f

Each of the above script header lines calls a different command interpreter, be it /bin/sh, the default shell (bash in a Linux system) or otherwise. [8] Using #!/bin/sh, the default Bourne shell in most commercial variants of UNIX, makes the script portable to non-Linux machines, though you sacrifice Bash-specific features. The script will, however, conform to the POSIX [9] sh standard.

Note that the path given at the "sha-bang" must be correct, otherwise an error message -- usually "Command not found." -- will be the only result of running the script. [10]

#! can be omitted if the script consists only of a set of generic system commands, using no internal shell directives. The second example, above, requires the initial #!, since the variable assignment line, lines=50, uses a shell-specific construct. [11] Note again that #!/bin/sh invokes the default shell interpreter, which defaults to /bin/bash on a Linux machine.

Tip

This tutorial encourages a modular approach to constructing a script. Make note of and collect "boilerplate" code snippets that might be useful in future scripts. Eventually you will build quite an extensive library of nifty routines. As an example, the following script prolog tests whether the script has been invoked with the correct number of parameters.

E_WRONG_ARGS=85
script_parameters="-a -h -m -z"
#                  -a = all, -h = help, etc.

if [ $# -ne $Number_of_expected_args ]
then
  echo "Usage: `basename $0` $script_parameters"
  # `basename $0` is the script's filename.
  exit $E_WRONG_ARGS
fi

Do you need more help?X

Many times, you will write a script that carries out one particular task. The first script in this chapter is an example. Later, it might occur to you to generalize the script to do other, similar tasks. Replacing the literal ("hard-wired") constants by variables is a step in that direction, as is replacing repetitive code blocks by functions.

2.1. Invoking the script

Having written the script, you can invoke it by sh scriptname, [12] or alternatively bash scriptname. (Not recommended is using sh <scriptname, since this effectively disables reading from stdin within the script.) Much more convenient is to make the script itself directly executable with a chmod.

Either:

chmod 555 scriptname (gives everyone read/execute permission) [13]

or

chmod +rx scriptname (gives everyone read/execute permission)

chmod u+rx scriptname (gives only the script owner read/execute permission)

Having made the script executable, you may now test it by ./scriptname. [14] If it begins with a "sha-bang" line, invoking the script calls the correct command interpreter to run it.

As a final step, after testing and debugging, you would likely want to move it to /usr/local/bin (as root, of course), to make the script available to yourself and all other users as a systemwide executable. The script could then be invoked by simply typing scriptname [ENTER] from the command-line.

Can we help you?X

2.2. Preliminary Exercises

  1. System administrators often write scripts to automate common tasks. Give several instances where such scripts would be useful.

  2. Write a script that upon invocation shows the time and date, lists all logged-in users, and gives the system uptime. The script then saves this information to a logfile.

Part 2. Basics

Table of Contents
3. Special Characters
4. Introduction to Variables and Parameters
4.1. Variable Substitution
4.2. Variable Assignment
4.3. Bash Variables Are Untyped
4.4. Special Variable Types
5. Quoting
5.1. Quoting Variables
5.2. Escaping
6. Exit and Exit Status
7. Tests
7.1. Test Constructs
7.2. File test operators
7.3. Other Comparison Operators
7.4. Nested if/then Condition Tests
7.5. Testing Your Knowledge of Tests
8. Operations and Related Topics
8.1. Operators
8.2. Numerical Constants

Chapter 3. Special Characters

What makes a character special? If it has a meaning beyond its literal meaning, a meta-meaning, then we refer to it as a special character.

Special Characters Found In Scripts and Elsewhere

#

Comments. Lines beginning with a # (with the exception of #!) are comments and will not be executed.

# This line is a comment.

Can't find what you're looking for?X

Comments may also occur following the end of a command.

echo "A comment will follow." # Comment here.
#                            ^ Note whitespace before #

Comments may also follow whitespace at the beginning of a line.

     # A tab precedes this comment.

Comments may even be embedded within a pipe.

initial=( `cat "$startfile" | sed -e '/#/d' | tr -d '\n' |\
# Delete lines containing '#' comment character.
           sed -e 's/\./\. /g' -e 's/_/_ /g'` )
# Excerpted from life.sh script

Caution

A command may not follow a comment on the same line. There is no method of terminating the comment, in order for "live code" to begin on the same line. Use a new line for the next command.

Don't know where to look next?X

Note

Of course, a quoted or an escaped # in an echo statement does not begin a comment. Likewise, a # appears in certain parameter-substitution constructs and in numerical constant expressions. 
echo "The # here does not begin a comment."
echo 'The # here does not begin a comment.'
echo The \# here does not begin a comment.
echo The # here begins a comment.

echo ${PATH#*:}       # Parameter substitution, not a comment.
echo $(( 2#101011 ))  # Base conversion, not a comment.

# Thanks, S.C.
The standard quoting and escape characters (" ' \) escape the #.

Certain pattern matching operations also use the #.

;

Command separator [semicolon]. Permits putting two or more commands on the same line.

echo hello; echo there


if [ -x "$filename" ]; then    #  Note the space after the semicolon.
#+                   ^^
  echo "File $filename exists."; cp $filename $filename.bak

Confused? Frustrated?X

else   #                       ^^
  echo "File $filename not found."; touch $filename
fi; echo "File test complete."

Note that the ";" sometimes needs to be escaped.

;;

Terminator in a case option [double semicolon]. 

case "$variable" in
  abc)  echo "\$variable = abc" ;;
  xyz)  echo "\$variable = xyz" ;;
esac

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X
;;&, ;&

Terminators in a case option (version 4+ of Bash).

.

"dot" command [period]. Equivalent to source (see Example 14-22). This is a bash builtin.

.

"dot", as a component of a filename. When working with filenames, a leading dot is the prefix of a "hidden" file, a file that an ls will not normally show. 
bash$ touch .hidden-filebash$ ls -l	      
total 10
 -rw-r--r--    1 bozo      4034 Jul 18 22:04 data1.addressbook
 -rw-r--r--    1 bozo      4602 May 25 13:58 data1.addressbook.bak
 -rw-r--r--    1 bozo       877 Dec 17  2000 employment.addressbookbash$ ls -al	      
total 14
 drwxrwxr-x    2 bozo  bozo      1024 Aug 29 20:54 ./
 drwx------   52 bozo  bozo      3072 Aug 29 20:51 ../
 -rw-r--r--    1 bozo  bozo      4034 Jul 18 22:04 data1.addressbook
 -rw-r--r--    1 bozo  bozo      4602 May 25 13:58 data1.addressbook.bak
 -rw-r--r--    1 bozo  bozo       877 Dec 17  2000 employment.addressbook
 -rw-rw-r--    1 bozo  bozo         0 Aug 29 20:54 .hidden-file

When considering directory names, a single dot represents the current working directory, and two dots denote the parent directory.

bash$ pwd/home/bozo/projectsbash$ cd .bash$ pwd/home/bozo/projectsbash$ cd ..bash$ pwd/home/bozo/

The dot often appears as the destination (directory) of a file movement command, in this context meaning current directory.

Do you need help?X

bash$ cp /home/bozo/current_work/junk/* .
Copy all the "junk" files to $PWD.

.

"dot" character match. When matching characters, as part of a regular expression, a "dot" matches a single character.

"

partial quoting [double quote]. "STRING" preserves (from interpretation) most of the special characters within STRING. See Chapter 5.

'

full quoting [single quote]. 'STRING' preserves all special characters within STRING. This is a stronger form of quoting than "STRING". See Chapter 5.

,

comma operator. The comma operator [15] links together a series of arithmetic operations. All are evaluated, but only the last one is returned. 
let "t2 = ((a = 9, 15 / 3))"  # Set "a =
	       9" and "t2 = 15 / 3"

The comma operator can also concatenate strings. 
for file in /{,usr/}bin/*calc
#             ^    Find all executable files ending in "calc"
#+                 in /bin and /usr/bin directories.
do
        if [ -x "$file" ]
        then
          echo $file
        fi
done

# /bin/ipcalc
# /usr/bin/kcalc
# /usr/bin/oidcalc
# /usr/bin/oocalc


# Thank you, Rory Winston, for pointing this out.

,, ,

Lowercase conversion in parameter substitution (added in version 4 of Bash).

\

escape [backslash]. A quoting mechanism for single characters.

Do you need more help?X

\X escapes the character X. This has the effect of "quoting" X, equivalent to 'X'. The \ may be used to quote " and ', so they are expressed literally.

See Chapter 5 for an in-depth explanation of escaped characters.

/

Filename path separator [forward slash]. Separates the components of a filename (as in /home/bozo/projects/Makefile).

This is also the division arithmetic operator.

`

command substitution. The `command` construct makes available the output of command for assignment to a variable. This is also known as backquotes or backticks.

:

null command [colon]. This is the shell equivalent of a "NOP" (no op, a do-nothing operation). It may be considered a synonym for the shell builtin true. The ":" command is itself a Bash builtin, and its exit status is true (0).

:
echo $?   # 0

Can we help you?X

Endless loop:

while :
do
   operation-1
   operation-2
   ...
   operation-n
done

# Same as:
#    while true
#    do
#      ...
#    done

Placeholder in if/then test:

if condition
then :   # Do nothing and branch ahead
else     # Or else ...
   take-some-action
fi

Provide a placeholder where a binary operation is expected, see Example 8-2 and default parameters.

: ${username=`whoami`}
# ${username=`whoami`}   Gives an error without the leading :
#                        unless "username" is a command or builtin...

Provide a placeholder where a command is expected in a here document. See Example 18-10.

Evaluate string of variables using parameter substitution (as in Example 9-16). 
: ${HOSTNAME?} ${USER?} ${MAIL?}
#  Prints error message
#+ if one or more of essential environmental variables not set.

Can't find what you're looking for?X

Variable expansion / substring replacement.

In combination with the > redirection operator, truncates a file to zero length, without changing its permissions. If the file did not previously exist, creates it. 
: > data.xxx   # File "data.xxx" now empty.	      

# Same effect as   cat /dev/null >data.xxx
# However, this does not fork a new process, since ":" is a builtin.
See also Example 15-15.

In combination with the >> redirection operator, has no effect on a pre-existing target file (: >> target_file). If the file did not previously exist, creates it.

Note

This applies to regular files, not pipes, symlinks, and certain special files.

May be used to begin a comment line, although this is not recommended. Using # for a comment turns off error checking for the remainder of that line, so almost anything may appear in a comment. However, this is not the case with :. 
: This is a comment that generates an error, ( if [ $x -eq 3] ).

The ":" also serves as a field separator, in /etc/passwd, and in the $PATH variable. 
bash$ echo $PATH/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/usr/games

!

reverse (or negate) the sense of a test or exit status [bang]. The ! operator inverts the exit status of the command to which it is applied (see Example 6-2). It also inverts the meaning of a test operator. This can, for example, change the sense of equal ( = ) to not-equal ( != ). The ! operator is a Bash keyword.

Don't know where to look next?X

In a different context, the ! also appears in indirect variable references.

In yet another context, from the command line, the ! invokes the Bash history mechanism (see Appendix K). Note that within a script, the history mechanism is disabled.

*

wild card [asterisk]. The * character serves as a "wild card" for filename expansion in globbing. By itself, it matches every filename in a given directory.

bash$ echo *abs-book.sgml add-drive.sh agram.sh alias.sh

The * also represents any number (or zero) characters in a regular expression.

*

arithmetic operator. In the context of arithmetic operations, the * denotes multiplication.

** A double asterisk can represent the exponentiation operator or extended file-match globbing.

Confused? Frustrated?X
?

test operator. Within certain expressions, the ? indicates a test for a condition.

In a double-parentheses construct, the ? can serve as an element of a C-style trinary operator, ?:. 
(( var0 = var1<98?9:21 ))
#                ^ ^

# if [ "$var1" -lt 98 ]
# then
#   var0=9
# else
#   var0=21
# fi

In a parameter substitution expression, the ? tests whether a variable has been set.

?

wild card. The ? character serves as a single-character "wild card" for filename expansion in globbing, as well as representing one character in an extended regular expression.

$

Variable substitution (contents of a variable). 
var1=5
var2=23skidoo

echo $var1     # 5
echo $var2     # 23skidoo

A $ prefixing a variable name indicates the value the variable holds.

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X
$

end-of-line. In a regular expression, a "$" addresses the end of a line of text.

${}

Parameter substitution.

$*, $@

positional parameters.

$?

exit status variable. The $? variable holds the exit status of a command, a function, or of the script itself.

$$

process ID variable. The $$ variable holds the process ID [16] of the script in which it appears.

()

command group. 
(a=hello; echo $a)

Important

A listing of commands within parentheses starts a subshell.

Do you need help?X

Variables inside parentheses, within the subshell, are not visible to the rest of the script. The parent process, the script, cannot read variables created in the child process, the subshell. 
a=123
( a=321; )	      

echo "a = $a"   # a = 123
# "a" within parentheses acts like a local variable.

array initialization. 
Array=(element1 element2 element3)

Do you need more help?X

{xxx,yyy,zzz,...}

Brace expansion. 
echo \"{These,words,are,quoted}\"   # " prefix and suffix
# "These" "words" "are" "quoted"


cat {file1,file2,file3} > combined_file
# Concatenates the files file1, file2, and file3 into combined_file.

cp file22.{txt,backup}
# Copies "file22.txt" to "file22.backup"

A command may act upon a comma-separated list of file specs within braces. [17] Filename expansion (globbing) applies to the file specs between the braces.

Caution

No spaces allowed within the braces unless the spaces are quoted or escaped.

echo {file1,file2}\ :{\ A," B",' C'}

file1 : A file1 : B file1 : C file2 : A file2 : B file2 : C

Can we help you?X

{a..z}

Extended Brace expansion. 
echo {a..z} # a b c d e f g h i j k l m n o p q r s t u v w x y z
# Echoes characters between a and z.

echo {0..3} # 0 1 2 3
# Echoes characters between 0 and 3.

The {a..z} extended brace expansion construction is a feature introduced in version 3 of Bash.

{}

Block of code [curly brackets]. Also referred to as an inline group, this construct, in effect, creates an anonymous function (a function without a name). However, unlike in a "standard" function, the variables inside a code block remain visible to the remainder of the script.

bash$ { local a;
	      a=123; }bash: local: can only be used in a
function
	      


a=123
{ a=321; }
echo "a = $a"   # a = 321   (value inside code block)

# Thanks, S.C.

The code block enclosed in braces may have I/O redirected to and from it.

Example 3-1. Code blocks and I/O redirection

Can't find what you're looking for?X
#!/bin/bash
# Reading lines in /etc/fstab.

File=/etc/fstab

{
read line1
read line2
} < $File

echo "First line in $File is:"
echo "$line1"
echo
echo "Second line in $File is:"
echo "$line2"

exit 0

# Now, how do you parse the separate fields of each line?
# Hint: use awk, or . . .
# . . . Hans-Joerg Diers suggests using the "set" Bash builtin.

Example 3-2. Saving the output of a code block to a file
#!/bin/bash
# rpm-check.sh

#  Queries an rpm file for description, listing,
#+ and whether it can be installed.
#  Saves output to a file.
# 
#  This script illustrates using a code block.

SUCCESS=0
E_NOARGS=65

if [ -z "$1" ]
then
  echo "Usage: `basename $0` rpm-file"
  exit $E_NOARGS
fi  

{ # Begin code block.
  echo
  echo "Archive Description:"
  rpm -qpi $1       # Query description.
  echo
  echo "Archive Listing:"
  rpm -qpl $1       # Query listing.
  echo
  rpm -i --test $1  # Query whether rpm file can be installed.
  if [ "$?" -eq $SUCCESS ]
  then
    echo "$1 can be installed."
  else
    echo "$1 cannot be installed."
  fi  
  echo              # End code block.
} > "$1.test"       # Redirects output of everything in block to file.

echo "Results of rpm test in file $1.test"

# See rpm man page for explanation of options.

exit 0

Note

Unlike a command group within (parentheses), as above, a code block enclosed by {braces} will not normally launch a subshell. [18]

{}

placeholder for text. Used after xargs -i (replace strings option). The {} double curly brackets are a placeholder for output text.

ls . | xargs -i -t cp ./{} $1
#            ^^         ^^

Don't know where to look next?X


# From "ex42.sh" (copydir.sh) example.

anchor id="semicolonesc">

{} \;

pathname. Mostly used in find constructs. This is not a shell builtin.

Confused? Frustrated?X

Note

The ";" ends the -exec option of a find command sequence. It needs to be escaped to protect it from interpretation by the shell.

[ ]

test.

Test expression between [ ]. Note that [ is part of the shell builtin test (and a synonym for it), not a link to the external command /usr/bin/test.

[[ ]]

test.

Test expression between [[ ]]. More flexible than the single-bracket [ ] test, this is a shell keyword.

See the discussion on the [[ ... ]] construct.

[ ]

array element.

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

In the context of an array, brackets set off the numbering of each element of that array. 
Array[1]=slot_1
echo ${Array[1]}

[ ]

range of characters.

As part of a regular expression, brackets delineate a range of characters to match.

$[ ... ]

integer expansion.

Evaluate integer expression between $[ ]. 
a=3
b=7

echo $[$a+$b]   # 10
echo $[$a*$b]   # 21

Note that this usage is deprecated, and has been replaced by the (( ... )) construct.

(( ))

integer expansion.

Expand and evaluate integer expression between (( )).

Do you need help?X

See the discussion on the (( ... )) construct.

> &> >& >> < <>

redirection.

scriptname >filename redirects the output of scriptname to file filename. Overwrite filename if it already exists.

command &>filename redirects both the stdout and the stderr of command to filename.

Note

This is useful for suppressing output when testing for a condition. For example, let us test whether a certain command exists. 

bash$ type bogus_command &>/dev/nullbash$ echo $?1

Do you need more help?X

Or in a script:

command_test () { type "$1" &>/dev/null; }
#                                      ^

cmd=rmdir            # Legitimate command.
command_test $cmd; echo $?   # 0


cmd=bogus_command    # Illegitimate command
command_test $cmd; echo $?   # 1

command >&2 redirects stdout of command to stderr.

scriptname >>filename appends the output of scriptname to file filename. If filename does not already exist, it is created.

[i]<>filename opens file filename for reading and writing, and assigns file descriptor i to it. If filename does not exist, it is created.

process substitution.

Can we help you?X

(command)>

<(command)

In a different context, the "<" and ">" characters act as string comparison operators.

In yet another context, the "<" and ">" characters act as integer comparison operators. See also Example 15-9.

<<

redirection used in a here document.

<<<

redirection used in a here string.

<, >

ASCII comparison. 
veg1=carrots
veg2=tomatoes

if [[ "$veg1" < "$veg2" ]]
then
  echo "Although $veg1 precede $veg2 in the dictionary,"
  echo -n "this does not necessarily imply anything "
  echo "about my culinary preferences."
else
  echo "What kind of dictionary are you using, anyhow?"
fi

\<, \>

word boundary in a regular expression.

Can't find what you're looking for?X

bash$ grep '\<the\>' textfile

|

pipe. Passes the output (stdout of a previous command to the input (stdin) of the next one, or to the shell. This is a method of chaining commands together.

echo ls -l | sh
#  Passes the output of "echo ls -l" to the shell,
#+ with the same result as a simple "ls -l".


cat *.lst | sort | uniq
# Merges and sorts all ".lst" files, then deletes duplicate lines.

A pipe, as a classic method of interprocess communication, sends the stdout of one process to the stdin of another. In a typical case, a command, such as cat or echo, pipes a stream of data to a filter, a command that transforms its input for processing. [19]

cat $filename1 $filename2 | grep $search_word

For an interesting note on the complexity of using UNIX pipes, see the UNIX FAQ, Part 3.

Don't know where to look next?X

The output of a command or commands may be piped to a script. 
#!/bin/bash
# uppercase.sh : Changes input to uppercase.

tr 'a-z' 'A-Z'
#  Letter ranges must be quoted
#+ to prevent filename generation from single-letter filenames.

exit 0
Now, let us pipe the output of ls -l to this script. 
bash$ ls -l | ./uppercase.sh-RW-RW-R--    1 BOZO  BOZO       109 APR  7 19:49 1.TXT
 -RW-RW-R--    1 BOZO  BOZO       109 APR 14 16:48 2.TXT
 -RW-R--R--    1 BOZO  BOZO       725 APR 20 20:56 DATA-FILE

Note

The stdout of each process in a pipe must be read as the stdin of the next. If this is not the case, the data stream will block, and the pipe will not behave as expected. 
cat file1 file2 | ls -l | sort
# The output from "cat file1 file2" disappears.

A pipe runs as a child process, and therefore cannot alter script variables. 
variable="initial_value"
echo "new_value" | read variable
echo "variable = $variable"     # variable = initial_value

If one of the commands in the pipe aborts, this prematurely terminates execution of the pipe. Called a broken pipe, this condition sends a SIGPIPE signal.

>|

force redirection (even if the noclobber option is set). This will forcibly overwrite an existing file.

||

OR logical operator. In a test construct, the || operator causes a return of 0 (success) if either of the linked test conditions is true.

Confused? Frustrated?X
&

Run job in background. A command followed by an & will run in the background.

bash$ sleep 10 &[1] 850[1]+  Done                    sleep 10

Within a script, commands and even loops may run in the background.

Example 3-3. Running a loop in the background
#!/bin/bash
# background-loop.sh

for i in 1 2 3 4 5 6 7 8 9 10            # First loop.
do
  echo -n "$i "
done & # Run this loop in background.
       # Will sometimes execute after second loop.

echo   # This 'echo' sometimes will not display.

for i in 11 12 13 14 15 16 17 18 19 20   # Second loop.
do
  echo -n "$i "
done  

echo   # This 'echo' sometimes will not display.

# ======================================================

# The expected output from the script:
# 1 2 3 4 5 6 7 8 9 10 
# 11 12 13 14 15 16 17 18 19 20 

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X


# Sometimes, though, you get:
# 11 12 13 14 15 16 17 18 19 20 
# 1 2 3 4 5 6 7 8 9 10 bozo $
# (The second 'echo' doesn't execute. Why?)

# Occasionally also:
# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
# (The first 'echo' doesn't execute. Why?)

# Very rarely something like:
# 11 12 13 1 2 3 4 5 6 7 8 9 10 14 15 16 17 18 19 20 
# The foreground loop preempts the background one.

exit 0

#  Nasimuddin Ansari suggests adding    sleep 1
#+ after the   echo -n "$i"   in lines 6 and 14,
#+ for some real fun.

Caution

A command run in the background within a script may cause the script to hang, waiting for a keystroke. Fortunately, there is a remedy for this.

&&

AND logical operator. In a test construct, the && operator causes a return of 0 (success) only if both the linked test conditions are true.

Do you need help?X
-

option, prefix. Option flag for a command or filter. Prefix for an operator. Prefix for a default parameter in parameter substitution.

COMMAND -[Option1][Option2][...]

ls -al

sort -dfu $filename

if [ $file1 -ot $file2 ]
then #      ^
  echo "File $file1 is older than $file2."
fi

if [ "$a" -eq "$b" ]
then      ^
  echo "$a is equal to $b."
fi

if [ "$c" -eq 24 -a "$d" -eq 47 ]
then      ^              ^
  echo "$c equals 24 and $d equals 47."
fi


param2=${param1:-$DEFAULTVAL}
#               ^

--

The double-dash -- prefixes long (verbatim) options to commands.

Do you need more help?X

sort --ignore-leading-blanks

Used with a Bash builtin, it means the end of options to that particular command.

Tip

This provides a handy means of removing files whose names begin with a dash. 
bash$ ls -l-rw-r--r-- 1 bozo bozo 0 Nov 25 12:29 -badnamebash$ rm -- -badnamebash$ ls -ltotal 0

The double-dash is also used in conjunction with set.

set -- $variable (as in Example 14-18)

-

redirection from/to stdin or stdout [dash]. 

bash$ cat -abcabc...Ctl-D

Can we help you?X

As expected, cat - echoes stdin, in this case keyboarded user input, to stdout. But, does I/O redirection using - have real-world applications?

(cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xpvf -)
# Move entire file tree from one directory to another
# [courtesy Alan Cox , with a minor change]

# 1) cd /source/directory
#    Source directory, where the files to be moved are.
# 2) &&
#   "And-list": if the 'cd' operation successful,
#    then execute the next command.
# 3) tar cf - .
#    The 'c' option 'tar' archiving command creates a new archive,
#    the 'f' (file) option, followed by '-' designates the target file
#    as stdout, and do it in current directory tree ('.').
# 4) |
#    Piped to ...
# 5) ( ... )
#    a subshell
# 6) cd /dest/directory
#    Change to the destination directory.
# 7) &&
#   "And-list", as above
# 8) tar xpvf -
#    Unarchive ('x'), preserve ownership and file permissions ('p'),
#    and send verbose messages to stdout ('v'),
#    reading data from stdin ('f' followed by '-').
#
#    Note that 'x' is a command, and 'p', 'v', 'f' are options.

Can't find what you're looking for?X

#
# Whew!



# More elegant than, but equivalent to:
#   cd source/directory
#   tar cf - . | (cd ../dest/directory; tar xpvf -)
#
#     Also having same effect:
# cp -a /source/directory/* /dest/directory
#     Or:
# cp -a /source/directory/* /source/directory/.[^.]* /dest/directory
#     If there are hidden files in /source/directory.


bunzip2 -c linux-2.6.16.tar.bz2 | tar xvf -
#  --uncompress tar file--    | --then pass it to "tar"--
#  If "tar" has not been patched to handle "bunzip2",
#+ this needs to be done in two discrete steps, using a pipe.
#  The purpose of the exercise is to unarchive "bzipped" kernel source.

Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain UNIX utilities that write to stdout, such as tar, cat, etc.

bash$ echo "whatever" | cat -whatever

Where a filename is expected, - redirects output to stdout (sometimes seen with tar cf), or accepts input from stdin, rather than from a file. This is a method of using a file-oriented utility as a filter in a pipe.

bash$ fileUsage: file [-bciknvzL] [-f namefile] [-m magicfiles] file...
By itself on the command-line, file fails with an error message.

Add a "-" for a more useful result. This causes the shell to await user input. 
bash$ file -abcstandard input:              ASCII textbash$ file -#!/bin/bashstandard input:              Bourne-Again shell script text executable
Now the command accepts input from stdin and analyzes it.

Don't know where to look next?X

The "-" can be used to pipe stdout to other commands. This permits such stunts as prepending lines to a file.

Using diff to compare a file with a section of another:

grep Linux file1 | diff file2 -

Finally, a real-world example using - with tar.

Example 3-4. Backup of all files changed in last day
#!/bin/bash

#  Backs up all files in current directory modified within last 24 hours
#+ in a "tarball" (tarred and gzipped file).

BACKUPFILE=backup-$(date +%m-%d-%Y)
#                 Embeds date in backup filename.
#                 Thanks, Joshua Tschida, for the idea.
archive=${1:-$BACKUPFILE}
#  If no backup-archive filename specified on command-line,
#+ it will default to "backup-MM-DD-YYYY.tar.gz."

tar cvf - `find . -mtime -1 -type f -print` > $archive.tar
gzip $archive.tar
echo "Directory $PWD backed up in archive file \"$archive.tar.gz\"."


#  Stephane Chazelas points out that the above code will fail
#+ if there are too many files found
#+ or if any filenames contain blank characters.

# He suggests the following alternatives:
# -------------------------------------------------------------------
#   find . -mtime -1 -type f -print0 | xargs -0 tar rvf "$archive.tar"
#      using the GNU version of "find".


#   find . -mtime -1 -type f -exec tar rvf "$archive.tar" '{}' \;
#         portable to other UNIX flavors, but much slower.
# -------------------------------------------------------------------

Confused? Frustrated?X



exit 0

Caution

Filenames beginning with "-" may cause problems when coupled with the "-" redirection operator. A script should check for this and add an appropriate prefix to such filenames, for example ./-FILENAME, $PWD/-FILENAME, or $PATHNAME/-FILENAME.

If the value of a variable begins with a -, this may likewise create problems. 
var="-n"
echo $var		
# Has the effect of "echo -n", and outputs nothing.

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

-

previous working directory. A cd - command changes to the previous working directory. This uses the $OLDPWD environmental variable.

Caution

Do not confuse the "-" used in this sense with the "-" redirection operator just discussed. The interpretation of the "-" depends on the context in which it appears.

-

Minus. Minus sign in an arithmetic operation.

=

Equals. Assignment operator
a=28
echo $a   # 28

In a different context, the "=" is a string comparison operator.

+

Plus. Addition arithmetic operator.

In a different context, the + is a Regular Expression operator.

Do you need help?X
+

Option. Option flag for a command or filter.

Certain commands and builtins use the + to enable certain options and the - to disable them. In parameter substitution, the + prefixes an alternate value that a variable expands to.

%

modulo. Modulo (remainder of a division) arithmetic operation.

let "z = 5 % 3"
echo $z  # 2

In a different context, the % is a pattern matching operator.

~

home directory [tilde]. This corresponds to the $HOME internal variable. ~bozo is bozo's home directory, and ls ~bozo lists the contents of it. ~/ is the current user's home directory, and ls ~/ lists the contents of it. 
bash$ echo ~bozo/home/bozobash$ echo ~/home/bozobash$ echo ~//home/bozo/bash$ echo ~:/home/bozo:bash$ echo ~nonexistent-user~nonexistent-user

~+

current working directory. This corresponds to the $PWD internal variable.

~-

previous working directory. This corresponds to the $OLDPWD internal variable.

Do you need more help?X
=~

regular expression match. This operator was introduced with version 3 of Bash.

^

beginning-of-line. In a regular expression, a "^" addresses the beginning of a line of text.

^, ^^

Uppercase conversion in parameter substitution (added in version 4 of Bash).

Control Characters

change the behavior of the terminal or text display. A control character is a CONTROL + key combination (pressed simultaneously). A control character may also be written in octal or hexadecimal notation, following an escape.

Control characters are not normally useful inside a script.

Whitespace

functions as a separator between commands and/or variables. Whitespace consists of either spaces, tabs, blank lines, or any combination thereof. [21] In some contexts, such as variable assignment, whitespace is not permitted, and results in a syntax error.

Blank lines have no effect on the action of a script, and are therefore useful for visually separating functional sections.

$IFS, the special variable separating fields of input to certain commands. It defaults to whitespace.

Definition: A field is a discrete chunk of data expressed as a string of consecutive characters. Separating each field from adjacent fields is either whitespace or some other designated character (often determined by the $IFS). In some contexts, a field may be called a record.

Confused? Frustrated?X

To preserve whitespace within a string or in a variable, use quoting.

UNIX filters can target and operate on whitespace using the POSIX character class [:space:].

Chapter 4. Introduction to Variables and Parameters

Variables are how programming and scripting languages represent data. A variable is nothing more than a label, a name assigned to a location or set of locations in computer memory holding an item of data.

Variables appear in arithmetic operations and manipulation of quantities, and in string parsing.

4.1. Variable Substitution

The name of a variable is a placeholder for its value, the data it holds. Referencing (retrieving) its value is called variable substitution.

$

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

Let us carefully distinguish between the name of a variable and its value. If variable1 is the name of a variable, then $variable1 is a reference to its value, the data item it contains. [22]

bash$ variable1=23bash$ echo variable1variable1bash$ echo $variable123

The only time a variable appears "naked" -- without the $ prefix -- is when declared or assigned, when unset, when exported, or in the special case of a variable representing a signal (see Example 29-5). Assignment may be with an = (as in var1=27), in a read statement, and at the head of a loop (for var2 in 1 2 3).

Enclosing a referenced value in double quotes (" ... ") does not interfere with variable substitution. This is called partial quoting, sometimes referred to as "weak quoting." Using single quotes (' ... ') causes the variable name to be used literally, and no substitution will take place. This is full quoting, sometimes referred to as 'strong quoting.' See Chapter 5 for a detailed discussion.

Note that $variable is actually a simplified form of ${variable}. In contexts where the $variable syntax causes an error, the longer form may work (see Section 9.3, below).

Example 4-1. Variable assignment and substitution
#!/bin/bash
# ex9.sh

# Variables: assignment and substitution

a=375
hello=$a

#-------------------------------------------------------------------------
# No space permitted on either side of = sign when initializing variables.
# What happens if there is a space?

#  "VARIABLE =value"
#           ^
#% Script tries to run "VARIABLE" command with one argument, "=value".

#  "VARIABLE= value"
#            ^
#% Script tries to run "value" command with
#+ the environmental variable "VARIABLE" set to "".
#-------------------------------------------------------------------------


echo hello    # hello
# Not a variable reference, just the string "hello" . . .

echo $hello   # 375
#    ^          This *is* a variable reference.
echo ${hello} # 375
# Also a variable reference, as above.

# Quoting . . .
echo "$hello"    # 375
echo "${hello}"  # 375

echo

hello="A B  C   D"
echo $hello   # A B C D
echo "$hello" # A B  C   D
# As you see, echo $hello   and   echo "$hello"   give different results.
# Why?
# =======================================
# Quoting a variable preserves whitespace.
# =======================================

echo

echo '$hello'  # $hello
#    ^      ^
#  Variable referencing disabled (escaped) by single quotes,
#+ which causes the "$" to be interpreted literally.

# Notice the effect of different types of quoting.


hello=    # Setting it to a null value.

Do you need help?X

echo "\$hello (null value) = $hello"
#  Note that setting a variable to a null value is not the same as
#+ unsetting it, although the end result is the same (see below).

# --------------------------------------------------------------

#  It is permissible to set multiple variables on the same line,
#+ if separated by white space.
#  Caution, this may reduce legibility, and may not be portable.

var1=21  var2=22  var3=$V3
echo
echo "var1=$var1   var2=$var2   var3=$var3"

# May cause problems with older versions of "sh" . . .

# --------------------------------------------------------------

echo; echo

numbers="one two three"
#           ^   ^
other_numbers="1 2 3"
#               ^ ^
#  If there is whitespace embedded within a variable,
#+ then quotes are necessary.
#  other_numbers=1 2 3                  # Gives an error message.
echo "numbers = $numbers"
echo "other_numbers = $other_numbers"   # other_numbers = 1 2 3
#  Escaping the whitespace also works.
mixed_bag=2\ ---\ Whatever
#           ^    ^ Space after escape (\).

echo "$mixed_bag"         # 2 --- Whatever

echo; echo

echo "uninitialized_variable = $uninitialized_variable"
# Uninitialized variable has null value (no value at all!).
uninitialized_variable=   #  Declaring, but not initializing it --
                          #+ same as setting it to a null value, as above.
echo "uninitialized_variable = $uninitialized_variable"
                          # It still has a null value.

uninitialized_variable=23       # Set it.
unset uninitialized_variable    # Unset it.
echo "uninitialized_variable = $uninitialized_variable"
                                # It still has a null value.
echo

exit 0

Do you need more help?X
Caution

An uninitialized variable has a "null" value -- no assigned value at all (not zero!). 
if [ -z "$unassigned" ]
then
  echo "\$unassigned is NULL."
fi     # $unassigned is NULL.

Can we help you?X

Using a variable before assigning a value to it may cause problems. It is nevertheless possible to perform arithmetic operations on an uninitialized variable. 
echo "$uninitialized"                                # (blank line)
let "uninitialized += 5"                             # Add 5 to it.
echo "$uninitialized"                                # 5

#  Conclusion:
#  An uninitialized variable has no value,
#+ however it acts as if it were 0 in an arithmetic operation.
#  This is undocumented (and probably non-portable) behavior,
#+ and should not be used in a script.
See also Example 14-23.

4.2. Variable Assignment

=

the assignment operator (no space before and after)

Caution

Do not confuse this with = and -eq, which test, rather than assign!

Note that = can be either an assignment or a test operator, depending on context.

Can't find what you're looking for?X

Example 4-2. Plain Variable Assignment
#!/bin/bash
# Naked variables

echo

# When is a variable "naked", i.e., lacking the '$' in front?
# When it is being assigned, rather than referenced.

# Assignment
a=879
echo "The value of \"a\" is $a."

# Assignment using 'let'
let a=16+5
echo "The value of \"a\" is now $a."

echo

# In a 'for' loop (really, a type of disguised assignment):
echo -n "Values of \"a\" in the loop are: "
for a in 7 8 9 11
do
  echo -n "$a "
done

echo
echo

# In a 'read' statement (also a type of assignment):
echo -n "Enter \"a\" "
read a
echo "The value of \"a\" is now $a."

echo

exit 0

Example 4-3. Variable Assignment, plain and fancy
#!/bin/bash

a=23              # Simple case
echo $a
b=$a
echo $b

# Now, getting a little bit fancier (command substitution).

a=`echo Hello!`   # Assigns result of 'echo' command to 'a' ...
echo $a
#  Note that including an exclamation mark (!) within a
#+ command substitution construct will not work from the command-line,
#+ since this triggers the Bash "history mechanism."
#  Inside a script, however, the history functions are disabled.

a=`ls -l`         # Assigns result of 'ls -l' command to 'a'
echo $a           # Unquoted, however, it removes tabs and newlines.
echo
echo "$a"         # The quoted variable preserves whitespace.
                  # (See the chapter on "Quoting.")

exit 0

Variable assignment using the $(...) mechanism (a newer method than backquotes). This is actually a form of command substitution.

# From /etc/rc.d/rc.local
R=$(cat /etc/redhat-release)
arch=$(uname -m)

4.3. Bash Variables Are Untyped

Don't know where to look next?X

Unlike many other programming languages, Bash does not segregate its variables by "type." Essentially, Bash variables are character strings, but, depending on context, Bash permits arithmetic operations and comparisons on variables. The determining factor is whether the value of a variable contains only digits.

Example 4-4. Integer or string?
#!/bin/bash
# int-or-string.sh

Confused? Frustrated?X


a=2334                   # Integer.
let "a += 1"
echo "a = $a "           # a = 2335
echo                     # Integer, still.


b=${a/23/BB}             # Substitute "BB" for "23".
                         # This transforms $b into a string.
echo "b = $b"            # b = BB35
declare -i b             # Declaring it an integer doesn't help.
echo "b = $b"            # b = BB35

let "b += 1"             # BB35 + 1
echo "b = $b"            # b = 1
echo                     # Bash sets the "integer value" of a string to 0.

c=BB34
echo "c = $c"            # c = BB34
d=${c/BB/23}             # Substitute "23" for "BB".
                         # This makes $d an integer.
echo "d = $d"            # d = 2334
let "d += 1"             # 2334 + 1
echo "d = $d"            # d = 2335
echo


# What about null variables?
e=''                     # ... Or e="" ... Or e=
echo "e = $e"            # e =
let "e += 1"             # Arithmetic operations allowed on a null variable?
echo "e = $e"            # e = 1
echo                     # Null variable transformed into an integer.

# What about undeclared variables?
echo "f = $f"            # f =
let "f += 1"             # Arithmetic operations allowed?
echo "f = $f"            # f = 1
echo                     # Undeclared variable transformed into an integer.
#
# However ...
let "f /= $undecl_var"   # Divide by zero?
#   let: f /= : syntax error: operand expected (error token is " ")
# Syntax error! Variable $undecl_var is not set to zero here!
#
# But still ...
let "f /= 0"
#   let: f /= 0: division by 0 (error token is "0")
# Expected behavior.


#  Bash (usually) sets the "integer value" of null to zero
#+ when performing an arithmetic operation.
#  But, don't try this at home, folks!
#  It's undocumented and probably non-portable behavior.

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X



# Conclusion: Variables in Bash are untyped,
#+ with all attendant consequences.

exit $?

Untyped variables are both a blessing and a curse. They permit more flexibility in scripting and make it easier to grind out lines of code (and give you enough rope to hang yourself!). However, they likewise permit subtle errors to creep in and encourage sloppy programming habits.

To lighten the burden of keeping track of variable types in a script, Bash does permit declaring variables.

4.4. Special Variable Types

Local variables

Variables visible only within a code block or function (see also local variables in functions)

Environmental variables

Variables that affect the behavior of the shell and user interface

Do you need help?X
Note

In a more general context, each process has an "environment", that is, a group of variables that the process may reference. In this sense, the shell behaves like any other process.

Every time a shell starts, it creates shell variables that correspond to its own environmental variables. Updating or adding new environmental variables causes the shell to update its environment, and all the shell's child processes (the commands it executes) inherit this environment.

Caution

The space allotted to the environment is limited. Creating too many environmental variables or ones that use up excessive space may cause problems.

bash$ eval "`seq 10000 | sed -e 's/.*/export var&=ZZZZZZZZZZZZZZ/'`"bash$ dubash: /usr/bin/du: Argument list too long

Note: this "error" has been fixed, as of kernel version 2.6.23.

(Thank you, Stéphane Chazelas for the clarification, and for providing the above example.)

If a script sets environmental variables, they need to be "exported," that is, reported to the environment local to the script. This is the function of the export command.

Do you need more help?X

Note

A script can export variables only to child processes, that is, only to commands or processes which that particular script initiates. A script invoked from the command-line cannot export variables back to the command-line environment. Child processes cannot export variables back to the parent processes that spawned them.

Definition: A child process is a subprocess launched by another process, its parent.

Positional parameters

Arguments passed to the script from the command line [23] : $0, $1, $2, $3 . . .

$0 is the name of the script itself, $1 is the first argument, $2 the second, $3 the third, and so forth. [24] After $9, the arguments must be enclosed in brackets, for example, ${10}, ${11}, ${12}.

The special variables $* and $@ denote all the positional parameters.

Example 4-5. Positional Parameters
#!/bin/bash

# Call this script with at least 10 parameters, for example
# ./scriptname 1 2 3 4 5 6 7 8 9 10
MINPARAMS=10

echo

echo "The name of this script is \"$0\"."
# Adds ./ for current directory
echo "The name of this script is \"`basename $0`\"."
# Strips out path name info (see 'basename')

echo

if [ -n "$1" ]              # Tested variable is quoted.
then
 echo "Parameter #1 is $1"  # Need quotes to escape #
fi 

if [ -n "$2" ]
then
 echo "Parameter #2 is $2"
fi 

if [ -n "$3" ]
then
 echo "Parameter #3 is $3"
fi 

# ...


if [ -n "${10}" ]  # Parameters > $9 must be enclosed in {brackets}.
then
 echo "Parameter #10 is ${10}"
fi 

echo "-----------------------------------"
echo "All the command-line parameters are: "$*""

if [ $# -lt "$MINPARAMS" ]
then
  echo
  echo "This script needs at least $MINPARAMS command-line arguments!"
fi  

echo

exit 0

Bracket notation for positional parameters leads to a fairly simple way of referencing the last argument passed to a script on the command-line. This also requires indirect referencing.

Can we help you?X

args=$#           # Number of args passed.
lastarg=${!args}
# Note: This is an *indirect reference* to $args ...


# Or:       lastarg=${!#}             (Thanks, Chris Monson.)
# This is an *indirect reference* to the $# variable.
# Note that lastarg=${!$#} doesn't work.

Some scripts can perform different operations, depending on which name they are invoked with. For this to work, the script needs to check $0, the name it was invoked by. There must also exist symbolic links to all the alternate names of the script. See Example 15-2.

Tip

If a script expects a command-line parameter but is invoked without one, this may cause a null variable assignment, generally an undesirable result. One way to prevent this is to append an extra character to both sides of the assignment statement using the expected positional parameter. 

variable1_=$1_  # Rather than variable1=$1
# This will prevent an error, even if positional parameter is absent.

critical_argument01=$variable1_

# The extra character can be stripped off later, like so.
variable1=${variable1_/_/}
# Side effects only if $variable1_ begins with an underscore.
# This uses one of the parameter substitution templates discussed later.
# (Leaving out the replacement pattern results in a deletion.)

#  A more straightforward way of dealing with this is
#+ to simply test whether expected positional parameters have been passed.

Can't find what you're looking for?X

if [ -z $1 ]
then
  exit $E_MISSING_POS_PARAM
fi


#  However, as Fabian Kreutz points out,
#+ the above method may have unexpected side-effects.
#  A better method is parameter substitution:
#         ${1:-$DefaultVal}
#  See the "Parameter Substition" section
#+ in the "Variables Revisited" chapter.

---

Example 4-6. wh, whois domain name lookup

Don't know where to look next?X
#!/bin/bash
# ex18.sh

# Does a 'whois domain-name' lookup on any of 3 alternate servers:
#                    ripe.net, cw.net, radb.net

# Place this script -- renamed 'wh' -- in /usr/local/bin

# Requires symbolic links:
# ln -s /usr/local/bin/wh /usr/local/bin/wh-ripe
# ln -s /usr/local/bin/wh /usr/local/bin/wh-cw
# ln -s /usr/local/bin/wh /usr/local/bin/wh-radb

E_NOARGS=65


if [ -z "$1" ]
then
  echo "Usage: `basename $0` [domain-name]"
  exit $E_NOARGS
fi

# Check script name and call proper server.
case `basename $0` in    # Or:    case ${0##*/} in
    "wh"     ) whois $1@whois.ripe.net;;
    "wh-ripe") whois $1@whois.ripe.net;;
    "wh-radb") whois $1@whois.radb.net;;
    "wh-cw"  ) whois $1@whois.cw.net;;
    *        ) echo "Usage: `basename $0` [domain-name]";;
esac 

exit $?

---

The shift command reassigns the positional parameters, in effect shifting them to the left one notch.

$1 <--- $2, $2 <--- $3, $3 <--- $4, etc.

The old $1 disappears, but $0 (the script name) does not change. If you use a large number of positional parameters to a script, shift lets you access those past 10, although {bracket} notation also permits this.

Example 4-7. Using shift
#!/bin/bash
# shft.sh: Using 'shift' to step through all the positional parameters.

#  Name this script something like shft.sh,
#+ and invoke it with some parameters.
#+ For example:
#             sh shft.sh a b c def 23 Skidoo

until [ -z "$1" ]  # Until all parameters used up . . .
do
  echo -n "$1 "
  shift
done

echo               # Extra linefeed.

# But, what happens to the "used-up" parameters?
echo "$2"
#  Nothing echoes!
#  When $2 shifts into $1 (and there is no $3 to shift into $2)
#+ then $2 remains empty.
#  So, it is not a parameter *copy*, but a *move*.

exit

#  See also the echo-params.sh script for a "shiftless"
#+ alternative method of stepping through the positional params.

Confused? Frustrated?X

The shift command can take a numerical parameter indicating how many positions to shift.

#!/bin/bash
# shift-past.sh

shift 3    # Shift 3 positions.
#  n=3; shift $n
#  Has the same effect.

echo "$1"

exit 0

# ======================== #


$ sh shift-past.sh 1 2 3 4 5
4

#  However, as Eleni Fragkiadaki, points out,
#+ attempting a 'shift' past the number of
#+ positional parameters ($#) returns an exit status of 1,
#+ and the positional parameters themselves do not change.
#  This means possibly getting stuck in an endless loop. . . .
#  For example:
#      until [ -z "$1" ]
#      do
#         echo -n "$1 "
#         shift 20    #  If less than 20 pos params,
#      done           #+ then loop never ends!
#
# When in doubt, add a sanity check. . . .
#           shift 20 || break
#                    ^^^^^^^^

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

Note

The shift command works in a similar fashion on parameters passed to a function. See Example 33-16.

Chapter 5. Quoting

Quoting means just that, bracketing a string in quotes. This has the effect of protecting special characters in the string from reinterpretation or expansion by the shell or shell script. (A character is "special" if it has an interpretation other than its literal meaning. For example, the asterisk * represents a wild card character in globbing and Regular Expressions).

bash$ ls -l [Vv]*-rw-rw-r--    1 bozo  bozo       324 Apr  2 15:05 VIEWDATA.BAT
 -rw-rw-r--    1 bozo  bozo       507 May  4 14:25 vartrace.sh
 -rw-rw-r--    1 bozo  bozo       539 Apr 14 17:11 viewdata.shbash$ ls -l '[Vv]*'ls: [Vv]*: No such file or directory

In everyday speech or writing, when we "quote" a phrase, we set it apart and give it special meaning. In a Bash script, when we quote a string, we set it apart and protect its literal meaning.

Do you need help?X

Certain programs and utilities reinterpret or expand special characters in a quoted string. An important use of quoting is protecting a command-line parameter from the shell, but still letting the calling program expand it.

bash$ grep '[Ff]irst' *.txtfile1.txt:This is the first line of file1.txt.
 file2.txt:This is the First line of file2.txt.

Note that the unquoted grep [Ff]irst *.txt works under the Bash shell. [25]

Quoting can also suppress echo's "appetite" for newlines.

bash$ echo $(ls -l)total 8 -rw-rw-r-- 1 bo bo 13 Aug 21 12:57 t.sh -rw-rw-r-- 1 bo bo 78 Aug 21 12:57 u.shbash$ echo "$(ls -l)"total 8
 -rw-rw-r--  1 bo bo  13 Aug 21 12:57 t.sh
 -rw-rw-r--  1 bo bo  78 Aug 21 12:57 u.sh

5.1. Quoting Variables

When referencing a variable, it is generally advisable to enclose its name in double quotes. This prevents reinterpretation of all special characters within the quoted string -- except $, ` (backquote), and \ (escape). [26] Keeping $ as a special character within double quotes permits referencing a quoted variable ("$variable"), that is, replacing the variable with its value (see Example 4-1, above).

Do you need more help?X

Use double quotes to prevent word splitting. [27] An argument enclosed in double quotes presents itself as a single word, even if it contains whitespace separators.

List="one two three"

for a in $List     # Splits the variable in parts at whitespace.
do
  echo "$a"
done
# one
# two
# three

echo "---"

for a in "$List"   # Preserves whitespace in a single variable.
do #     ^     ^
  echo "$a"
done
# one two three

A more elaborate example:

variable1="a variable containing five words"
COMMAND This is $variable1    # Executes COMMAND with 7 arguments:
# "This" "is" "a" "variable" "containing" "five" "words"

COMMAND "This is $variable1"  # Executes COMMAND with 1 argument:
# "This is a variable containing five words"


variable2=""    # Empty.

COMMAND $variable2 $variable2 $variable2
                # Executes COMMAND with no arguments. 
COMMAND "$variable2" "$variable2" "$variable2"
                # Executes COMMAND with 3 empty arguments. 
COMMAND "$variable2 $variable2 $variable2"
                # Executes COMMAND with 1 argument (2 spaces). 

# Thanks, Stéphane Chazelas.

Tip

Enclosing the arguments to an echo statement in double quotes is necessary only when word splitting or preservation of whitespace is an issue.

Example 5-1. Echoing Weird Variables

Can we help you?X
#!/bin/bash
# weirdvars.sh: Echoing weird variables.

echo

var="'(]\\{}\$\""
echo $var        # '(]\{}$"
echo "$var"      # '(]\{}$"     Doesn't make a difference.

Can't find what you're looking for?X


echo

IFS='\'
echo $var        # '(] {}$"     \ converted to space. Why?
echo "$var"      # '(]\{}$"

# Examples above supplied by Stephane Chazelas.

echo

var2="\\\\\""
echo $var2       #   "
echo "$var2"     # \\"
echo
# But ... var2="\\\\"" is illegal. Why?
var3='\\\\'
echo "$var3"     # \\\\
# Strong quoting works, though.

exit

Single quotes (' ') operate similarly to double quotes, but do not permit referencing variables, since the special meaning of $ is turned off. Within single quotes, every special character except ' gets interpreted literally. Consider single quotes ("full quoting") to be a stricter method of quoting than double quotes ("partial quoting").

Note

Since even the escape character (\) gets a literal interpretation within single quotes, trying to enclose a single quote within single quotes will not yield the expected result. 
echo "Why can't I write 's between single quotes"

echo

# The roundabout method.
echo 'Why can'\''t I write '"'"'s between single quotes'
#    |-------|  |----------|   |-----------------------|
# Three single-quoted strings, with escaped and quoted single quotes between.

# This example courtesy of Stéphane Chazelas.

5.2. Escaping

Escaping is a method of quoting single characters. The escape (\) preceding a character tells the shell to interpret that character literally.

Caution

With certain commands and utilities, such as echo and sed, escaping a character may have the opposite effect - it can toggle on a special meaning for that character.

Special meanings of certain escaped characters

Don't know where to look next?X
used with echo and sed

\n

means newline

\r

means return

\t

means tab

\v

means vertical tab

\b

means backspace

\a

means alert (beep or flash)

\0xx

translates to the octal ASCII equivalent of 0nn, where nn is a string of digits

Confused? Frustrated?X

Example 5-2. Escaped Characters
#!/bin/bash
# escaped.sh: escaped characters

echo; echo

# Escaping a newline.
# ------------------

echo ""

echo "This will print
as two lines."
# This will print
# as two lines.

echo "This will print \
as one line."
# This will print as one line.

echo; echo

echo "============="


echo "\v\v\v\v"      # Prints \v\v\v\v literally.
# Use the -e option with 'echo' to print escaped characters.
echo "============="
echo "VERTICAL TABS"
echo -e "\v\v\v\v"   # Prints 4 vertical tabs.
echo "=============="

echo "QUOTATION MARK"
echo -e "\042"       # Prints " (quote, octal ASCII character 42).
echo "=============="

# The $'\X' construct makes the -e option unnecessary.
echo; echo "NEWLINE AND BEEP"
echo $'\n'           # Newline.
echo $'\a'           # Alert (beep).

echo "==============="
echo "QUOTATION MARKS"
# Version 2 and later of Bash permits using the $'\nnn' construct.
# Note that in this case, '\nnn' is an octal value.
echo $'\t \042 \t'   # Quote (") framed by tabs.

# It also works with hexadecimal values, in an $'\xhhh' construct.
echo $'\t \x22 \t'  # Quote (") framed by tabs.
# Thank you, Greg Keraunen, for pointing this out.

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

# Earlier Bash versions allowed '\x022'.
echo "==============="
echo





# Assigning ASCII characters to a variable.
# ----------------------------------------
quote=$'\042'        # " assigned to a variable.
echo "$quote This is a quoted string, $quote and this lies outside the quotes."

echo

# Concatenating ASCII chars in a variable.
triple_underline=$'\137\137\137'  # 137 is octal ASCII code for '_'.
echo "$triple_underline UNDERLINE $triple_underline"

echo

ABC=$'\101\102\103\010'           # 101, 102, 103 are octal A, B, C.
echo $ABC

echo; echo

escape=$'\033'                    # 033 is octal for escape.
echo "\"escape\" echoes as $escape"
#                                   no visible output.

echo; echo

exit 0

See Example 34-1 for another example of the $' ... ' string-expansion construct.

\"

gives the quote its literal meaning

echo "Hello"                     # Hello
echo "\"Hello\" ... he said."    # "Hello" ... he said.

\$

gives the dollar sign its literal meaning (variable name following \$ will not be referenced)

echo "\$variable01"           # $variable01
echo "The book cost \$7.98."  # The book cost $7.98.

\\

gives the backslash its literal meaning

echo "\\"  # Results in \

# Whereas . . .

echo "\"   # Invokes secondary prompt from the command-line.
           # In a script, gives an error message.

# However . . .

echo '\'   # Results in \

Do you need help?X

Note

The behavior of \ depends on whether it is escaped, strong-quoted, weak-quoted, or appearing within command substitution or a here document. 
                      #  Simple escaping and quoting
echo \z               #  z
echo \\z              # \z
echo '\z'             # \z
echo '\\z'            # \\z
echo "\z"             # \z
echo "\\z"            # \z

                      #  Command substitution
echo `echo \z`        #  z
echo `echo \\z`       #  z
echo `echo \\\z`      # \z
echo `echo \\\\z`     # \z
echo `echo \\\\\\z`   # \z
echo `echo \\\\\\\z`  # \\z
echo `echo "\z"`      # \z
echo `echo "\\z"`     # \z

                      # Here document
cat <
Do you need more help?X

\\z                     
EOF                   # \z

# These examples supplied by Stéphane Chazelas.

Elements of a string assigned to a variable may be escaped, but the escape character alone may not be assigned to a variable. 
variable=\
echo "$variable"
# Will not work - gives an error message:
# test.sh: : command not found
# A "naked" escape cannot safely be assigned to a variable.
#
#  What actually happens here is that the "\" escapes the newline and
#+ the effect is        variable=echo "$variable"
#+                      invalid variable assignment

variable=\
23skidoo
echo "$variable"        #  23skidoo
                        #  This works, since the second line
                        #+ is a valid variable assignment.

variable=\ 
#        \^    escape followed by space
echo "$variable"        # space

variable=\\
echo "$variable"        # \

variable=\\\
echo "$variable"
# Will not work - gives an error message:
# test.sh: \: command not found
#
#  First escape escapes second one, but the third one is left "naked",
#+ with same result as first instance, above.

variable=\\\\
echo "$variable"        # \\
                        # Second and fourth escapes escaped.
                        # This is o.k.

Escaping a space can prevent word splitting in a command's argument list. 
file_list="/bin/cat /bin/gzip /bin/more /usr/bin/less /usr/bin/emacs-20.7"
# List of files as argument(s) to a command.

# Add two files to the list, and list all.
ls -l /usr/X11R6/bin/xsetroot /sbin/dump $file_list

echo "-------------------------------------------------------------------------"

# What happens if we escape a couple of spaces?
ls -l /usr/X11R6/bin/xsetroot\ /sbin/dump\ $file_list
# Error: the first three files concatenated into a single argument to 'ls -l'
#        because the two escaped spaces prevent argument (word) splitting.

The escape also provides a means of writing a multi-line command. Normally, each separate line constitutes a different command, but an escape at the end of a line escapes the newline character, and the command sequence continues on to the next line.

(cd /source/directory && tar cf - . ) | \
(cd /dest/directory && tar xpvf -)
# Repeating Alan Cox's directory tree copy command,

Can we help you?X

# but split into two lines for increased legibility.

# As an alternative:
tar cf - -C /source/directory . |
tar xpvf - -C /dest/directory
# See note below.
# (Thanks, Stéphane Chazelas.)

Can't find what you're looking for?X
Note

If a script line ends with a |, a pipe character, then a \, an escape, is not strictly necessary. It is, however, good programming practice to always escape the end of a line of code that continues to the following line.

echo "foo
bar" 
#foo
#bar

echo

echo 'foo
bar'    # No difference yet.
#foo
#bar

echo

echo foo\
bar     # Newline escaped.
#foobar

echo

echo "foo\
bar"     # Same here, as \ still interpreted as escape within weak quotes.
#foobar

echo

echo 'foo\
bar'     # Escape character \ taken literally because of strong quoting.
#foo\
#bar

# Examples suggested by Stéphane Chazelas.

Chapter 6. Exit and Exit Status

 

... there are dark corners in the Bourne shell, and people use all of them.

--Chet Ramey

The exit command terminates a script, just as in a C program. It can also return a value, which is available to the script's parent process.

Every command returns an exit status (sometimes referred to as a return status or exit code). A successful command returns a 0, while an unsuccessful one returns a non-zero value that usually can be interpreted as an error code. Well-behaved UNIX commands, programs, and utilities return a 0 exit code upon successful completion, though there are some exceptions.

Likewise, functions within a script and the script itself return an exit status. The last command executed in the function or script determines the exit status. Within a script, an exit nnn command may be used to deliver an nnn exit status to the shell (nnn must be an integer in the 0 - 255 range).

Don't know where to look next?X

Note

When a script ends with an exit that has no parameter, the exit status of the script is the exit status of the last command executed in the script (previous to the exit).

#!/bin/bash

COMMAND_1

. . .

COMMAND_LAST

# Will exit with status of last command.

exit

The equivalent of a bare exit is exit $? or even just omitting the exit.

#!/bin/bash

COMMAND_1

. . .

COMMAND_LAST

# Will exit with status of last command.

exit $?


#!/bin/bash

COMMAND1

. . . 

COMMAND_LAST

# Will exit with status of last command.

$? reads the exit status of the last command executed. After a function returns, $? gives the exit status of the last command executed in the function. This is Bash's way of giving functions a "return value." [28]

Confused? Frustrated?X

Following the execution of a pipe, a $? gives the exit status of the last command executed.

After a script terminates, a $? from the command-line gives the exit status of the script, that is, the last command executed in the script, which is, by convention, 0 on success or an integer in the range 1 - 255 on error.

Example 6-1. exit / exit status
#!/bin/bash

echo hello
echo $?    # Exit status 0 returned because command executed successfully.

lskdf      # Unrecognized command.
echo $?    # Non-zero exit status returned because command failed to execute.

echo

exit 113   # Will return 113 to shell.
           # To verify this, type "echo $?" after script terminates.

#  By convention, an 'exit 0' indicates success,
#+ while a non-zero exit value means an error or anomalous condition.

$? is especially useful for testing the result of a command in a script (see Example 15-35 and Example 15-20).

Note

The !, the logical not qualifier, reverses the outcome of a test or command, and this affects its exit status.

Example 6-2. Negating a condition using !
true    # The "true" builtin.
echo "exit status of \"true\" = $?"     # 0

! true
echo "exit status of \"! true\" = $?"   # 1
# Note that the "!" needs a space between it and the command.
#    !true   leads to a "command not found" error
#
# The '!' operator prefixing a command invokes the Bash history mechanism.

true
!true
# No error this time, but no negation either.
# It just repeats the previous command (true).


# =========================================================== #
# Preceding a _pipe_ with ! inverts the exit status returned.

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

ls | bogus_command     # bash: bogus_command: command not found
echo $?                # 127

! ls | bogus_command   # bash: bogus_command: command not found
echo $?                # 0
# Note that the ! does not change the execution of the pipe.
# Only the exit status changes.
# =========================================================== #

# Thanks, Stéphane Chazelas and Kristopher Newsome.

Do you need help?X
Caution

Certain exit status codes have reserved meanings and should not be user-specified in a script.

Chapter 7. Tests

Every reasonably complete programming language can test for a condition, then act according to the result of the test. Bash has the test command, various bracket and parenthesis operators, and the if/then construct.

7.1. Test Constructs

  • An if/then construct tests whether the exit status of a list of commands is 0 (since 0 means "success" by UNIX convention), and if so, executes one or more commands.

  • There exists a dedicated command called [ (left bracket special character). It is a synonym for test, and a builtin for efficiency reasons. This command considers its arguments as comparison expressions or file tests and returns an exit status corresponding to the result of the comparison (0 for true, 1 for false).

  • With version 2.02, Bash introduced the [[ ... ]] extended test command, which performs comparisons in a manner more familiar to programmers from other languages. Note that [[ is a keyword, not a command.

    Do you need more help?X

    Bash sees [[ $a -lt $b ]] as a single element, which returns an exit status.

  • The (( ... )) and let ... constructs also return an exit status, according to whether the arithmetic expressions they evaluate expand to a non-zero value. These arithmetic-expansion constructs may therefore be used to perform arithmetic comparisons.

    (( 0 && 1 ))                 # Logical AND
echo $?     # 1     ***
# And so ...
let "num = (( 0 && 1 ))"
echo $num   # 0
# But ...
let "num = (( 0 && 1 ))"
echo $?     # 1     ***


(( 200 || 11 ))              # Logical OR
echo $?     # 0     ***
# ...
let "num = (( 200 || 11 ))"
echo $num   # 1
let "num = (( 200 || 11 ))"
echo $?     # 0     ***


(( 200 | 11 ))               # Bitwise OR
echo $?                      # 0     ***
# ...
let "num = (( 200 | 11 ))"
echo $num                    # 203
let "num = (( 200 | 11 ))"
echo $?                      # 0     ***

# The "let" construct returns the same exit status
#+ as the double-parentheses arithmetic expansion.

  • An if can test any command, not just conditions enclosed within brackets.

    if cmp a b &> /dev/null  # Suppress output.
then echo "Files a and b are identical."
else echo "Files a and b differ."
fi

# The very useful "if-grep" construct:
# ----------------------------------- 
if grep -q Bash file
  then echo "File contains at least one occurrence of Bash."
fi

word=Linux
letter_sequence=inu
if echo "$word" | grep -q "$letter_sequence"
# The "-q" option to grep suppresses output.
then
  echo "$letter_sequence found in $word"
else
  echo "$letter_sequence not found in $word"
fi


if COMMAND_WHOSE_EXIT_STATUS_IS_0_UNLESS_ERROR_OCCURRED
  then echo "Command succeeded."
  else echo "Command failed."
fi

  • These last two examples courtesy of Stéphane Chazelas.

    Can we help you?X

Example 7-1. What is truth?
#!/bin/bash

#  Tip:
#  If you're unsure of how a certain condition would evaluate,
#+ test it in an if-test.

Can't find what you're looking for?X


echo

echo "Testing \"0\""
if [ 0 ]      # zero
then
  echo "0 is true."
else          # Or else ...
  echo "0 is false."
fi            # 0 is true.

echo

echo "Testing \"1\""
if [ 1 ]      # one
then
  echo "1 is true."
else
  echo "1 is false."
fi            # 1 is true.

echo

echo "Testing \"-1\""
if [ -1 ]     # minus one
then
  echo "-1 is true."
else
  echo "-1 is false."
fi            # -1 is true.

echo

echo "Testing \"NULL\""
if [ ]        # NULL (empty condition)
then
  echo "NULL is true."
else
  echo "NULL is false."
fi            # NULL is false.

echo

echo "Testing \"xyz\""
if [ xyz ]    # string
then
  echo "Random string is true."
else
  echo "Random string is false."
fi            # Random string is true.

echo

echo "Testing \"\$xyz\""
if [ $xyz ]   # Tests if $xyz is null, but...
              # it's only an uninitialized variable.
then
  echo "Uninitialized variable is true."
else
  echo "Uninitialized variable is false."
fi            # Uninitialized variable is false.

echo

echo "Testing \"-n \$xyz\""
if [ -n "$xyz" ]            # More pedantically correct.
then
  echo "Uninitialized variable is true."
else
  echo "Uninitialized variable is false."
fi            # Uninitialized variable is false.

echo


xyz=          # Initialized, but set to null value.

echo "Testing \"-n \$xyz\""
if [ -n "$xyz" ]
then
  echo "Null variable is true."
else
  echo "Null variable is false."
fi            # Null variable is false.


echo


# When is "false" true?

echo "Testing \"false\""
if [ "false" ]              #  It seems that "false" is just a string.
then
  echo "\"false\" is true." #+ and it tests true.
else
  echo "\"false\" is false."
fi            # "false" is true.

echo

echo "Testing \"\$false\""  # Again, uninitialized variable.
if [ "$false" ]
then
  echo "\"\$false\" is true."

Don't know where to look next?X

else
  echo "\"\$false\" is false."
fi            # "$false" is false.
              # Now, we get the expected result.

#  What would happen if we tested the uninitialized variable "$true"?

echo

exit 0

Exercise. Explain the behavior of Example 7-1, above.

if [ condition-true ]
then
   command 1
   command 2
   ...
else  # Or else ...
      # Adds default code block executing if original condition tests false.
   command 3
   command 4
   ...
fi

Note

When if and then are on same line in a condition test, a semicolon must terminate the if statement. Both if and then are keywords. Keywords (or commands) begin statements, and before a new statement on the same line begins, the old one must terminate.

if [ -x "$filename" ]; then

Else if and elif

Confused? Frustrated?X
elif

elif is a contraction for else if. The effect is to nest an inner if/then construct within an outer one.

if [ condition1 ]
then
   command1
   command2
   command3
elif [ condition2 ]
# Same as else if
then
   command4
   command5
else
   default-command
fi

The if test condition-true construct is the exact equivalent of if [ condition-true ]. As it happens, the left bracket, [ , is a token [29] which invokes the test command. The closing right bracket, ] , in an if/test should not therefore be strictly necessary, however newer versions of Bash require it.

Note

The test command is a Bash builtin which tests file types and compares strings. Therefore, in a Bash script, test does not call the external /usr/bin/test binary, which is part of the sh-utils package. Likewise, [ does not call /usr/bin/[, which is linked to /usr/bin/test.

bash$ type testtest is a shell builtinbash$ type '['[ is a shell builtinbash$ type '[['[[ is a shell keywordbash$ type ']]']] is a shell keywordbash$ type ']'bash: type: ]: not found

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

If, for some reason, you wish to use /usr/bin/test in a Bash script, then specify it by full pathname.

Example 7-2. Equivalence of test, /usr/bin/test, [ ], and /usr/bin/[
#!/bin/bash

echo

if test -z "$1"
then
  echo "No command-line arguments."
else
  echo "First command-line argument is $1."
fi

echo

if /usr/bin/test -z "$1"      # Equivalent to "test" builtin.
#  ^^^^^^^^^^^^^              # Specifying full pathname.
then
  echo "No command-line arguments."
else
  echo "First command-line argument is $1."
fi

echo

if [ -z "$1" ]                # Functionally identical to above code blocks.
#   if [ -z "$1"                should work, but...
#+  Bash responds to a missing close-bracket with an error message.
then
  echo "No command-line arguments."
else
  echo "First command-line argument is $1."
fi

echo


if /usr/bin/[ -z "$1" ]       # Again, functionally identical to above.
# if /usr/bin/[ -z "$1"       # Works, but gives an error message.
#                             # Note:
#                               This has been fixed in Bash, version 3.x.
then
  echo "No command-line arguments."
else
  echo "First command-line argument is $1."
fi

echo

exit 0

The [[ ]] construct is the more versatile Bash version of [ ]. This is the extended test command, adopted from ksh88.

* * *

No filename expansion or word splitting takes place between [[ and ]], but there is parameter expansion and command substitution. 
file=/etc/passwd

if [[ -e $file ]]
then
  echo "Password file exists."
fi

Using the [[ ... ]] test construct, rather than [ ... ] can prevent many logic errors in scripts. For example, the &&, ||, <, and > operators work within a [[ ]] test, despite giving an error within a [ ] construct.

Do you need help?X

Arithmetic evaluation of octal / hexadecimal constants takes place automatically within a [[ ... ]] construct. 
# [[ Octal and hexadecimal evaluation ]]
# Thank you, Moritz Gronbach, for pointing this out.


decimal=15
octal=017   # = 15 (decimal)

Do you need more help?X

hex=0x0f    # = 15 (decimal)

if [ "$decimal" -eq "$octal" ]
then
  echo "$decimal equals $octal"
else
  echo "$decimal is not equal to $octal"       # 15 is not equal to 017
fi      # Doesn't evaluate within [ single brackets ]!


if [[ "$decimal" -eq "$octal" ]]
then
  echo "$decimal equals $octal"                # 15 equals 017
else
  echo "$decimal is not equal to $octal"
fi      # Evaluates within [[ double brackets ]]!

if [[ "$decimal" -eq "$hex" ]]
then
  echo "$decimal equals $hex"                  # 15 equals 0x0f
else
  echo "$decimal is not equal to $hex"
fi      # [[ $hexadecimal ]] also evaluates!

Note

Following an if, neither the test command nor the test brackets ( [ ] or [[ ]] ) are strictly necessary. 
dir=/home/bozo

if cd "$dir" 2>/dev/null; then   # "2>/dev/null" hides error message.
  echo "Now in $dir."
else
  echo "Can't change to $dir."
fi
The "if COMMAND" construct returns the exit status of COMMAND.

Similarly, a condition within test brackets may stand alone without an if, when used in combination with a list construct. 
var1=20
var2=22
[ "$var1" -ne "$var2" ] && echo "$var1 is not equal to $var2"

home=/home/bozo
[ -d "$home" ] || echo "$home directory does not exist."

The (( )) construct expands and evaluates an arithmetic expression. If the expression evaluates as zero, it returns an exit status of 1, or "false". A non-zero expression returns an exit status of 0, or "true". This is in marked contrast to using the test and [ ] constructs previously discussed.

Example 7-3. Arithmetic Tests using (( ))
#!/bin/bash
# Arithmetic tests.

# The (( ... )) construct evaluates and tests numerical expressions.
# Exit status opposite from [ ... ] construct!

(( 0 ))
echo "Exit status of \"(( 0 ))\" is $?."         # 1

(( 1 ))
echo "Exit status of \"(( 1 ))\" is $?."         # 0

(( 5 > 4 ))                                      # true
echo "Exit status of \"(( 5 > 4 ))\" is $?."     # 0

(( 5 > 9 ))                                      # false
echo "Exit status of \"(( 5 > 9 ))\" is $?."     # 1

(( 5 - 5 ))                                      # 0
echo "Exit status of \"(( 5 - 5 ))\" is $?."     # 1

(( 5 / 4 ))                                      # Division o.k.
echo "Exit status of \"(( 5 / 4 ))\" is $?."     # 0

(( 1 / 2 ))                                      # Division result < 1.
echo "Exit status of \"(( 1 / 2 ))\" is $?."     # Rounded off to 0.

Can we help you?X

                                                 # 1

(( 1 / 0 )) 2>/dev/null                          # Illegal division by 0.
#           ^^^^^^^^^^^
echo "Exit status of \"(( 1 / 0 ))\" is $?."     # 1

# What effect does the "2>/dev/null" have?
# What would happen if it were removed?
# Try removing it, then rerunning the script.

# ======================================= #

# (( ... )) also useful in an if-then test.

var1=5
var2=4

if (( var1 > var2 ))
then #^      ^      Note: Not $var1, $var2. Why?
  echo "$var1 is greater than $var2"
fi     # 5 is greater than 4

exit 0

7.2. File test operators

Can't find what you're looking for?X

Returns true if...

-e

file exists

-a

file exists

This is identical in effect to -e. It has been "deprecated," [30] and its use is discouraged.

-f

file is a regular file (not a directory or device file)

-s

file is not zero size

-d

file is a directory

-b

file is a block device

Don't know where to look next?X

-c

file is a character device

device0="/dev/sda2"    # /   (root directory)
if [ -b "$device0" ]
then
  echo "$device0 is a block device."
fi

# /dev/sda2 is a block device.



device1="/dev/ttyS1"   # PCMCIA modem card.
if [ -c "$device1" ]
then
  echo "$device1 is a character device."
fi

# /dev/ttyS1 is a character device.

-p

file is a pipe

-h

file is a symbolic link

-L

file is a symbolic link

-S

file is a socket

-t

file (descriptor) is associated with a terminal device

Confused? Frustrated?X

This test option may be used to check whether the stdin [ -t 0 ] or stdout [ -t 1 ] in a given script is a terminal.

-r

file has read permission (for the user running the test)

-w

file has write permission (for the user running the test)

-x

file has execute permission (for the user running the test)

-g

set-group-id (sgid) flag set on file or directory

If a directory has the sgid flag set, then a file created within that directory belongs to the group that owns the directory, not necessarily to the group of the user who created the file. This may be useful for a directory shared by a workgroup.

-u

set-user-id (suid) flag set on file

Call Pantek today for Open Source Technical Support at 1-877-546-8934 - 24/7/365X

A binary owned by root with set-user-id flag set runs with root privileges, even when an ordinary user invokes it. [31] This is useful for executables (such as pppd and cdrecord) that need to access system hardware. Lacking the suid flag, these binaries could not be invoked by a non-root user.

	      
-rwsr-xr-t    1 root       178236 Oct  2  2000 /usr/sbin/pppd

A file with the suid flag set shows an s in its permissions.

-k

sticky bit set

Commonly known as the sticky bit, the save-text-mode flag is a special type of file permission. If a file has this flag set, that file will be kept in cache memory, for quicker access. [32] If set on a directory, it restricts write permission. Setting the sticky bit adds a t to the permissions on the file or directory listing.

	      
drwxrwxrwt    7 root         1024 May 19 21:26 tmp/

If a user does not own a directory that has the sticky bit set, but has write permission in that directory, she can only delete those files that she owns in it. This keeps users from inadvertently overwriting or deleting each other's files in a publicly accessible directory, such as /tmp. (The owner of the directory or root can, of course, delete or rename files there.)

-O

you are owner of file

Do you need help?X
-G

group-id of file same as yours

-N

file modified since it was last read

f1 -nt f2

file f1 is newer than f2

f1 -ot f2

file f1 is older than f2

f1 -ef f2

files f1 and f2 are hard links to the same file

!

"not" -- reverses the sense of the tests above (returns true if condition absent).

Example 7-4. Testing for broken links
#!/bin/bash
# broken-link.sh
# Written by Lee bigelow 
# Used in ABS Guide with permission.

#  A pure shell script to find dead symlinks and output them quoted
#+ so they can be fed to xargs and dealt with :)
#+ eg. sh broken-link.sh /somedir /someotherdir|xargs rm
#
#  This, however, is a better method:
#
#  find "somedir" -type l -print0|\
#  xargs -r0 file|\
#  grep "broken symbolic"|
#  sed -e 's/^\|: *broken symbolic.*$/"/g'
#
#+ but that wouldn't be pure Bash, now would it.
#  Caution: beware the /proc file system and any circular links!
################################################################


#  If no args are passed to the script set directories-to-search 
#+ to current directory.  Otherwise set the directories-to-search 
#+ to the args passed.
######################

[ $# -eq 0 ] && directorys=`pwd` || directorys=$@


#  Setup the function linkchk to check the directory it is passed 

Do you need more help?X