Quoting, Word Splitting & Expansion

If you fix one class of bug in your scripts, make it this one. Unquoted variables are the most common defect in shell code, they pass every test you write with tidy input, and they fail the first time a filename contains a space.

Topic 1: The Expansion Pipeline

Before any command runs, the shell rewrites the line you typed. It happens in a fixed order, and knowing the order explains nearly every surprising result.

  1. Brace expansion        {a,b}      →  a b
  2. Tilde expansion        ~          →  /home/deploy
  3. Parameter expansion    $var       →  the value
  4. Command substitution   $(cmd)     →  the output
  5. Arithmetic expansion   $(( ))     →  the number
  6. WORD SPLITTING         on IFS     →  splits the RESULT of 3-5
  7. Pathname expansion     *.log      →  matching filenames
  8. Quote removal

Two facts drive everything in this lesson:

• Steps 6 and 7 happen after step 3. The shell substitutes your variable’s value, and then splits and globs whatever came out. It does not care that the value came from a variable.
• Quoting suppresses steps 6 and 7. That is the entire mechanism. "$var" is not “the string version” of $var — it is $var with splitting and globbing switched off.

Topic 2: Word Splitting

After substitution, the shell splits the result on every character in IFS (Internal Field Separator), which defaults to space, tab, and newline.

file="my report.txt"
rm $file        # becomes: rm my report.txt   -- TWO arguments
rm "$file"      # becomes: rm "my report.txt" -- one argument

The first form tells rm to delete two files that do not exist. It is not a syntax error, nothing warns you, and on a good day it simply fails.

Why this bites in production:

# Looks fine. Works in testing. Deletes the wrong thing eventually.
for f in $(ls /var/log); do
    process $f
done

Two bugs here. $(ls) output is split on whitespace, so access log.txt becomes two iterations. And ls output is unparseable in principle — it mangles non-printing characters. The correct form uses a glob, which never splits:

for f in /var/log/*; do
    process "$f"
done

Where splitting does NOT happen:

It is worth knowing the exempt contexts, because they explain why some unquoted code works:

• Inside [[ ]] — the bash conditional keyword does not split.
• On the right of = in an assignment: x=$y is safe.
• Inside $(( )) arithmetic.
• In a case word.

Everywhere else, quote. The exemptions are not worth memorising as licence to omit quotes; they are worth knowing so that reading other people’s code makes sense.

Topic 3: Pathname Expansion (Globbing)

After splitting, each resulting word containing *, ? or [ is matched against filenames.

pattern="*.log"
echo $pattern     # prints every .log file in the directory
echo "$pattern"   # prints the literal string *.log

This is why an unquoted variable holding a * behaves differently depending on what happens to be in the current directory — a bug that reproduces on one machine and not another.

The two options worth knowing:

shopt -s nullglob    # a pattern matching nothing expands to NOTHING
shopt -s failglob    # a pattern matching nothing is an ERROR
shopt -s dotglob     # * also matches hidden files
shopt -s globstar    # ** recurses into subdirectories

By default, a glob that matches nothing is passed through literally, which produces the classic:

for f in /empty/dir/*.log; do
    echo "$f"        # prints: /empty/dir/*.log   -- a file that does not exist
done

nullglob makes that loop run zero times, which is almost always what you meant.

Topic 4: The Three Quoting Forms

name="world"
echo 'Hello $name'      # Hello $name
echo "Hello $name"      # Hello world
echo Hello\ \$name      # Hello $name

Single quotes cannot contain a single quote.

There is no escape inside them. To include one, end the quoting, add an escaped quote, and start again:

echo 'it'\''s here'     # it's here

That is four tokens the shell concatenates: it, ', s here. Ugly, and unavoidable.

Nesting quotes inside command substitution:

Quotes reset inside $( ), so this is legal and correct:

echo "Found $(grep -c "ERROR" "$logfile") errors"

The inner "ERROR" and "$logfile" are their own context. This is a good reason to prefer $( ) over backticks, where escaping rules are painful.

Topic 5: "$@" vs $@ vs "$*"

This is the one that separates people who have been bitten from people who have not.

# Called as:  ./script.sh one "two three" four

"$@"    →  "one" "two three" "four"     # 3 args, boundaries PRESERVED
$@      →  "one" "two" "three" "four"   # 4 args, boundaries DESTROYED
"$*"    →  "one two three four"         # 1 arg, joined by IFS[0]
$*      →  "one" "two" "three" "four"   # 4 args, same as $@ unquoted

"$@" is the only correct way to forward arguments. It is special-cased in the shell grammar: it expands to one word per argument, with each word quoted individually. Nothing else does this.

# Correct wrapper
run_with_logging() {
    echo "running: $*"      # $* is fine for DISPLAY
    command "$@"            # "$@" is required for EXECUTION
}

That pairing is the idiom worth memorising: $* when you are printing, "$@" when you are running. The same distinction applies to array expansion — "${arr[@]}" preserves elements, "${arr[*]}" joins them into one string.

Topic 6: Parameter Expansion Beyond $var

The shell has a substantial string-manipulation language built into ${ }, and it runs without forking a process — which makes it dramatically faster than piping to sed in a loop.

Defaults and error handling:

: "${LOG_DIR:=/var/log/myapp}"      # set a default once, at the top
: "${API_TOKEN:?must be set}"       # fail fast with a clear message

The leading : is the null command — it does nothing but let the expansion’s side effect happen. ${var:?} is the cheapest input validation in shell, and it exits with a message naming the variable.

String manipulation:

The mnemonic for # and %: on a US keyboard # is left of %, and # strips from the left while % strips from the right. Doubling the character makes the match greedy.

path="/var/log/nginx/access.log"
echo "${path##*/}"      # access.log      -- basename, no fork
echo "${path%/*}"       # /var/log/nginx  -- dirname, no fork

Both replace an external command. In a loop over ten thousand files, that is ten thousand processes you did not start.

Try it yourself: Set f="/etc/nginx/nginx.conf" and produce nginx.conf, /etc/nginx, and conf using only parameter expansion.

Common mistake: Using ${var:-default} and expecting it to set the variable. It substitutes at that one point only. Use := to assign, and remember := does not work on positional parameters like $1.

Topic 7: The Quoting Checklist

Run these against any script you already have:

1. Every $var inside a command is "$var". The exceptions are rare and deliberate.
2. "$@", never $@ or $*, when passing arguments onward.
3. "${arr[@]}" when expanding arrays.
4. Never parse ls. Use a glob, or find -print0 piped to while IFS= read -r -d ''.
5. read gets -r — without it, backslashes in the input are eaten.
6. IFS= before read to stop leading and trailing whitespace being stripped.

# The safe file-reading idiom, all six rules applied
while IFS= read -r line; do
    printf '%s\n' "$line"
done < "$input_file"

ShellCheck flags most of this automatically — SC2086 is the unquoted-variable warning and it is the most frequently triggered rule in the entire tool. Running it is covered in the linting lesson; the point here is that the rule exists because this mistake is that common.

Common mistake: Adding quotes only where a bug appeared. The value of the rule is that it is unconditional — a variable that holds a safe value today holds user input tomorrow.