Input/Output Redirection

Overview

In Linux, every process has three standard data streams: standard input, standard output, and standard error. Redirection allows us to change the sources and destinations of these data streams.

Standard Data Streams

                     ┌─────────────────┐
    Standard Input (stdin)  │                 │ Standard Output (stdout)
    File Descriptor: 0    │                 │ File Descriptor: 1
    ─────────────────►     Process       ├─────────────────►
                     │                 │
                     │                 │ Standard Error (stderr)
                     │                 │ File Descriptor: 2
                     └────────┬────────┘
                              │
                              ▼

Stream	File Descriptor	Default Device	Description
stdin	0	Keyboard	Program input
stdout	1	Screen	Program normal output
stderr	2	Screen	Program error output

Output Redirection

Redirect to File (Overwrite)

Use > to redirect standard output to a file, which overwrites the file content:

bash

# Write command output to file
$ echo "Hello, World!" > output.txt

# View result
$ cat output.txt
Hello, World!

# Writing again overwrites
$ echo "New content" > output.txt
$ cat output.txt
New content

# Save command output to file
$ ls -la > filelist.txt
$ date > current_date.txt

Append to File

Use >> to append output to end of file:

bash

# Append content
$ echo "Line 1" > file.txt
$ echo "Line 2" >> file.txt
$ echo "Line 3" >> file.txt

$ cat file.txt
Line 1
Line 2
Line 3

# Append logs
$ date >> log.txt
$ echo "Task completed" >> log.txt

Redirect Standard Error

Use 2> to redirect standard error:

bash

# Write error messages to file
$ ls /nonexistent 2> error.txt

$ cat error.txt
ls: cannot access '/nonexistent': No such file or directory

# Append error messages
$ ls /another_nonexistent 2>> error.txt

Redirect Both Output and Error

bash

# Method 1: Redirect separately
$ command > output.txt 2> error.txt

# Method 2: Merge to same file
$ command > all.txt 2>&1

# Method 3: Shorthand (Bash 4+)
$ command &> all.txt

# Method 4: Append mode
$ command >> all.txt 2>&1
$ command &>> all.txt

Discard Output

Redirect output to /dev/null:

bash

# Discard standard output
$ command > /dev/null

# Discard standard error
$ command 2> /dev/null

# Discard all output
$ command > /dev/null 2>&1
$ command &> /dev/null

Redirection Order

Redirection order matters:

bash

# Correct: Redirect stdout first, then point stderr to stdout
$ command > file.txt 2>&1

# Incorrect: Wrong order, stderr still points to screen
$ command 2>&1 > file.txt

Input Redirection

Read from File

Use < to read input from a file:

bash

# Read from file
$ wc -l < file.txt
10

# Sort file content
$ sort < unsorted.txt

# Combine with output redirection
$ sort < unsorted.txt > sorted.txt

Here Document (Here Doc)

Use << to provide multi-line input:

bash

# Basic syntax
$ cat << EOF
Line 1
Line 2
Line 3
EOF

# Write to file
$ cat << EOF > file.txt
This is line 1
This is line 2
This is line 3
EOF

# Pass to command
$ mysql -u root << EOF
USE database;
SELECT * FROM table;
EOF

Terminator Description

EOF is a custom terminator, can be any string
Terminator must be on its own line
Terminator cannot have spaces before or after (unless using <<-)

Here String

Use <<< to provide single-line string input:

bash

# Basic syntax
$ cat <<< "Hello, World!"
Hello, World!

# Pass variable
$ name="Alice"
$ cat <<< "Hello, $name!"
Hello, Alice!

# As command input
$ bc <<< "2 + 3"
5

$ grep "pattern" <<< "This line contains pattern"
This line contains pattern

File Descriptors

Understanding File Descriptors

bash

# 0 = stdin (standard input)
# 1 = stdout (standard output)
# 2 = stderr (standard error)
# 3-9 = Available for custom use

# View process file descriptors
$ ls -l /proc/$$/fd/
lrwx------ 1 user user 64 Jan 1 10:00 0 -> /dev/pts/0
lrwx------ 1 user user 64 Jan 1 10:00 1 -> /dev/pts/0
lrwx------ 1 user user 64 Jan 1 10:00 2 -> /dev/pts/0

Use Custom File Descriptors

bash

# Open file descriptor 3 for writing
$ exec 3> output.txt
$ echo "Hello" >&3
$ echo "World" >&3
$ exec 3>&-  # Close file descriptor

# Open file descriptor 4 for reading
$ exec 4< input.txt
$ read line <&4
$ echo $line
$ exec 4<&-  # Close file descriptor

# Read/write simultaneously
$ exec 3<> file.txt

Copy File Descriptors

bash

# Point fd 3 to fd 1
$ exec 3>&1

# Point fd 4 to fd 0
$ exec 4<&0

# Point stderr to stdout
$ command 2>&1

Practical Examples

Logging

bash

#!/bin/bash
# Script execution logging

LOGFILE="script.log"

# Redirect all output to log file
exec > >(tee -a "$LOGFILE") 2>&1

echo "Script starts: $(date)"
# ... script content ...
echo "Script completes: $(date)"

Separate Output and Error

bash

# Normal output to one file, errors to another
$ find / -name "*.conf" > found.txt 2> errors.txt

# View only normal output, ignore errors
$ find / -name "*.conf" 2> /dev/null

Display and Save Output Simultaneously

Use tee command:

bash

# Display and save to file
$ ls -la | tee filelist.txt

# Append mode
$ ls -la | tee -a filelist.txt

# Save to multiple files
$ ls -la | tee file1.txt file2.txt

# Include stderr
$ command 2>&1 | tee output.txt

Input/Output Combinations

bash

# Read from file, process, write to another
$ sort < unsorted.txt > sorted.txt

# Multiple inputs
$ cat < file1.txt < file2.txt  # Only reads last one

# Correct way
$ cat file1.txt file2.txt > combined.txt

Create Empty File or Clear File

bash

# Create empty file
$ > newfile.txt

# Clear existing file
$ > existingfile.txt

# Use : command
$ : > file.txt

Append Timestamp to Logs

bash

# Add timestamp to each line
$ command | while read line; do
    echo "$(date '+%Y-%m-%d %H:%M:%S') $line"
done >> logfile.txt

noclobber Option

Prevent accidental file overwriting:

bash

# Enable noclobber
$ set -o noclobber

# Now > cannot overwrite existing file
$ echo "test" > existingfile.txt
bash: existingfile.txt: cannot overwrite existing file

# Force overwrite (use >|)
$ echo "test" >| existingfile.txt

# Disable noclobber
$ set +o noclobber

Advanced Redirection

Swap stdout and stderr

bash

# Swap stdout and stderr
$ command 3>&1 1>&2 2>&3

# Practical application: pipe stderr to grep
$ command 3>&1 1>&2 2>&3 | grep "error"

Process Substitution

Use <() and >():

bash

# Use command output as file
$ diff <(ls dir1) <(ls dir2)

# Distribute input to multiple commands
$ echo "hello" | tee >(cat -n) >(wc -c)

# Compare two command outputs
$ comm <(sort file1.txt) <(sort file2.txt)

Read to Variable

bash

# Read command output into variable
$ result=$(command)

# Read file to variable
$ content=$(< file.txt)

# Read first line
$ read first_line < file.txt

Common Errors and Pitfalls

Pitfall 1: Input and Output Using Same File

bash

# Wrong: Will clear the file!
$ sort < file.txt > file.txt

# Correct way
$ sort file.txt > temp.txt && mv temp.txt file.txt

# Or use sponge (requires moreutils)
$ sort file.txt | sponge file.txt

Pitfall 2: Variables in Pipes

bash

# Variables modified in sub-shell, won't affect parent shell
$ count=0
$ cat file.txt | while read line; do
    ((count++))
done
$ echo $count  # Still 0

# Correct way
$ count=0
$ while read line; do
    ((count++))
done < file.txt
$ echo $count

Pitfall 3: Redirection Order

bash

# Wrong order
$ command 2>&1 > file.txt
# stderr still outputs to terminal

# Correct order
$ command > file.txt 2>&1
# stdout and stderr both write to file

Summary

This chapter introduced Linux input/output redirection:

Output redirection: > (overwrite), >> (append)
Error redirection: 2>, 2>>
Merge redirection: &>, 2>&1
Input redirection: <, << (Here Doc), <<< (Here String)
File descriptors: 0 (stdin), 1 (stdout), 2 (stderr)
Practical tools: tee, /dev/null

Mastering redirection is an important skill for using Linux command-line, enabling you to flexibly handle data streams and automate tasks.

Previous chapter: Shell Introduction

Next chapter: Pipes and Filters

Input/Output Redirection ​

Overview ​

Standard Data Streams ​

Output Redirection ​

Redirect to File (Overwrite) ​

Append to File ​

Redirect Standard Error ​

Redirect Both Output and Error ​

Discard Output ​

Redirection Order ​

Input Redirection ​

Read from File ​

Here Document (Here Doc) ​

Here String ​

File Descriptors ​

Understanding File Descriptors ​

Use Custom File Descriptors ​

Copy File Descriptors ​

Practical Examples ​

Logging ​

Separate Output and Error ​

Display and Save Output Simultaneously ​

Input/Output Combinations ​

Create Empty File or Clear File ​

Append Timestamp to Logs ​

noclobber Option ​

Advanced Redirection ​

Swap stdout and stderr ​

Process Substitution ​

Read to Variable ​

Common Errors and Pitfalls ​

Pitfall 1: Input and Output Using Same File ​

Pitfall 2: Variables in Pipes ​

Pitfall 3: Redirection Order ​

Summary ​

Input/Output Redirection

Overview

Standard Data Streams

Output Redirection

Redirect to File (Overwrite)

Append to File

Redirect Standard Error

Redirect Both Output and Error

Discard Output

Redirection Order

Input Redirection

Read from File

Here Document (Here Doc)

Here String

File Descriptors

Understanding File Descriptors

Use Custom File Descriptors

Copy File Descriptors

Practical Examples

Logging

Separate Output and Error

Display and Save Output Simultaneously

Input/Output Combinations

Create Empty File or Clear File

Append Timestamp to Logs

noclobber Option

Advanced Redirection

Swap stdout and stderr

Process Substitution

Read to Variable

Common Errors and Pitfalls

Pitfall 1: Input and Output Using Same File

Pitfall 2: Variables in Pipes

Pitfall 3: Redirection Order

Summary