--- title: "Deleting Lines with Sed: Advanced Techniques" description: "Master sed command for deleting lines - remove specific lines, patterns, and ranges with practical examples and best practices, including GNU vs BSD sed notes." date: 2025-12-10 categories: ["vps"] tags: ["sed","linux-commands"] --- Need to remove specific lines from text files quickly? `sed` makes line deletion simple. Whether you're cleaning log files, removing comments, or filtering data, sed deletion techniques will help you process text efficiently. Most Linux distributions ship with **GNU sed**, while macOS uses **BSD sed**. The biggest practical difference in this article is in-place editing: - GNU sed: `sed -i.bak '...' file` - BSD sed (macOS): `sed -i '.bak' '...' file` (backup extension is required) If you want a portable approach, write to a temp file and move it into place (examples below). ## What is sed and Why Use It for Line Deletion? **sed** (Stream Editor) is a command-line utility for filtering and transforming text in Unix-like systems. It works well for line deletion because it processes text efficiently without loading entire files into memory. ### Key Advantages for Line Deletion **Non-interactive processing** - Make changes without opening text editors - Works well for automation and shell scripts - Handles large files and batch operations - Integrates with other Unix tools via pipes **Flexible targeting** - Delete lines with precision - Line numbers: `sed '5d'` deletes line 5 - Line ranges: `sed '10,20d'` deletes lines 10-20 - Pattern matching: `sed '/error/d'` deletes lines containing "error" - Regular expressions: Advanced pattern matching for complex criteria **Performance**: - Speed: Processes files rapidly, even large datasets - Memory efficiency: Streams data line by line - Scriptable: Automates repetitive deletion tasks ### Why Choose sed for Line Deletion? - Target exact lines, patterns, or ranges - Handle massive files without performance issues - Combine with other commands for complex workflows - Available on virtually all Unix systems Master sed's complete toolkit for text manipulation: - [Delete lines](https://www.bitdoze.com/sed-delete-lines/) using `d` command (this guide) - [Insert or append text](https://www.bitdoze.com/sed-insert-append-text/) with `i` and `a` commands - [Transform text case](https://www.bitdoze.com/sed-change-case/) for standardization - [Search and replace text](https://www.bitdoze.com/sed-search-replace/) with pattern matching ## Basic sed Syntax for Line Deletion Understanding sed's syntax matters for effective line deletion. The basic structure combines addresses (which lines to target) with the delete command (`d`). ### Core Components **Basic syntax:** ```shell sed 'ADDRESS d' filename ``` - **ADDRESS**: Specifies which lines to delete - **d**: The delete command - **filename**: Target file (or stdin if omitted) ### Addressing Methods **1. Line numbers:** ```shell sed '5d' file.txt # Delete line 5 sed '1d' file.txt # Delete first line sed '$d' file.txt # Delete last line ``` **2. Line ranges:** ```shell sed '2,5d' file.txt # Delete lines 2-5 sed '10,$d' file.txt # Delete from line 10 to end sed '1,3d' file.txt # Delete first 3 lines ``` **3. Pattern matching:** ```shell sed '/error/d' file.txt # Delete lines containing "error" sed '/^#/d' file.txt # Delete lines starting with # sed '/^$/d' file.txt # Delete empty lines ``` **4. Regular expressions:** ```shell sed '/^[0-9]/d' file.txt # Delete lines starting with digits sed '/\.log$/d' file.txt # Delete lines ending with .log ``` ### Key Options | Option | Function | Example | |--------|----------|---------| | `-i` | Edit files in-place | GNU: `sed -i '1d' file.txt` / BSD: `sed -i '' '1d' file.txt` | | `-i.bak` | Edit in-place with backup | GNU: `sed -i.bak '1d' file.txt` / BSD: `sed -i '.bak' '1d' file.txt` | | `-n` | Suppress default output | Used with `p` command | ### Execution Flow sed operates in a simple cycle: 1. **Read** a line into pattern space 2. **Apply** commands (like delete) 3. **Output** remaining lines (unless deleted) 4. **Repeat** for next line **Important**: sed processes each line independently, making it good for stream processing and large files. ## Deleting Specific Lines sed makes targeting and deleting specific lines straightforward with its flexible addressing system. ### Delete by Line Number **Single line deletion:** ```shell sed '2d' filename # Delete line 2 sed '1d' filename # Delete first line sed '$d' filename # Delete last line ``` **Multiple specific lines:** ```shell sed '1d;3d;5d' filename # Delete lines 1, 3, and 5 sed -e '2d' -e '5d' filename # Alternative syntax ``` ### Delete Line Ranges **Continuous ranges:** ```shell sed '10,20d' filename # Delete lines 10-20 sed '1,5d' filename # Delete first 5 lines sed '10,$d' filename # Delete from line 10 to end ``` **Practical examples:** ```shell sed '1d' config.txt # Remove header line sed '$d' data.txt # Remove footer/last line sed '2,4d' log.txt # Remove lines 2-4 ``` ### Delete by Pattern Matching **Simple patterns:** ```shell sed '/error/d' logfile.txt # Delete lines containing "error" sed '/^#/d' config.txt # Delete comment lines sed '/^$/d' file.txt # Delete empty lines ``` **Case sensitivity:** ```shell sed '/Error/d' file.txt # Case-sensitive (matches "Error") sed '/[Ee]rror/d' file.txt # Matches "Error" or "error" ``` ### Advanced Pattern Examples **Lines starting with specific characters:** ```shell sed '/^[0-9]/d' file.txt # Delete lines starting with digits sed '/^[A-Z]/d' file.txt # Delete lines starting with uppercase ``` **Lines ending with patterns:** ```shell sed '/\.log$/d' file.txt # Delete lines ending with ".log" sed '/;$/d' code.txt # Delete lines ending with semicolon ``` **Complex patterns:** ```shell sed '/^[[:space:]]*$/d' file.txt # Delete blank lines (including whitespace) sed '/^[[:space:]]*#/d' file.txt # Delete comment lines with leading spaces ``` ### Safety and Testing **Preview changes first:** ```shell sed '2d' file.txt # Shows output without modifying file sed '2d' file.txt > new.txt # Save to new file ``` **In-place editing with backup:** ```shell # GNU sed (Linux) sed -i.bak '2d' file.txt # Creates file.txt.bak # BSD sed (macOS) sed -i '.bak' '2d' file.txt # Creates file.txt.bak ``` **Test with line numbers:** ```shell nl file.txt | sed '2d' # Show line numbers to verify targeting ``` ### Practical Use Cases - Log cleaning: `sed '/DEBUG/d' app.log` - Config files: `sed '/^#/d' nginx.conf` - Data processing: `sed '1d' data.csv` (remove CSV header) - Code cleanup: `sed '/^\/\//d' script.js` (remove JS comments) **Pro tip**: Always test your sed commands on sample data before applying to important files. ## Pattern-Based Line Deletion with Regular Expressions sed's pattern matching with regular expressions provides powerful tools for deleting lines based on content rather than position. ### Basic Pattern Matching **Simple string patterns:** ```sh sed '/error/d' filename # Delete lines containing "error" sed '/warning/d' logfile.txt # Delete lines with "warning" sed '/DEBUG/d' app.log # Delete debug messages ``` **Word boundaries for exact matches:** ```sh # Note: \b word boundaries are not portable across sed implementations. # Portable alternatives: # 1) Use extended regex with explicit "word boundaries" built from non-word characters. # Keep in mind this defines "word" as [A-Za-z0-9_]. sed -E '/(^|[^[:alnum:]_])error([^[:alnum:]_]|$)/d' filename sed -E '/(^|[^[:alnum:]_])test([^[:alnum:]_]|$)/d' file.txt # 2) If you only care about whitespace boundaries: sed -E '/(^|[[:space:]])error([[:space:]]|$)/d' filename ``` ### Regular Expression Patterns **Character classes:** ```sh sed '/[0-9]/d' filename # Delete lines containing any digit sed '/[A-Z]/d' filename # Delete lines with uppercase letters sed '/[aeiou]/d' filename # Delete lines containing vowels ``` **Anchors (position matching):** ```sh sed '/^error/d' filename # Delete lines starting with "error" sed '/error$/d' filename # Delete lines ending with "error" sed '/^[0-9]/d' filename # Delete lines starting with digits ``` **Quantifiers:** ```sh sed '/[0-9]\{3\}/d' filename # Delete lines with 3+ consecutive digits sed '/^.\{80,\}/d' filename # Delete lines longer than 80 chars sed '/error.*critical/d' file # Delete lines with "error" followed by "critical" ``` ### Advanced Pattern Examples **Empty and whitespace lines:** ```sh sed '/^$/d' filename # Delete empty lines sed '/^[[:space:]]*$/d' filename # Delete blank lines (including whitespace) # Note: \s is not portable in sed regex. Prefer POSIX character classes: sed '/^[[:space:]]*$/d' filename # Whitespace-only lines (portable) ``` **Comment patterns:** ```sh sed '/^#/d' config.txt # Delete lines starting with # sed '/^[[:space:]]*#/d' file.txt # Delete comments with leading whitespace sed '/^\/\//d' script.js # Delete JavaScript comments sed '/^\/\*/d' style.css # Delete CSS comment starts ``` **Complex patterns:** ```sh sed '/^[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}/d' file.txt # Delete date lines (YYYY-MM-DD) sed '/^[a-zA-Z0-9._%+-]\+@[a-zA-Z0-9.-]\+\.[a-zA-Z]\{2,\}/d' file.txt # Delete email lines ``` ### Negation and Inverse Matching **Keep only matching lines (delete non-matches):** ```sh sed '/success/!d' filename # Keep only lines with "success" sed '/^#/!d' config.txt # Keep only comment lines sed '/error/!d' log.txt # Keep only error lines ``` ### Extended Regular Expressions **Using sed -E for enhanced patterns:** ```sh sed -E '/^[0-9]{3}-[0-9]{3}-[0-9]{4}$/d' file.txt # Delete phone numbers sed -E '/^(http|https):/d' urls.txt # Delete HTTP URLs sed -E '/^[A-Z]{2,}/d' file.txt # Delete lines starting with 2+ caps ``` ### Practical Examples **Log file cleanup:** ```sh sed '/INFO/d' app.log # Remove info messages sed '/^\[.*DEBUG.*\]/d' log # Remove debug entries sed '/^$/d' access.log # Remove empty lines ``` **Configuration files:** ```sh sed '/^#/d' nginx.conf # Remove comments sed '/^[[:space:]]*$/d' config.ini # Remove blank lines ``` **Data processing:** ```sh sed '/^test/d' data.txt # Remove test entries sed '/,$/d' csv.txt # Remove lines ending with comma ``` ### Safety and Testing **Preview patterns before deletion:** ```sh grep 'pattern' filename # See what will be deleted sed -n '/pattern/p' filename # Print matching lines ``` **Test with line numbers:** ```sh nl filename | sed '/pattern/d' # Show line numbers for context ``` **Common mistakes to avoid:** - Forgetting to escape special characters: `/./d` vs `/\./d` - Case sensitivity: Use `[Ee]rror` for both cases - Overly broad patterns: Use word boundaries `\b` when needed **Pro tip**: Regular expressions are powerful but can be tricky. Test your patterns thoroughly before applying to important files. ## Advanced Line Deletion Techniques For complex text processing tasks, sed offers features that go beyond basic pattern matching and line ranges. ### Range-Based Pattern Deletion **Delete between pattern markers:** ```sh sed '/START/,/END/d' file.txt # Delete from START to END markers sed '/BEGIN/,/FINISH/d' config.txt # Delete configuration blocks sed '//d' html.txt # Delete HTML comments ``` **Delete from pattern to line number:** ```sh sed '/ERROR/,10d' file.txt # Delete from first ERROR to line 10 sed '5,/STOP/d' file.txt # Delete from line 5 to first STOP ``` **Delete from pattern to end of file:** ```sh sed '/FOOTER/,$d' file.txt # Delete from FOOTER to end sed '/^---/,$d' document.txt # Delete from separator to end ``` ### Multi-Line Pattern Deletion **Delete paragraph blocks:** ```sh sed '/^$/,/^$/d' file.txt # Delete empty line blocks sed '/^[[:space:]]*$/,/^[[:space:]]*$/d' file.txt # Include whitespace-only lines ``` **Delete function definitions (example in code):** ```sh sed '/^function/,/^}/d' script.js # Delete JavaScript functions sed '/^def /,/^$/d' script.py # Delete Python function definitions ``` ### Conditional Deletion **Delete lines matching multiple conditions:** ```sh sed '/error.*critical/d' log.txt # Delete lines with both "error" and "critical" sed '/^[0-9].*error/d' file.txt # Delete lines starting with digit and containing "error" ``` **Delete except specific patterns:** ```sh sed '/INFO\|WARN\|ERROR/!d' log.txt # Keep only log levels, delete everything else sed '/^[A-Za-z]/!d' file.txt # Keep only lines starting with letters ``` ### Using Address Ranges with Steps **Delete every nth line:** ```sh sed '1~2d' file.txt # Delete every odd line (1st, 3rd, 5th...) sed '2~3d' file.txt # Delete every 3rd line starting from line 2 sed '0~5d' file.txt # Delete every 5th line ``` ### Working with Hold and Pattern Space **Delete duplicate consecutive lines:** ```sh sed '$!N; /^\(.*\)\n\1$/d' file.txt # Remove consecutive duplicate lines ``` **Delete lines based on next line content:** ```sh sed '$!N; /error\n/d' file.txt # Delete lines followed by lines containing "error" ``` ### Complex Multi-Command Operations **Combine multiple deletion criteria:** ```sh sed -e '/^#/d' -e '/^$/d' -e '/DEBUG/d' file.txt # Remove comments, empty lines, and debug ``` **Script-based complex deletion:** ```sh sed -f delete_script.sed file.txt ``` Where `delete_script.sed` contains: ``` /^#/d /^$/d /DEBUG/d /^[[:space:]]*$/d ``` ### Practical Advanced Examples **Clean log files:** ```sh # Remove debug, empty lines, and timestamp lines sed -e '/DEBUG/d' -e '/^$/d' -e '/^\[.*\]$/d' app.log ``` **Process configuration files:** ```sh # Remove comments and empty lines, keep only active config sed -e '/^[[:space:]]*#/d' -e '/^[[:space:]]*$/d' nginx.conf ``` **Code cleanup:** ```sh # Remove empty lines, single-line comments, and console.log statements sed -e '/^$/d' -e '/^[[:space:]]*\/\//d' -e '/console\.log/d' script.js ``` ### Advanced Safety Practices **Test complex commands step by step:** ```sh # Step 1: Test first condition sed '/^#/d' file.txt | head -20 # Step 2: Add second condition sed -e '/^#/d' -e '/^$/d' file.txt | head -20 # Step 3: Add final conditions sed -e '/^#/d' -e '/^$/d' -e '/DEBUG/d' file.txt | head -20 ``` **Use intermediate files for complex operations:** ```sh sed '/START/,/END/d' file.txt > temp1.txt sed '/ERROR/d' temp1.txt > temp2.txt sed '/^$/d' temp2.txt > final.txt ``` **Backup and restore capabilities:** ```sh cp original.txt original.txt.backup sed -i.$(date +%Y%m%d) 'complex_deletion_commands' original.txt ``` ### Performance Considerations - **Large files**: Use specific patterns rather than broad matches - **Multiple files**: Combine operations where possible - **Memory usage**: Complex hold space operations can consume memory - **Speed**: Simple line number ranges are faster than complex regex patterns **Pro tip**: For extremely complex deletion logic, consider combining sed with other tools like awk or writing a custom script for better maintainability. ## Conclusion sed's line deletion capabilities give you efficient tools for text processing. You can: - Delete specific lines by number, range, or pattern - Use regular expressions for precise pattern matching - Handle complex scenarios with advanced techniques - Process files safely with proper testing and backups ### Key Takeaways 1. Start simple with line numbers and basic patterns 2. Test first and preview changes before applying them permanently 3. Use backups with `-i.bak` for in-place editing safety 4. Combine different addressing methods for complex tasks 5. Practice regularly to build proficiency ### Best Practices - Test commands without `-i` flag first - Create backups before in-place editing - Use specific patterns to avoid unintended deletions - Verify results with sample files - Document complex commands for future reference ### When to Use Alternatives While sed works well for line deletion, consider these alternatives: - **grep -v**: Simple pattern exclusion - **awk**: Complex field-based processing - **text editors**: Interactive deletion with visual feedback ### Master sed's Complete Toolkit Expand your sed expertise with related techniques: - [Insert and append text](https://www.bitdoze.com/sed-insert-append-text/) - Add content precisely - [Transform text case](https://www.bitdoze.com/sed-change-case/) - Standardize capitalization - [Search and replace](https://www.bitdoze.com/sed-search-replace/) - Pattern-based substitution ### FAQ: Do hold space and multi-line pattern space matter for deletion? Yes. `sed` has two main buffers: - **pattern space**: holds the current line (or multiple lines if you use `N`) - **hold space**: a secondary buffer you can copy to/from (`h`, `H`, `g`, `G`, `x`) For deletion tasks, multi-line techniques help when you need decisions based on adjacent lines or blocks (for example, delete a line only if the next line matches a pattern). ### Why learn advanced techniques for text processing? They help you handle real-world inputs: logs, config files, semi-structured text, and messy data. Even if you don't use advanced `sed` daily, understanding ranges, negation, and multi-line tricks makes it easier to write safe automation.