How to Process Text with Bash Using Grep, Sed, and Awk Commands
Introduction
Text processing in Bash using the grep, sed, and awk commands lets you filter and manipulate input to perform specific tasks, such as file content processing. You can combine the text processing commands with loops, conditional statements, and expressions for advanced processing.
This article explains how to process text in Bash using the grep, sed, and awk commands.
Create a New Bash Script
Follow the steps below to create a new Bash script that loads an input file with multiple lines of content to a variable for further text processing.
Create a new
log.txtfile to store multiple lines of text.CONSOLEAdd the following contents to the
log.txtfile.TEXTSave and close the file.
Create a new
processing.shBash script that inputs thelog.txtfile as a variable.CONSOLEAdd the following contents to the
processing.shfile.BASHSave and close the file.
In the above script, the
LOGvariable takes thelog.txtas an input. You'll process the variable value using thegrep,sed, andawkcommands in the next sections.
Use the grep Command to Process Text File
The grep command searches specific text patterns in files, such as the log.txt file you created, and outputs the results for additional processing in a Bash script. Follow the steps below to use the grep command and display the results.
Open the
processing.shscript you created earlier.CONSOLEAdd the following command at the end of the file.
BASHSave and close the file.
The above
grepcommand searches for the termERRORin thelog.txtfile and displays all matching results.Run the script using Bash.
CONSOLEOutput:
Use the awk Command to Process Text File
The awk command analyzes structured data and searches for specific patterns in a file. You can use the awk command to customize file contents, split lines into fields or columns for processing, and match specific patterns. Follow the steps below to process the log.txt file columns using the awk command.
Open the
processing.shBash script.CONSOLEAdd the following contents to the file.
BASHSave and close the file.
The above
awkcommand prints the first and third columns in thelog.txtfile to display the date and error type.Run the script using Bash.
CONSOLEOutput:
Use sed Command to Process Text File
Stream Editor (sed) is a text processing command that processes and transforms text into a file. The sed command searches, modifies, and replaces contents in standard input or a specific file using Bash. Follow the steps below to use the sed command to search and replace contents in the log.txt file.
Open the
processing.shBash script.CONSOLEAdd the following command at the end of the file.
BASHSave and close the file.
The above
sedcommand replacesERRORwithNew ERRORandWARNINGwithNew WARNINGvalues in thelog.txtfile.Run the script using Bash.
CONSOLEOutput:
Combine grep, sed, and awk Commands
You can combine grep, sed, and awk commands to analyze and manipulate large inputs and perform advanced text processing tasks using Bash. Follow the steps below to combine the grep, sed, and awk commands and process the log.txt file.
Create a new
combine.shBash script.CONSOLEAdd the following contents to the file.
BASHSave and close the file.
In the above Bash script, the combined
grep,sed, andawkcommands use pipes|to redirect the output of each command and display the modified contents of thelog.txtfile in a well-structured format that includes all log entries.Run the script using Bash.
CONSOLEOutput:
Use grep, sed, and awk with Conditional Bash Statements
You can use grep, sed, and awk commands to process text further with if-else, case, and if-then conditional statements to enhance output readability and handle errors. Follow the steps below to use the conditional statement to process the log.txt file using the search commands in a Bash script.
Create a new
conditional-processing.shBash script.CONSOLEAdd the following contents to the
conditional-processing.shfile.BASHSave and close the file.
In the above file:
- The outer
if-elseconditional statement validates the input variable. - The inner
if-elseconditional statement runs agrepcondition to find theERRORorWARNINGfields in the file. The combinedgrep,sed, andawkcommands run when the condition evaluates totrueand perform additional processing to display the modified output.
- The outer
Run the script using Bash.
CONSOLEOutput:
Use grep, sed, and awk with Bash Loops
Loops such as while, for, and until let you run advanced text processing tasks when using the grep,sed, and awk commands. Loops run continuous tasks when a condition returns true or false and are useful when reading multiple lines in files. Follow the steps below to use a while loop to display the contents of the log.txt file using the output from the grep,sed, and awk commands as the input.
Create a new
loop-combined.shscript.CONSOLEAdd the following contents to the
loop-combined.shfile.BASHSave and close the file.
The above Bash script prompts a user to enter a command option to run the
whileloop that displays the contents of thelog.txtfile using the piped output from thegrep,sed, andawkcommands on each new line.Run the script using Bash and set
ERRORas a command option to test thewhileloop.CONSOLEOutput:
Run the script again and set
WARNINGas the command option.CONSOLEOutput:
Conclusion
You have used the grep, sed, and awk commands in Bash to process text. Combining the commands enables advanced text processing and filtering when handling large data from inputs such as log files. Combine the search commands with loops, functions, and advanced conditional expressions in Bash to create efficient scripts that perform specific tasks such as log monitoring.
0 Comments
Be the first to comment and share your perspective with the community.