Grep is one among the system administrator’s “Swiss Army knife” set of tools, and is extremely useful to search for strings and patterns in a group of files, or even sub-folders. This article introduces the basics of Grep, provides examples of advanced use and links you to further reading.
Grep (an acronym for “Global Regular Expression Print”) is installed by default on almost every distribution of Linux, BSD and UNIX, and is even available for Windows. GNU and the Free Software Foundation distribute Grep as part of their suite of open source tools. This tutorial focuses primarily on this GNU version, as it is currently the most widely used.
Grep finds a string in a given file or input, quickly and efficiently. While most everyday uses of the command are simple, there are a variety of more advanced uses that most people don’t know about — including regular expressions and more, which can become quite complicated.
The tool has its roots in an extended regular expression syntax that was added to UNIX after Ken Thompson’s original regular expression implementation. The latter searches for any of a list of fixed strings, using the Aho-Corasick algorithm. These variants are embodied in most modern Grep implementations as command-line switches (and standardised as -E and -F in POSIX.2). In such combined implementations, Grep may also behave differently depending on the name by which it is invoked, allowing fGrep, eGrep, and Grep to be links to the same program.
There are two ways to provide input to Grep, each with its own particular uses. First, Grep can be used to search a given file or files on a system (including a recursive search through sub-folders). Grep also accepts inputs (usually via a pipe) from another command or series of commands.
Regular expressions
A regular expression, often shortened to “regex” or “regexp”, is a way of specifying a pattern (a particular set of characters or words) in text that can be applied to variable inputs to find all occurrences that match the pattern. Regexes enhance the ability to meaningfully process text content, especially when combined with other commands.
Usually, regular expressions are included in the Grep command in the following format:
grep [options] [regexp] [filename]
GNU Grep uses the GNU version of regular expressions, which is very similar (but not identical) to POSIX regular expressions. In fact, most varieties of regular expressions are quite similar, but have differences in escapes, meta-characters, or special operators.
GNU Grep has two regular expression feature sets: Basic and Extended. In basic regular expressions, the meta-characters ?, +, {, |, (, and ) lose their special meaning (whose uses are described later in this article). As mentioned below, to switch to using extended regular expressions, you need to add the option -E to the grep command.
It is customary to enclose the regular expression in single quotation marks, to prevent the shell (Bash or others) from trying to interpret and expand the expression before launching the grep process. For example, if a pair of back-ticks in the regexp is not quoted, it would result in the text between the back-ticks being executed as a Bash sub-process — and if this happens to be a valid command, the text returned by it takes the regular expression’s place in the command-line parameters given to Grep! Not at all what we want.
Again, due to shell behaviour, you can also enclose the regex in double quotes — in this case, you can use environment variables in the regex, and the shell will substitute them before calling Grep. This can be very useful, depending on what you’re trying to do — or it could turn out to be a nuisance. Remember the difference in behaviour.
Basic usage
Now let’s go on to some practical examples of using Grep. To better understand the results, I’ve created a simple text file on which we will run our Grep searches; the file contains the following lines:
Hi this is test file to carry out few regular expressions practical with grep 123 456 Abcd ABCD
Case-insensitive search (grep -i):
[manish@clone ~]$ grep -i 'abcd' testfile Abcd ABCD
As you can see, the -i flag causes a search for “abcd” to return matches that have different cases for the characters from what the search string does.
Whole-word search (grep -w):
[manish@clone ~]$ grep -w 'test' testfile is test file
This type of search only returns lines where the sought-for string is a whole word and not part of a larger word.
Recursively search through sub-folders (grep -r <pattern> <path>):
[manish@clone ~]$ grep -r '456' /root/ /root/testfile:Year is 2010
Inverted search (grep -v):
[manish@clone ~]$ grep -v 'practical' testfile Hi this is test file to carry out few regular expressions 123 456 Abcd ABCD
This prints all the lines in the file, except the line that contains the word “practical”.
An interesting relative is the -L flag (you can also use --files-without-match), which outputs the names of files that do NOT contain matches for your search pattern. The matches for your search pattern are not themselves printed, only the names are.
[manish@clone ~]$ grep -r -L "Network" /var/log/* /var/log/anaconda.log /var/log/anaconda.syslog /var/log/audit/audit.log /var/log/boot.log /var/log/boot.log.1 ...
The “opposite” flag to -L is -l or --files-with-matches, which prints out (only) the names of files that do contain matches for your search pattern.
Print additional (trailing) context lines after match (grep -A <NUM>):
[manish@clone ~]$ grep -A1 '123' testfile 123 456 Abcd
For each line that matches the search, Grep prints the matching line, as well as the next one line after the match. Varying the number provided to -A changes the number of additional lines that are in the output.
Print additional (leading) context lines before match (grep -B <NUM>):
[manish@clone ~]$ grep -B2 'Abcd' testfile practical with grep 123 456 Abcd
Print additional (leading and trailing) context lines before and after the match (grep -C <NUM>):
[manish@clone ~]$ grep -C2 'carry' testfile this is test file to carry out few regular expressions practical with grep 123 456
As you can see, this has printed out two lines before and after the single match found in the file; if there are multiple matches, Grep inserts a line containing -- between each group of lines (each match and its context lines).
Print the filename for each match (grep -H <pattern> filename):
[manish@clone ~]$ grep -H 'a' testfile testfile:to carry out few regular expressions testfile:practical with grep
Now, let’s run the search a bit differently:
[manish@clone ~]$ cat testfile | grep -H 'a' (standard input):to carry out few regular expressions (standard input):practical with grep
When the stream that Grep is asked to search is passed to its standard input via a pipe from a previous command in the chain, grep -H displays (standard input) as the filename.
Run in “quiet” mode (grep -q): When run with this flag, Grep does not write anything to standard output, but sets its return value (also known as exit status) to reflect whether a match was found or not. This option is mainly used in scripts that need to check if a given file contains a particular match. A return status of 0 (zero) indicates that a match was found; 1 indicates that no match was found.
[manish@clone ~]$ grep -q '2010' testfile [manish@clone ~]$ echo $? 1 [manish@clone ~]$ grep -q '456' testfile [manish@clone ~]$ echo $? 0
Using regular expressions
[manish@clone ~]$ grep 'c.r' testfile to carry out few regular expressions
In the search above, . is used to match any single character — which is why it matches “car” in “carry”. Grep has a powerful regular expression matching engine, which we can’t hope to cover in depth here, but we will include a few important points:
- Most characters, including all letters and digits, are actually regular expressions that match themselves.
- Any meta-character (with special meaning to Grep, like the
.in the example above) may be quoted by preceding it with a backslash. This makes Grep treat it as an ordinary character.
[manish@clone ~]$ grep 'c\.r' testfile [manish@clone ~]$
As you can see, preceding . with a backslash has removed its significance as a meta-character.
A regular expression may be followed by one of several repetition operators:
- The period (
.) matches any single character. ?means that the preceding item is optional, and if found, will be matched at the most, once.*means that the preceding item will be matched zero or more times.+means the preceding item will be matched one or more times.{n}means the preceding item is matched exactly n times, while{n,}means the item is matched n or more times.{n,m}means that the preceding item is matched at least n times, but not more than m times.{,m}means that the preceding item is matched, at the most, m times.
However, the repetition operators are part of GNU Grep’s extended regular expression syntax, so to use these effectively, remember to add the -E option to your command.
Read this tutorial for an introduction to more of Grep regular expression features. For more information on regular expression syntax, refer to the Regular Expressions chapter in the Grep manual. Meanwhile, we will present some examples of regular expressions and try to show how they work.
Character classes in regular expressions
The “character class” tool is one of the more flexible and often-used features of regular expressions. There are two basic ways to use character classes: to specify a list of characters (for example, [aeiou] is a list of vowel characters), or a range (like [m-t], which expands to [mnopqrst]). Ranges are a convenience that saves having to type an entire sequence of characters. A character class can also include a list of special characters, but they can’t be used as a range.
A single character class instance will match only one character; to match multiple occurrences of the class, you would need to add a repetition operator, like those mentioned above. For example, to find an eleven-letter string comprising only lower-case alphabets, the regex would be: [a-z]{11}. As mentioned earlier, to use the repetition operators, we need to add the option -E. Let’s run this on our test file:
[manish@clone ~]$ Grep -E '[a-z]{11}' testfile
to carry out few regular expressions
Here, “expressions” is the only all-lowercase 11-character string in the file; so this is the only line printed as the output.
There are quite a few character classes that are very commonly used in regular expressions, and these are provided as named classes. For example, the [a-z] class of lower-case alphabets that we used above, has the named class [:lower:]. Naturally, [:upper:] is upper-case letters A to Z, and [:alpha:] is all alphabetic characters, equivalent to [:lower:] plus [:upper:]. [:digit:] is the digits 0 to 9, and [:alnum:] is alphanumeric characters — a combination of [:alpha:] and [:digit:]. The Grep manual lists out more of these named classes.
When a carat (^) is used as the first character in a character class, it is a negation of the class, effectively meaning, “none of these characters”.
Line and word anchors
The ^ anchor specifies that the pattern following it should be at the start of the line:
[manish@clone ~]$ grep '^th' testfile this
The $ anchor specifies that the pattern before it should be at the end of the line.
[manish@clone ~]$ grep 'i$' testfile Hi
The operator \< anchors the pattern to the start of a word.
[manish@clone ~]$ grep '\<fe' testfile to carry out few regular expressions
Similarly, \> anchors the pattern to the end of a word.
[manish@clone ~]$ grep 'le\>' testfile is test file
The \b (word boundary) anchor can be used in place of \< and \> to signify the beginning or end of a word:
[manish@clone ~]$ grep -e '\breg' testfile to carry out few regular expressions
Finally, we look at the | (alternation) operator, which is part of the extended regex features. A pattern containing this operator separately matches the parts on either side of it; if either one is found, the line containing it is a match. The parts can themselves be complex regular expressions, so this means you can check each line in a file for multiple search patterns in one pass.
[manish@clone ~]$ grep -E 'hi|bc' testfile this Abcd
That was pretty simple; so let’s try a more complicated one. Can you reason out why the output lines for this regex are as shown below?
[manish@clone ~]$ grep -E '^[t-z]+|[^a-z]+$' testfile this to carry out few regular expressions 123 456 ABCD
Using shell expansions in the pattern input to Grep
As mentioned earlier, if you don’t single-quote the pattern passed to Grep, the shell could perform shell expansion on the pattern and actually feed a changed pattern to Grep. This can also be done intentionally, when you need it — let’s look at a few examples.
[root@clone ~]# grep "$HOME" /etc/passwd root:x:0:0:root:/root:/bin/bash operator:x:11:0:operator:/root:/sbin/nologin
Here, we intentionally use double quotes to make the Bash shell replace the environment variable $HOME with the actual value of the variable (in this case, /root). Thus, Grep searches the /etc/passwd file for the text /root, yielding the two lines that match.
[root@clone ~]# grep `whoami` /etc/passwd root:x:0:0:root:/root:/bin/bash operator:x:11:0:operator:/root:/sbin/nologin
Here, back-tick expansion is done by the shell, replacing `whoami` with the user name (root) that is returned by the whoami command.
Well, we hope this has set you on your way to using this very efficient tool.
Thank you for this tutorial. It cleared lot of basic doubts about grep from my brain.
Thanks. There’s a lot I could learn from this. ;)
Thanks. This is very helpful information.
Mast likhaa hai yaar!
Answer to the “Can you reason” script:
Expression “^[t-z]+” prints lines having one or more occurrences of any character from t to z at the beginning.
Expression “[^a-z]+$” prints lines NOT having one or more occurrences of any character from a to z at the end.
The overall script prints the lines which satisfy either or both of the above command expressions.
Therefore, lines
“this” and
“to carry out few regular expressions”
satisfy the first command expression, while lines
“123 456”
“ABCD”
satisfy the second command expression.
[…] 1. Grep in Depth 2. Open Source For U 3. GNU Grep […]
Please explain this command “grep -a –null-data U-Boot u-boot.img ” What does U-Boot stand for here, is it a file name, an alias ? u-boot.img is a file I can see it .
Thank you so much for this. Learned a lot.
Great tutorial on Grep, thanks.