Skip to content
Search

Regular expression

lastmod 2025-03-09created 2015-10-1638 edits

A regular expression (regexp or regex for short) is a formal language used to describe a set of strings that follow specific rules. Many text editors and programming languages support regular expressions for searching and replacing strings, and Perl and Tcl in particular implement powerful regular expressions in the language itself.

They originate from regular languages in computer science, but depending on the implementation they can sometimes describe a wider class of languages than regular languages, and there are even several different syntaxes for regular expressions themselves. Among them, POSIX extended regular expressions are standardized, and Perl regular expressions and their alternative implementation PCRE, though not standardized, are also widely used.

When converting strings into tokens, they are best suited to scanning text from left to right and trying varying numbers of characters to match against multiple rules; they are not suited to parsing.

An expression is a set of characters called metacharacters, which are interpreted beyond their literal meaning. For example, a quote symbol can indicate what someone said, but it can also give a meta meaning to the symbols that follow it. Regular expressions are a set of characters and metacharacters that give Unix its special characteristics. They are mainly used to describe a set of strings that follow specific rules.

Cheat Sheet

Regular_expressions_cheat_sheet_v2.png

Symbols for logical relationships

|
The alternation symbol selects one of several expressions.
abc|adc covers both the string abc and the string adc.
()
The grouping symbol can group several expressions into one.
abc|adc and a(b|d)c mean the same thing.
*
Zero or more.
a*b covers b, ab, aab, aaab, and so on.
+
One or more.
a+b covers ab, aab, aaab, but not b.
?
Zero or one.
a?b covers b and ab.
{m, n}
At least m and at most n.
a{1,3}b covers ab, aab, aaab, but not b or aaaab.
Accordingly, (fa|mo|b?o)ther denotes father, mother, bother, and other.
  • {M,} M or more characters
  • {,N} N or fewer characters
[]
Selects one of the characters between [ and ]. Same as using multiple |.
For example, [abc]d means ad, bd, cd.
Combined with the - symbol, it can specify a range of characters.
[a-z] means one of a through z, and [1-9] means one of 1 through 9.
[^]
Selects one character other than those between [^ and ].
[^abc]d does not cover ad, bd, cd, but covers ed, fd, and so on.
[^a-z] denotes every character that does not start with a lowercase letter.
^, $
Denote the start and end of the string, respectively.

Non-greedy matching

By default, * behaves greedily, matching as much as possible.

However, appending ? makes it try non-greedy matching, which matches as little as possible.

Example: in the string "abcabc", the regex a.*c matches the whole "abcabc", but a.*?c matches only "abc".

Character class

Instead of writing patterns as [a-z], [~A-Z], or [0-9], you can express them more simply with character classes as follows.

  • [[:alpha:]]: one alphabetic character; equivalent to [a-zA-Z] or [a-Z].
  • [[:digit:]]: one digit; equivalent to [0-9].
  • [[:alnum:]]: one alphabetic character or digit; equivalent to [a-zA-Z0-9].
  • [[:space:]]: a whitespace character ( ).

Escape

The following list of special sequences isn't complete. For a complete list of sequences and expanded class definitions for Unicode string patterns, see the last part of Regular Expression Syntax in the Standard Library reference. In general, the Unicode versions match any character that's in the appropriate category in the Unicode database.

  • \d - Matches any decimal digit; this is equivalent to the class [0-9].
  • \D - Matches any non-digit character; this is equivalent to the class [^0-9].
  • \s - Matches any whitespace character; this is equivalent to the class [ \t\n\r\f\v].
  • \S - Matches any non-whitespace character; this is equivalent to the class [^ \t\n\r\f\v].
  • \w - Matches any alphanumeric character; this is equivalent to the class [a-zA-Z0-9_].
  • \W - Matches any non-alphanumeric character; this is equivalent to the class [^a-zA-Z0-9_].

Representing special characters in regular expressions

To use any of the special characters covered so far for their literal meaning rather than as special characters, you must escape them by putting a backslash ('\') before them. The special characters that must be escaped in regular expressions are: 

^ . [ ] $ ( ) | * + ? { } \

Extended Regular Expressions

The extended regular expression (ERE) notation and construction rules shall apply to utilities defined as using extended regular expressions; any exceptions to the following rules are noted in the descriptions of the specific utilities using EREs.

Examples

^[0-9]+$
Unsigned integer
^[0-9]+([.][0-9]+)?$
Unsigned floating-point number
^[+-]?[0-9]+([.][0-9]+)?$
Signed floating-point number
find . -mindepth 1 -maxdepth 1 -name '*.m4a' | grep -E '.*_23[0-9]{4}_[0-9]{6}.m4a$' | xargs -I{} mv "{}" ./2023/
Moves files like 통화 녹음 15662566_230711_203603.m4a (a "call recording" file) into the 2023 directory. (Filters files with the date in the suffix)

Special Groups

Feature

Syntax

Description

Comment

(?#comment)

Everything between (?# and ) is ignored by the regex engine.

Branch reset group

(?|regex)

If the regex inside the branch reset group has multiple alternatives with capturing groups, then the capturing group numbers are the same in all the alternatives.

Atomic group

(?>regex)

Atomic groups prevent the regex engine from backtracking back into the group after a match has been found for the group. If the remainder of the regex fails, the engine may backtrack over the group if a quantifier or alternation makes it optional. But it will not backtrack into the group to try other permutations of the group.

Positive lookahead

(?=regex)

Matches at a position where the pattern inside the lookahead can be matched. Matches only the position. It does not consume any characters or expand the match. In a pattern like one(?=two)three, both two and three have to match at the position where the match of one ends.

Negative lookahead

(?!regex)

Similar to positive lookahead, except that negative lookahead only succeeds if the regex inside the lookahead fails to match.

Positive lookbehind

(?<=regex)

Matches at a position if the pattern inside the lookbehind can be matched ending at that position.

Negative lookbehind

(?<!regex)

Matches at a position if the pattern inside the lookbehind cannot be matched ending at that position.

Lookbehind

(?<=regex|longer regex)

Alternatives inside lookbehind can differ in length.

Lookbehind

(?<=x{n,m})

Quantifiers with a finite maximum number of repetitions can be used inside lookbehind.

Lookbehind

(?<=regex)

The full regular expression syntax can be used inside lookbehind.

Lookbehind

(group)(?<=\1)

Backreferences can be used inside lookbehind. Syntax prohibited in lookbehind is also prohibited in the referenced capturing group.

Keep text out of the regex match

\K

The text matched by the part of the regex to the left of the \K is omitted from the overall regex match. Other than that the regex is matched normally from left to right. Capturing groups to the left of the \K capture as usual.

Lookaround conditional

(?(?=regex)then|else) where (?=regex) is any valid lookaround and then and else are any valid regexes

If the lookaround succeeds, the “then” part must match for the overall regex to match. If the lookaround fails, the “else” part must match for the overall regex to match. The lookaround is zero-length. The “then” and “else” parts consume their matches like normal regexes.

Implicit lookahead conditional

(?(regex)then|else) where regex, then, and else are any valid regexes and regex is not the name of a capturing group

If “regex” is not the name of a capturing group, then it is interpreted as a lookahead as if you had written (?(?=regex)then|else). If the lookahead succeeds, the “then” part must match for the overall regex to match. If the lookahead fails, the “else” part must match for the overall regex to match. The lookaround is zero-length. The “then” and “else” parts consume their matches like normal regexes.

Named conditional

(?(name)then|else) where name is the name of a capturing group and then and else are any valid regexes

If the capturing group with the given name took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Named conditional

(?(<name>)then|else) where name is the name of a capturing group and then and else are any valid regexes

If the capturing group with the given name took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Named conditional

(?('name')then|else) where name is the name of a capturing group and then and else are any valid regexes

If the capturing group with the given name took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Conditional

(?(1)then|else) where 1 is the number of a capturing group and then and else are any valid regexes

If the referenced capturing group took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Relative conditional

(?(-1)then|else) where -1 is a negative integer and then and else are any valid regexes

Conditional that tests the capturing group that can be found by counting as many opening parentheses of named or numbered capturing groups as specified by the number from right to left starting immediately before the conditional. If the referenced capturing group took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Forward conditional

(?(+1)then|else) where +1 is a positive integer and then and else are any valid regexes

Conditional that tests the capturing group that can be found by counting as many opening parentheses of named or numbered capturing groups as specified by the number from left to right starting at the “then” part of conditional. If the referenced capturing group took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Conditional

(?(+1)then|else) where 1 is the number of a capturing group and then and else are any valid regexes

The + is ignored and the number is taken as an absolute reference to a capturing group. If the referenced capturing group took part in the match attempt thus far, the “then” part must match for the overall regex to match. If the capturing group did not take part in the match thus far, the “else” part must match for the overall regex to match.

Lookaround

Pattern

Type

matches

X(?=Y)

Positive Lookahead

When X is followed by Y

X(?!Y)

Negative Lookahead

When X is not followed by Y

(?<=Y)X

Positive Lookbehind

When X is preceded by Y

(?<!Y)X

Negative Lookbehind

When X is not preceded by Y

(Note: the lookahead and lookbehind syntaxes differ because with something like X{Pattern}X, there would be no way to tell whether to search ahead or behind.)

Lookahead and Lookbehind together are called Lookaround.

To explain how it works in an easy-to-understand way:

  1. First find the places that match the main pattern,
  2. then additionally check whether the Lookaround condition holds.
  3. Then include or exclude the result depending on whether it is Positive or Negative.

The difference between Positive and Negative:

  • Positive means that if condition Y is satisfied, the match is kept in the results. In other words, if condition Y is not met, it won't be included in the results.
  • Negative means that if condition Y is satisfied, the match is excluded from the results. In other words, if condition Y is not met, it will be included in the results.

The difference between Lookahead and Lookbehind is simple.

  • Lookahead is for checking the suffix,
  • Lookbehind is for checking the prefix.

Sounds a bit backwards, doesn't it? In the actual syntax, you can tell them apart by whether < is included. Since < points left, just remember it as the front side.

In general, it is mostly used as a Negative Lookaround when you simply want to exclude results. (reference, Stack Overflow)

One notable characteristic is that with Lookaround, the result is not consumed. This means it is useful when conditions need to overlap. See the example on the original site.

Non-matching text

A simple example:

  • install(?!ed) finds the text install not followed by the suffix ed.
  • (?<!de)install finds the text install not preceded by the prefix de.

How to use it in grep

You must use Perl-compatible expressions with the -P flag.

A simple example: 

dpkg --get-selections | grep -P '(?<!de)install$'

Examples

AWS Amplify SPA redirect pattern
</^[^.]+$|\.(?!(css|gif|ico|jpg|js|png|txt|svg|woff|woff2|ttf|map|json|webp)$)([^.]+$)/>

Globstar

If set, the pattern ** used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a /, only directories and subdirectories match.

See also

Favorite site

Online Tools

References

  1. MSDN_-Regular_Expressions-_C++.pdf ↩

  2. Regular_expression_theorem.pdf ↩

author Zer0 sitemap