This chapter is from the book
This chapter is from the book
This chapter is from the book
9.4 Regular Expressions
Regular expressions specify string patterns. Use them whenever you need to locate strings that match a particular pattern. For example, suppose you want to find hyperlinks in an HTML file. You need to look for strings of the pattern <a href="...">. But wait—there may be extra spaces, or the URL may be enclosed in single quotes. Regular expressions give you a precise syntax for specifying what sequences of characters are legal matches.
In the following sections, you will see the regular expression syntax used by the Java API, and how to put regular expressions to work.
9.4.1 The Regular Expression Syntax
In a regular expression, a character denotes itself unless it is one of the reserved characters
. * + ? { | ( ) [ \ ^ $
For example, the regular expression Java only matches the string Java.
The symbol . matches any single character. For example, .a.a matches Java and data.
The * symbol indicates that the preceding constructs may be repeated 0 or more times; for a +, it is 1 or more times. A suffix of ? indicates that a construct is optional (0 or 1 times). For example, be+s? matches be, bee, and bees. You can specify other multiplicities with { } (see Table 9-4).
A | denotes an alternative: .(oo|ee)f matches beef or woof. Note the parentheses—without them, .oo|eef would be the alternative between .oo and eef. Parentheses are also used for grouping—see Section 9.4.4, “Groups” (page 330).
A character class is a set of character alternatives enclosed in brackets, such as [Jj], [0-9], [A-Za-z], or [^0-9]. Inside a character class, the - denotes a range (all characters whose Unicode values fall between the two bounds). However, a - that is the first or last character in a character class denotes itself. A ^ as the first character in a character class denotes the complement (all characters except those specified).
There are many predefined character classes such as \d (digits) or \p{Sc} (Unicode currency symbols). See Tables 9-4 and 9-5.
The characters ^ and $ match the beginning and end of input.
If you need to have a literal . * + ? { | ( ) [ \ ^ $, precede it by a backslash. Inside a character class, you only need to escape [ and \, provided you are careful about the positions of ] - ^. For example, []^-] is a class containing all three of them.
Alternatively, surround a string with \Q and \E. For example, \(\$0\.99\) and \Q($0.99)\E both match the string ($0.99).
TIP
If you have a string that may contain some of the many special characters in the regular expression syntax, you can escape them all by calling Parse.quote(str). This simply surrounds the string with \Q and \E, but it takes care of the special case where str may contain \E.
Table 9-4 Regular Expression Syntax
Expression |
Description |
Example |
|---|---|---|
Characters |
||
c, not one of . * + ? { | ( ) [ \ ^ $ |
The character c. |
J |
. |
Any character except line terminators, or any character if the DOTALL flag is set. |
|
\x{p} |
The Unicode code point with hex code p. |
\x{1D546} |
\uhhhh, \xhh, \0o, \0oo, \0ooo |
The UTF-16 code unit with the given hex or octal value. |
\uFEFF |
\a, \e, \f, \n, \r, \t |
Alert (\x{7}), escape (\x{1B}), form feed (\x{B}), newline (\x{A}), carriage return (\x{D}), tab (\x{9}). |
\n |
\cc, where c is in [A-Z] or one of @ [ \ ] ^ _ ? |
The control character corresponding to the character c. |
\cH is a backspace (\x{8}). |
\c, where c is not in [A-Za-z0-9] |
The character c. |
\\ |
\Q ... \E |
Everything between the start and the end of the quotation. |
\Q(...)\E matches the string (...). |
Character Classes |
||
[C1C2...], where Ci are characters, ranges c-d, or character classes |
Any of the characters represented by C1, C2,... |
[0-9+-] |
[^...] |
Complement of a character class. |
[^\d\s] |
[...&&...] |
Intersection of character classes. |
[\p{L}&&[^A-Za-z]] |
\p{...}, \P{...} |
A predefined character class (see Table 9-5); its complement. |
\p{L} matches a Unicode letter, and so does \pL—you can omit braces around a single letter. |
\d, \D |
Digits ([0-9], or \p{Digit} when the UNICODE_CHARACTER_CLASS flag is set); the complement. |
\d+ is a sequence of digits. |
\w, \W |
Word characters ([a-zA-Z0-9_], or Unicode word characters when the UNICODE_CHARACTER_CLASS flag is set); the complement. |
|
\s, \S |
Spaces ([\n\r\t\f\x{B}], or \p{IsWhite_Space} when the UNICODE_CHARACTER_CLASS flag is set); the complement. |
\s*,\s* is a comma surrounded by optional white space. |
\h, \v, \H, \V |
Horizontal whitespace, vertical whitespace, their complements. |
|
Sequences and Alternatives |
||
XY |
Any string from X, followed by any string from Y. |
[1-9][0-9]* is a positive number without leading zero. |
X|Y |
Any string from X or Y. |
http|ftp |
Grouping |
||
(X) |
Captures the match of X. |
'([^']*)' captures the quoted text. |
\n |
The nth group. |
(['"]).*\1 matches 'Fred' or "Fred" but not "Fred'. |
(?<name>X) |
Captures the match of X with the given name. |
'(?<id>[A-Za-z0-9]+)' captures the match with name id. |
\k<name> |
The group with the given name. |
\k<id> matches the group with name id. |
(?:X) |
Use parentheses without capturing X. |
In (?:http|ftp)://(.*), the match after :// is \1. |
(?f1f2...:X), (?f1...-fk...:X), with fi in [dimsuUx] |
Matches, but does not capture, X with the given flags on or off (after -). |
(?i:jpe?g) is a case-insensitive match. |
Other (?...) |
See the Pattern API documentation. |
|
Quantifiers |
||
X? |
Optional X. |
\+? is an optional + sign. |
X*, X+ |
0 or more X, 1 or more X. |
[1-9][0-9]+ is an integer ≥ 10. |
X{n}, X{n,}, X{m,n} |
n times X, at least n times X, between m and n times X. |
[0-7]{1,3} are one to three octal digits. |
Q?, where Q is a quantified expression |
Reluctant quantifier, attempting the shortest match before trying longer matches. |
.*(<.+?>).* captures the shortest sequence enclosed in angle brackets. |
Q+, where Q is a quantified expression |
Possessive quantifier, taking the longest match without backtracking. |
'[^']*+' matches strings enclosed in single quotes and fails quickly on strings without a closing quote. |
Boundary Matches |
||
^ $ |
Beginning, end of input (or beginning, end of line in multiline mode). |
^Java$ matches the input or line Java. |
\A \Z \z |
Beginning of input, end of input, absolute end of input (unchanged in multiline mode). |
|
\b \B |
Word boundary, nonword boundary. |
\bJava\b matches the word Java. |
\R |
A Unicode line break. |
|
\G |
The end of the previous match. |
|
Table 9-5 Predefined Character Classes \p{...}
Name |
Description |
|---|---|
posixClass |
posixClass is one of Lower, Upper, Alpha, Digit, Alnum, Punct, Graph, Print, Cntrl, XDigit, Space, Blank, ASCII, interpreted as POSIX or Unicode class, depending on the UNICODE_CHARACTER_CLASS flag. |
IsScript, sc=Script, script=Script |
A script accepted by Character.UnicodeScript.forName. |
InBlock, blk=Block, block=Block |
A block accepted by Character.UnicodeBlock.forName. |
Category, InCategory, gc=Category, general_category=Category |
A one- or two-letter name for a Unicode general category. |
IsProperty |
Property is one of Alphabetic, Ideographic, Letter, Lowercase, Uppercase, Titlecase, Punctuation, Control, White_Space, Digit, Hex_Digit, Join_Control, Noncharacter_Code_Point, Assigned. |
javaMethod |
Invokes the method Character.isMethod (must not be deprecated). |
9.4.2 Testing a Match
Generally, there are two ways to use a regular expression: Either you want to test whether a string conforms to the expression, or you want to find all matches of the expressions in a string.
In the first case, simply use the static matches method:
String regex = "[+-]?\\d+";
CharSequence input = ...;
if (Pattern.matches(regex, input)) {
...
}
If you need to use the same regular expression many times, it is more efficient to compile it. Then, create a Matcher for each input:
Pattern pattern = Pattern.compile(regex); Matcher matcher = pattern.matcher(input); if (matcher.matches()) ...
If the match succeeds, you can retrieve the location of matched groups—see Section 9.4.4, “Groups” (page 330).
If you want to test whether the input contains a match, use the find method instead:
if (matcher.find()) ...
You can turn the pattern into a predicate:
Pattern digits = Pattern.compile("[0-9]+");
List<String> strings = List.of("December", "31st", "1999");
List<String> matchingStrings = strings.stream()
.filter(digits.asMatchPredicate())
.toList(); // ["1999"]
The result contains all strings that match the regular expression.
Use the asPredicate method to test whether a string contains a match:
List<String> sringsContainingMatch = strings.stream()
.filter(digits.asPredicate())
.toList(); // ["31st", "1999"]
9.4.3 Finding All Matches
In this section, we consider the other common use case for regular expressions—finding all matches in an input. Use this loop:
String input = ...;
Matcher matcher = pattern.matcher(input);
while (matcher.find()) {
String match = matcher.group();
int matchStart = matcher.start();
int matchEnd = matcher.end();
...
}
In this way, you can process each match in turn. As shown in the code fragment, you can get the matched string as well as its position in the input string.
More elegantly, you can call the results method to get a Stream<MatchResult>. The MatchResult interface has methods group, start, and end, just like Matcher. (In fact, the Matcher class implements this interface.) Here is how you get a list of all matches:
List<String> matches = pattern.matcher(input)
.results()
.map(Matcher::group)
.toList();
If you have the data in a file, then you can use the Scanner.findAll method to get a Stream<MatchResult>, without first having to read the contents into a string. You can pass a Pattern or a pattern string:
var in = new Scanner(path, StandardCharsets.UTF_8);
Stream<String> words = in.findAll("\\pL+")
.map(MatchResult::group);
9.4.4 Groups
It is common to use groups for extracting components of a match. For example, suppose you have a line item in the invoice with item name, quantity, and unit price such as
Blackwell Toaster USD29.95
Here is a regular expression with groups for each component:
(\p{Alnum}+(\s+\p{Alnum}+)*)\s+([A-Z]{3})([0-9.]*)
After matching, you can extract the nth group from the matcher as
String contents = matcher.group(n);
Groups are ordered by their opening parenthesis, starting at 1. (Group 0 is the entire input.) In this example, here is how to take the input apart:
Matcher matcher = pattern.matcher(input);
if (matcher.matches()) {
item = matcher.group(1);
currency = matcher.group(3);
price = matcher.group(4);
}
We aren’t interested in group 2; it only arose from the parentheses that were required for the repetition. For greater clarity, you can use a noncapturing group:
(\p{Alnum}+(?:\s+\p{Alnum}+)*)\s+([A-Z]{3})([0-9.]*)
Or, even better, capture by name:
(?<item>\p{Alnum}+(\s+\p{Alnum}+)*)\s+(?<currency>[A-Z]{3})(?<price>[0-9.]*)
Then, you can retrieve the items by name:
item = matcher.group("item");
With the start and end methods, you can get the group positions in the input:
int itemStart = matcher.start("item");
int itemEnd = matcher.end("item");
NOTE
Retrieving groups by name only works with a Matcher, not with a MatchResult.
NOTE
When you have a group inside a repetition, such as (\s+\p{Alnum}+)* in the example above, it is not possible to get all of its matches. The group method only yields the last match, which is rarely useful. You need to capture the entire expression with another group.
9.4.5 Splitting along Delimiters
Sometimes, you want to break an input along matched delimiters and keep everything else. The Pattern.split method automates this task. You obtain an array of strings, with the delimiters removed:
String input = ...;
Pattern commas = Pattern.compile("\\s*,\\s*");
String[] tokens = commas.split(input);
// "1, 2, 3" turns into ["1", "2", "3"]
If there are many tokens, you can fetch them lazily:
Stream<String> tokens = commas.splitAsStream(input);
If you don’t care about precompiling the pattern or lazy fetching, you can just use the String.split method:
String[] tokens = input.split("\\s*,\\s*");
If the input is in a file, use a scanner:
var in = new Scanner(path, StandardCharsets.UTF_8);
in.useDelimiter("\\s*,\\s*");
Stream<String> tokens = in.tokens();
9.4.6 Replacing Matches
If you want to replace all matches of a regular expression with a string, call replaceAll on the matcher:
Matcher matcher = commas.matcher(input);
String result = matcher.replaceAll(",");
// Normalizes the commas
Or, if you don’t care about precompiling, use the replaceAll method of the String class.
String result = input.replaceAll("\\s*,\\s*", ",");
The replacement string can contain group numbers $n or names ${name}. They are replaced with the contents of the corresponding captured group.
String result = "3:45".replaceAll(
"(\\d{1,2}):(?<minutes>\\d{2})",
"$1 hours and ${minutes} minutes");
// Sets result to "3 hours and 45 minutes"
You can use \ to escape $ and \ in the replacement string, or you can call the Matcher.quoteReplacement convenience method:
matcher.replaceAll(Matcher.quoteReplacement(str))
If you want to carry out a more complex operation than splicing in group matches, then you can provide a replacement function instead of a replacement string. The function accepts a MatchResult and yields a string. For example, here we replace all words with at least four letters with their uppercase version:
String result = Pattern.compile("\\pL{4,}")
.matcher("Mary had a little lamb")
.replaceAll(m -> m.group().toUpperCase());
// Yields "MARY had a LITTLE LAMB"
The replaceFirst method replaces only the first occurrence of the pattern.
9.4.7 Flags
Several flags change the behavior of regular expressions. You can specify them when you compile the pattern:
Pattern pattern = Pattern.compile(regex,
Pattern.CASE_INSENSITIVE | Pattern.UNICODE_CHARACTER_CLASS);
Or you can specify them inside the pattern:
String regex = "(?iU:expression)";
Here are the flags:
Pattern.CASE_INSENSITIVE or i: Match characters independently of the letter case. By default, this flag takes only US ASCII characters into account.
Pattern.UNICODE_CASE or u: When used in combination with CASE_INSENSITIVE, use Unicode letter case for matching.
Pattern.UNICODE_CHARACTER_CLASS or U: Select Unicode character classes instead of POSIX. Implies UNICODE_CASE.
Pattern.MULTILINE or m: Make ^ and $ match the beginning and end of a line, not the entire input.
Pattern.UNIX_LINES or d: Only '\n' is a line terminator when matching ^ and $ in multiline mode.
Pattern.DOTALL or s: Make the . symbol match all characters, including line terminators.
Pattern.COMMENTS or x: Whitespace and comments (from # to the end of a line) are ignored.
Pattern.LITERAL: The pattern is taking literally and must be matched exactly, except possibly for letter case.
Pattern.CANON_EQ: Take canonical equivalence of Unicode characters into account. For example, u followed by ¨ (diaeresis) matches ü.
The last two flags cannot be specified inside a regular expression.