- Rules
- Risk Assessment Summary
- IDS00-J. Sanitize untrusted data passed across a trust boundary
- IDS01-J. Normalize strings before validating them
- IDS02-J. Canonicalize path names before validating them
- IDS03-J. Do not log unsanitized user input
- IDS04-J. Limit the size of files passed to ZipInputStream
- IDS05-J. Use a subset of ASCII for file and path names
- IDS06-J. Exclude user input from format strings
- IDS07-J. Do not pass untrusted, unsanitized data to the Runtime.exec() method
- IDS08-J. Sanitize untrusted data passed to a regex
- IDS09-J. Do not use locale-dependent methods on locale-dependent data without specifying the appropriate locale
- IDS10-J. Do not split characters between two data structures
- IDS11-J. Eliminate noncharacter code points before validation
- IDS12-J. Perform lossless conversion of String data between differing character encodings
- IDS13-J. Use compatible encodings on both sides of file or network I/O
This chapter is from the book
This chapter is from the book
This chapter is from the book
IDS01-J. Normalize strings before validating them
Many applications that accept untrusted input strings employ input filtering and validation mechanisms based on the strings’ character data.
For example, an application’s strategy for avoiding cross-site scripting (XSS) vulnerabilities may include forbidding <script> tags in inputs. Such blacklisting mechanisms are a useful part of a security strategy, even though they are insufficient for complete input validation and sanitization. When implemented, this form of validation must be performed only after normalizing the input.
Character information in Java SE 6 is based on the Unicode Standard, version 4.0 [Unicode 2003]. Character information in Java SE 7 is based on the Unicode Standard, version 6.0.0 [Unicode 2011].
According to the Unicode Standard [Davis 2008a], annex #15, Unicode Normalization Forms:
When implementations keep strings in a normalized form, they can be assured that equivalent strings have a unique binary representation.
Normalization Forms KC and KD must not be blindly applied to arbitrary text. Because they erase many formatting distinctions, they will prevent round-trip conversion to and from many legacy character sets, and unless supplanted by formatting markup, they may remove distinctions that are important to the semantics of the text. It is best to think of these Normalization Forms as being like uppercase or lowercase mappings: useful in certain contexts for identifying core meanings, but also performing modifications to the text that may not always be appropriate. They can be applied more freely to domains with restricted character sets.
Frequently, the most suitable normalization form for performing input validation on arbitrarily encoded strings is KC (NFKC) because normalizing to KC transforms the input into an equivalent canonical form that can be safely compared with the required input form.
Noncompliant Code Example
This noncompliant code example attempts to validate the String before performing normalization. Consequently, the validation logic fails to detect inputs that should be rejected because the check for angle brackets fails to detect alternative Unicode representations.
// String s may be user controllable
// \uFE64 is normalized to < and \uFE65 is normalized to > using NFKC
String s = "\uFE64" + "script" + "\uFE65";
// Validate
Pattern pattern = Pattern.compile("[<>]"); // Check for angle brackets
Matcher matcher = pattern.matcher(s);
if (matcher.find()) {
// Found black listed tag
throw new IllegalStateException();
} else {
// ...
}
// Normalize
s = Normalizer.normalize(s, Form.NFKC);
The normalize() method transforms Unicode text into an equivalent composed or decomposed form, allowing for easier searching of text. The normalize method supports the standard normalization forms described in Unicode Standard Annex #15—Unicode Normalization Forms.
Compliant Solution
This compliant solution normalizes the string before validating it. Alternative representations of the string are normalized to the canonical angle brackets. Consequently, input validation correctly detects the malicious input and throws an IllegalStateException.
String s = "\uFE64" + "script" + "\uFE65";
// Normalize
s = Normalizer.normalize(s, Form.NFKC);
// Validate
Pattern pattern = Pattern.compile("[<>]");
Matcher matcher = pattern.matcher(s);
if (matcher.find()) {
// Found black listed tag
throw new IllegalStateException();
} else {
// ...
}
Risk Assessment
Validating input before normalization affords attackers the opportunity to bypass filters and other security mechanisms. This can result in the execution of arbitrary code.
|
Rule |
Severity |
Likelihood |
Remediation Cost |
Priority |
Level |
|
IDS00-J |
high |
probable |
medium |
P12 |
L1 |
Related Guidelines
|
ISO/IEC TR 24772:2010 |
Cross-site scripting [XYT] |
|
MITRE CWE |
CWE-289. Authentication bypass by alternate name |
|
CWE-180. Incorrect behavior order: Validate before canonicalize |
Bibliography
[API 2006]
[Davis 2008a]
[Weber 2009]