Vulnerability Note VU#162289

C compilers may silently discard some wraparound checks

Overview

Some C compilers optimize away pointer arithmetic overflow tests that depend on undefined behavior without providing a diagnostic (a warning). Applications containing these tests may be vulnerable to buffer overflows if compiled with these compilers.

I. Description

In the C language, given the following types:

        char *buf;
        int len;

some C compilers will assume that buf+len >= buf. As a result, code that performs wrapping checks similar to the following:

len = 1<<30;
[...]
if(buf+len < buf)  /* wrap check */
  [...overflow occurred...]

are optimized out by these compilers; no object code to perform the check will appear in the resulting executable program. In the case where the wrap test expression is optimized out, a subsequent manipulation of lencould cause an overflow. As a result, applications that perform such checks may be vulnerable to buffer overflows.

Wrapping checks that use methods similar to the one described above depend on undefined behavior. Conforming implementations are permitted to perform the optimization by the ISO/IEC 9899:1999 C specification (§6.5.6p8) as undefined behavior. Even if a conforming implementation currently generates object code for an undefined behavior, future versions of the compiler are not obligated to do the same; this behavior may be viewed as an opportunity for further optimization. To ensure that such changes to the compiler do not invalidate assumptions, developers should follow the recommendations described in CERT C Secure Coding recommendation MSC15-A and rule ARR38-C to avoid this error. Furthermore, compilers are not required to issue diagnostics for undefined behavior, so there is frequently no easy way to identify undefined behavior in code, particularly during manual code audits.

Note that this issue does not strictly constitute a vulnerability in the compilers themselves. Rather, this behavior may introduce vulnerabilities in applications that include similar code and are compiled with affected compiler implementations. Existing code that relies on the undefined behavior in the wrapping check is particularly susceptible to this behavior.

Multiple implementations are known to perform this optimization. This optimization may be affected by the setting of the optimization level as well as other flags. Additional information about affected implementations can be found in the Systems Affected section of this document.

II. Impact

An application that performs wrapping checks based on an expression such as the one described above may be vulnerable to buffer overflow if compiled with affected compiler implementations. The nature of the resulting vulnerability would be specific to the application and depends on how the affected code is used.

III. Solution

Use casts

Cast objects of type char* to uintptr_t before comparison. The faulty wrapping check listed above would be written

#include <stdint.h>
[...]
if((uintptr_t)buf+len < (uintptr_t)buf)
   [...]

Alternatively, developers can use size_t on platforms that do not provide the uintptr_t type. Developers should also follow the recommendations described in CERT C Secure Coding rule ARR38-C.

Avoid affected compiler implementations
Application developers and vendors of large codebases that cannot be audited for use of the defective wrapping checks are urged to avoid using compiler implementations that perform the offending optimization. Vendors and developers should carefully evaluate the conditions under which their compiler may perform the offending optimization. In some cases, downgrading the version of the compiler in use or sticking with versions of the compiler that do not perform the offending optimization may mitigate resulting vulnerabilities in applications.

Systems Affected

Vendor	Status	Date Updated
Cray Inc.	Unknown	23-Apr-2008
GNU Compiler Collection	Vulnerable	2-May-2008
Hewlett-Packard Company	Unknown	17-Apr-2008
IBM Corporation	Unknown	17-Apr-2008
Intel Corporation	Not Vulnerable	25-Apr-2008
Microsoft Corporation	Unknown	17-Apr-2008
Silicon Graphics, Inc.	Unknown	17-Apr-2008
Sun Microsystems, Inc.	Unknown	17-Apr-2008
Sybase	Unknown	17-Apr-2008
Wind River Systems, Inc.	Unknown	17-Apr-2008

References

https://www.securecoding.cert.org/confluence/x/SgHm
https://www.securecoding.cert.org/confluence/x/EoLu
http://archive.cert.uni-stuttgart.de/bugtraq/2006/04/msg00347.html
http://gcc.gnu.org/ml/gcc-bugs/2006-04/msg01297.html

Credit

Felix von Leitner originally published information about this behavior in the gcc compiler. Russ Cox later noticed this behavior as well and provided additional information and assistance.

This document was written by Chad R Dougherty and Robert C Seacord.

Other Information

Date Public	04/17/2006
Date First Published	04/04/2008 09:07:25 AM
Date Last Updated	05/02/2008
CERT Advisory
CVE Name
US-CERT Technical Alerts
Metric	0.00
Document Revision	56