Chapter 2.3, Problem 2.37PP

Practice Problem 2.37 solution page 155

You are given the task of patching the vulnerability in the XDR code shown in the aside on page 100 for the case where both data types int and size_t are 32 bits. You decide to .eliminate the possibility of the multiplication overflowing by computing the number of bytes to allocate using data type unit64_t. You replace

In 2002, it was discovered that code supplied by Sun Microsystems to implement the XDR library, a widely used facility for sharing data structures between programs, had a security vulnerability arising from the fact that multiplication can overflow without any notice being given to the program.

Code similar to that containing the vulnerability is shown below:

The function copy_elements is designed to copy ele_cnt data structures, each consisting of ele_ size bytes into a buffer allocated by the function on line 9. The number of bytes required is computed as ele_cnt * ele_size.

Imagine, however, that a malicious programmer calls this function with ele_cnt being 1,048,577 (2²⁰ + 1) and ele_size being 4,096 (2¹²) with the program compiled for 32 bits. Then the multiplication on line 9 will overflow, causing only 4,096 bytes to be allocated, rather than the 4,294,971,392 bytes required to hold that much data. The loop starting at line 15 will attempt to copy all of those bytes, overrunning the end of the allocated buffer, and therefore corrupting other data structures. This could 1 cause the program to crash or otherwise misbehave.

The Sun code was used by almost every operating system and. in such widely used programs as Internet Explorer and the Kerberos authentication system. The Computer Emergency Response Team , (CERT), an organization run by the Carnegie Mellon Software Engineering Institute to track security vulnerabilities and breaches, issued advisory “CA-2002-25,” and many companies rushed to patch their code. Fortunately, there were no reported security breaches caused by this vulnerability.

A similar vulnerability existed in many implementations of the library function calloc. These have since been patched. Unfortunately, many-programmers call allocation functions, such as malloc using arithmetic expressions as arguments, without checking these expressions for overflow. Writing a reliable version of calloc is left as an exercise (Problem 2.76)

the original call to malloc (line 9) as follows: