Numeric overflow crash in tall areas


All versions of the Doom engine, including the variants used for Heretic, Hexen, and Strife, have exhibited limitations with very tall areas. This sometimes simply manifests as rendering glitches (such as the screen filling up with a horizon-like flat), but it can also reliably crash the game. Although not commonly known, these limitations can begin to manifest with as little as 2500 units difference between floor and ceiling. It should be noted that this phenomenon is related to but is distinct from Moiré error, which was restricted to visual effects and occurred at much smaller height differences. That problem was repaired in version 1.4 of Doom, but the greater issue remained.

Although in theory Doom should be able to handle areas up to 32767 units tall, limitations arise when large numbers must be added or multiplied together. In order to save memory and to achieve better cache performance, Doom scales down texture mapping coordinates by 12 bits in order to fit them into a clipping array made up of 16-bit "short" integers. For normal situations this works fine, but if the result of a clipping operation yields a value too small (less than -32768), the game engine forces this out-of-range value into the clipping array and ends up with a very large value instead.

Very large values in the clipping array are equivalent to telling the game engine to draw off the bottom of the screen, and in interest of efficiency, no bounds checking is done on values retrieved from the arrays. This can cause any of Doom's drawing routines to run past the boundaries of the framebuffer, and it may also cause a loop in the function R_MakeSpans to run out of bounds on the spanstart array, overwriting most of Doom's static variable memory (also known as the BSS segment). Any of the three different ways in which this can fail may result in a Venetian blind crash.