Starting on a vertex between visplanes causes illegal memory access

From DoomWiki.org

Revision as of 14:19, 28 January 2013 by Gez (talk | contribs) (templated links)


Under certain circumstances, if a player start is placed on a vertex between visplanes, Vanilla Doom will crash due to an illegal memory access.

Details

If the player starts on a vertex on a two-sided line, and the sectors on either side can each be rendered as different visplanes (e.g. differing heights or light levels), Vanilla Doom performs an illegal memory access. Depending on the environment, this may throw an exception 0Eh (page fault) in DOS/4GW.

Cause

The cause is reportedly due to a bug in the R_PointToDist function in r_segs.c, but the exact nature has yet to be verified.

James Haley's explanation:

"All I can figure out is that vanilla DOOM must install a handler that causes integer division by zero exceptions to be effectively ignored. Unfortunately it apparently leaves the value in edx:eax undefined after the idiv instruction, which then leads to out-of-bounds indexes into tantoangle and/or finesine. What happens from there's anybody's guess. You say visplanes seem to influence it; I cannot rule out or confirm that myself :) BTW this was independently fixed by Jim Flynn in BOOM 2.02 and Lee Killough in MBF, in slightly different ways, so that R_PointToDist returns 0 if you are on the seg's v1 vertex instead of dividing by zero. BOOM once again proves 0/0 == 0 apparently"

Boom 2.02 and MBF fixes this bug.

Sources