Starting on a vertex between visplanes causes illegal memory access


Under certain circumstances, if a player start is placed on a vertex between visplanes, vanilla Doom will crash due to an illegal memory access.


If the player starts on a vertex on a two-sided line, and the sectors on either side can each be rendered as different visplanes (e.g. differing heights or light levels), vanilla Doom performs an illegal memory access. Depending on the environment, this may throw an exception 0Eh (page fault) in DOS/4GW.


The cause is reportedly due to a bug in the R_PointToDist function in r_segs.c, but the exact nature has yet to be verified.

Possible explanation by Quasar:[1]

All I can figure out is that vanilla DOOM must install a handler that causes integer division by zero exceptions to be effectively ignored. Unfortunately it apparently leaves the value in edx:eax undefined after the idiv instruction, which then leads to out-of-bounds indexes into tantoangle and/or finesine. What happens from there's anybody's guess. You say visplanes seem to influence it; I cannot rule out or confirm that myself :)
BTW this was independently fixed by Jim Flynn in BOOM 2.02 and Lee Killough in MBF, in slightly different ways, so that R_PointToDist returns 0 if you are on the seg's v1 vertex instead of dividing by zero. BOOM once again proves 0/0 == 0 apparently.

Boom 2.02 and MBF fix this bug.



  1. Post by Quasar on BTSX thread