In the Doom games, Heretic, and Hexen, many of the enemy monsters can be tricked into fighting each other, a phenomenon which is generally called monster infighting. This can usually be accomplished by maneuvering into a position where one monster is in the line of fire of another. Since a monster does not have the intelligence to cease fire in the event that a fellow monster blocks his line of sight, it will attack without regard, possibly resulting in friendly fire. The damaged monster may then target the attacker, where combat between the two monsters will soon follow and continue until one participant either dies or is distracted by a third party's attacks. In the Doom games monsters will also retaliate when damaged indirectly by exploding barrels.
Monster infighting has become a novel aspect of Doom and its progeny. Indeed, most games do not give the enemies the ability to hurt each other, much less the ability to retaliate. Additionally, in more recent first-person shooters, enemies are either intelligent enough not to fire if a fellow enemy is in the way, or to avoid the line of fire of other enemies.
Monsters capable of causing infighting
Monsters that have inaccurate, repeating, or blast-damage attacks (zombiemen, shotgun guys, heavy weapon dudes, Wolfenstein SS officers, mancubi, arachnotrons, spiderdemons and cyberdemons) are particularly likely to cause infighting.
Any monster that fires projectiles (the previous plus imps, cacodemons, revenants, Hell knights, and barons of Hell) can be maneuvered into doing this. Lost souls, being projectiles themselves, can damage other monsters found on their attacking path, likewise causing these to retaliate.
Monsters unable to cause infighting
Monsters damaged by an arch-vile's attack will never retaliate because there is an exception in the source code that prevents other monsters from targeting them. This does not stop arch-viles themselves from targeting and damaging other monsters, though. Arch-viles are easy to distract, as, unlike other monsters, their lack of a targeting threshold means they will always switch target when hit.
Pain elementals, whose attack is carried by proxy monsters, cannot provoke other monsters, but will still turn on attackers if provoked.
Hanged Commander Keens do not have an attack mode of any sort, so they can neither provoke other monsters nor retaliate when they are damaged by projectiles.
Also, in Heretic and Hexen, no monster will retaliate against "boss" opponents who accidentally strike them outside of the mass infighting detailed below. Code-wise, this is handled by a flag attached to the boss mobjs.
Melee monsters and infighting
Monsters that are melee attackers cannot initiate monster infighting by mistake. This concerns enemies such as Doom's demons and spectres; Heretic's gargoyles, golems and sabreclaws; Hexen's ettins, centaurs and stalkers; and so on. The source code reveals that melee attacks are impossible to misdirect; the damage is directly inflicted on the target if it is within range. Aiming is not involved and therefore hitting another target by mistake is impossible.
However, Hank Leukart's Official Doom FAQ states that misdirected melee attacks do happen and can trigger infighting. Before v1.5 of the engine, A_SargAttack (the demon attack routine) used a short-range hitscan attack. Demons and spectres retain this behavior in Atari Jaguar-based console ports, including those for the PlayStation and Saturn, leaving infighting between them possible. From v1.5 onward, the hitscan routine is replaced by a simple distance check and direct damage.
Monster infighting is an important tactic in levels where ammunition is very scarce, such as those of Alien Vendetta or when recording speedruns, especially in a restrictive category such as UV Tyson. Causing many monsters to destroy or seriously damage one another greatly reduces the burden on the player's resources. The easiest way to do this with a large group of mixed monsters is to run in a circle several times around the group, which will almost certainly start an infight due to stray projectiles. This is allowed by the unusually high player-versus-monsters speed ratio in Doom engine games that grants the player the ability to maneuver around groups of monsters with more ease than in other first person shooter games.
Arguably one of the most prominent examples of this strategy can occur in Doom II MAP20: Gotcha!. This level happens to feature two of the most powerful enemies in the game (a spiderdemon and a cyberdemon) in the same room. Each is easily capable of dispatching the player, but with quick maneuvering the player can actually cause one to accidentally hit the other, starting a lengthy and entertaining fight that inevitably results in one killing the other, but not before being severely injured, making it much easier for the player to finish the job.
Another advantage is that causing infighting will not alert peacefully waiting monsters like using a weapon does. For example, it may be wise to wake up a demon-spitter as late as possible.
Although arch-viles can only single-sidedly infight another monster, this can still be useful for the player: first, the opponent is being damaged. The only caveat is that once the enemy is dead, it may instantly be resurrected to its full strength, which may be worse than before the arch-vile attack. Second, the arch-vile will follow the other monster instead of the player. A cyberdemon accidentally hitting an arch-vile often leads to the arch-vile moving close to the cyberdemon and into its line of fire, and likely finished off by consecutive cyberdemon rockets targeted at the player.
The strategy of provoking monster infighting is complicated by a few factors.
Primarily, monsters of any species are immune to their own kind's projectile attack (but not hitscan attacks), unless a barrel is hit and damages another of their kind. Hell knights and barons of Hell are of a single species in this sense, and cannot hurt one another with their green fireballs (except indirectly by destroying a barrel). Although a given species of monster is immune to its own type of attack, they still employ it with its customary frequency while targeting one another. The immunity also means that those individuals from the same species cannot actually inflict damage no matter how many times they attack each other except by destroying additional barrels or if they get close enough to each other to use a non-projectile attack, like claws or a bite. Arch-viles are subject to peripheral blast damage from the attacks of other arch-viles on nearby monsters, but do not retaliate because they cannot be targeted by any monster.
Additional factors that lessen monster infighting have to do with some characteristics of the Doom engine's monster AI. Monsters resurrected by arch-viles or spawned by the final boss immediately target the player, not their creator's target. In addition, lost souls damaged by other monsters forget their targets after a single counterattack (whether successful or not) and then turn back to the player, although the monster that the lost soul had damaged does not forget the flaming skull's offense. Certain source ports include various AI options, and allow the user to decide how monsters should behave in respect to monster infighting.
There is another AI related showstopper for monster infighting: sometimes, one of the monsters that can be tricked into infighting is located on another height level. For example, if a Hell knight on the street is hit by an imp behind a first-floor window. In this case, projectile attacks may only be successful from far away. Unfortunately, monsters often first walk up toward the opponent, making it impossible to ever reach the other party. Sometimes, the player can help them by moving into their way.
During an ongoing infight, the player can attack and damage the infighting monsters. However, the infighting monsters will typically continue with the infighting and ignore the player's attacks.
During an infight, the player can naturally be injured by missed attacks and blast damage or simply by walking into the monsters' line of fire.
Monster infighting in Doom 3
Monster infighting can occur in Doom 3 as well. It does not happen very often, though, because of the significantly smaller monster numbers and due to the lack of room where the player could lure the monsters into each others line of fire. Infighting in Doom 3 is further complicated by the monsters following a certain hierarchy. For example, an imp will not attack a Hell knight because of the Hell knight's higher "rank".
True infighting in Doom 3 is technically impossible without mods since the monster hierarchy essentially consists of only two classes: An "upper" class (Hell knights, Revenants) and a "lower" class (Imps, Z-Secs). Only monsters in the upper class are allowed to attack other monsters, and even then, they will only attack monsters of the lower class. The lower class is simply incapable of fighting back. Thus, any "infighting" that occurs is very one-sided.
Note: According to the developer's comments on the files found in game, monsters of equal rank are supposed to be able to fight each other. However, at least in the PC/MAC version of the game, this does not occur for whatever reason.
This information is based on the code files found in the game:
Rank 0: Zombies, Lost Souls
Rank 1: Imps, Z-Secs, Commandos, Pinky Demons, Maggots, Wraiths, Ticks, Trites, Cherubs
Rank 2: Hell knights, Revenants, Arch-viles, Cacodemons, Mancubi, Vagary, Guardian of Hell, Sabaoth
Rank 3: Cyberdemon
Note: It seems strange that Commandos, who are considered quite powerful opponents, would be relegated to the Lower class. This could be due to them being formerly human.
Mass infighting in Heretic and Hexen
Raven Software added a feature to Heretic and Hexen which causes all monsters within sight of the player to begin attacking randomly chosen targets when the player dies in a single-player game. Monsters will fight with any living thing within their sight when this happens, including members of their own species and normally ignored creatures such as bosses (i.e anything shootable, including Voodoo dolls, in which case the player' screen will, despite being dead, still flash red each time the doll is struck). However, monsters remain unable to hurt other monsters of their own type with their projectiles even in this case, as can be viewed in p_map.c line 331, where there is no check as to whether or not the player is dead or alive when rejecting same-species projectile hits.