WAD
From DoomWiki.org
WAD (which, according to the Doom Bible, is an acronym for "Where's All the Data?"[1]) is the file format used by Doom and all Doom-engine-based games for storing data. A WAD file consists of a header, a directory, and the data lumps that make up the resources stored within the file. A WAD file can be of two types:
- IWAD: An "Internal WAD" (or "Initial WAD"), or a core WAD that is loaded automatically (or from a game select menu on source ports) by the engine and generally provides all the data required to run the game.
- PWAD: A "Patch WAD", or an optional file that replaces data from the IWAD loaded or provides additional data to the engine.
A WAD file can be read and/or edited by many tools, such as WAD editors.
Contents
Historical background[edit]
After the release of Wolfenstein 3D in 1992, fans developed unauthorized mods of the game that changed elements such as levels and monster graphics. Rather than causing id Software to seek to suppress such mods, these efforts encouraged the developers to make Doom more easily moddable.[2][3]:134
The WAD file system is the primary mechanism that allows easy modding by fans. Mods containing new levels, graphics, sound effects and music are natively supported by vanilla Doom without needing any special third party tooling; they can simply be loaded using the -file command line argument. This is in comparison to Wolfenstein mods, where the main game resource files had to be modified in order to make any change.
WAD files contain certain limitations: for example, much of the game behavior cannot be changed without patching DOOM.EXE using a tool such as DeHackEd (modern source ports allow DeHackEd patch files to be included in WAD files). Vanilla Doom does not support replacing sprites from inside PWAD files; older graphical mods would commonly be distributed with a small tool named DeuSF that would work around this limitation (modern source ports have entirely removed the restriction).
Header[edit]
A WAD file always starts with a 12-byte header. It contains three values:
wadinfo_t | |||
---|---|---|---|
Position | Length | Name | Description |
0x00 | 4 | identification | The ASCII characters "IWAD" or "PWAD". |
0x04 | 4 | numlumps | An integer specifying the number of lumps in the WAD. |
0x08 | 4 | infotableofs | An integer holding a pointer to the location of the directory. |
All integers are 4 bytes long in x86-style little-endian order. Their values can never exceed 231-1, since Doom reads them as signed ints. For some ports based on the Jaguar code (PlayStation-derived ports being the notable exception), big-endian order is used instead.
The IWAD or PWAD signature is meant to define whether the file is an IWAD or a PWAD; however, this is not actually checked by the engine. Loading an IWAD as a PWAD is possible, and inversely, loading a PWAD as an IWAD is also possible. For example, CHEX.WAD and TNT.WAD have a PWAD signature despite serving as IWADs.
Directory[edit]
The directory associates names of lumps with the data that belong to them. It consists of a number of entries, each with a length of 16 bytes. The length of the directory is determined by the number given in the WAD header. The structure of each entry is as follows:
filelump_t | |||
---|---|---|---|
Offset | Length | Name | Content |
0x00 | 4 | filepos | An integer holding a pointer to the start of the lump's data in the file. |
0x04 | 4 | size | An integer representing the size of the lump in bytes. |
0x08 | 8 | name | An ASCII string defining the lump's name. The name has a limit of 8 characters, the same as the main portion of an MS-DOS filename. The name must be nul-terminated if less than 8 characters; for maximum tool compatibility, it is best to pad any remainder with nul characters as well. |
Tools should not assume the lump-order in the WAD to be sorted by their byte offset into the WAD.
"Virtual" lumps (such as F_START) only exist in the directory, having a size of 0. Their offset value therefore is nonsensical (often 0).
It is possible for more than one lump to have the same offset value, as well as having offsets that overlap other lump data.
When a modder imports lumps into a WAD from other files, file extensions are not included. Doom's executable examines only the name field to determine whether all required entries are present at launch. This means, for example, that the encoding of a music track may not be immediately obvious if the WAD's intended port supports multiple formats.
Certain characters (such as '\') can cause issues for import and export of lumps from the file system. Tools usually work around these issues by translating to and from different file-system-safe characters.
Typical WAD file:
wadfile_t |
---|
Header |
Lump Data |
Names and pointers of Lumps |
Compression[edit]
The Atari Jaguar port introduced a form of LZSS compression for lumps. If the first character of the name has the 0x80 flag set then it is considered to be compressed. The compressed size of the lump is not stored as the compressed stream has a terminating sequence. It should not be assumed that the compressed data is smaller than the uncompressed size since compression is required for some lumps in the console ports. (Likewise some lumps are required to be uncompressed in order to be read directly from cartridge ROM.)
The stream has a 12-bit sliding window. The format of the compressed stream is a flag byte followed by 8 chunks. The size of a chunk is determined by the flag byte in least significant bit first order. If the bit is not set then it is an uncompressed byte, otherwise it is a 16-bit offset length pair. The first byte of the pair is the upper 8-bits of the offset. The upper 4-bits of the second byte is the lower 4-bits of the offset. The remaining 4 bits are the length to copy. The offset is from the current output position (or from the end of the sliding window) and may overlap. Length is incremented by 1 so it becomes some value 1-16. If length is 1 then it is the end of the stream.
In order to know in advance how much information to read from the compressed lump, console versions of Doom which utilize this compression algorithm take the difference between the offset of the target lump and the next lump in the WAD directory. As a result, all such WAD files are terminated with an empty ENDOFWAD lump which marks the end of the directory, and has an offset equal to the end of the previous lump. This avoids undefined behavior if the last lump in the directory happens to be compressed.
Lump order[edit]
The majority of lumps have no restrictions on where they must be located in WAD files, although there are typically some guidelines to make the file easily readable by other people. For certain lumps, however, the location is crucial.
Map data lumps[edit]
A map in Doom is made up of several lumps, each containing specific data required to construct and execute the map. The first lump gives the internal name of the map. In Doom, this had to be in the form ExMy or MAPxx, where x and y could not exceed 4 and 9 respectively (Ultimate Doom), and xx could not exceed 32 (Doom 2/Final Doom). Other than defining the name of the map, the lump is usually empty but can contain data. The DOOM64.WAD file for Doom 64 EX, converted out of the Nintendo 64 ROM content, contains embedded IWADs in MAPxx lumps. In Hexen, they contain version control information; which is not used by the game but was presumably used by Raven Software's editing tools. SMMU introduced using it to store map information and FraggleScript. The level name marks the start of this map. In order to work properly, the following lumps must follow immediately, in that order, after the level name:
- THINGS: A lump listing all the Things present in this map: their X and Y coordinates, starting angles, type and flags. In the Hexen map format, this lump also contains information on Z-height and a thing's TID, special, and arguments. As with all of these lumps, this list will be generated by your level editor and should generally be left alone.
- LINEDEFS: A list of linedefs, defined by their starting and ending vertices, flags, type, tag, args, and front and back sidedefs (if any). Note: The standard Doom format does not contain args.
- SIDEDEFS: A list of the sidedefs that are linked to the linedefs. These contain the data for what textures appear where on the side of each line, their X and Y offsets, and what sector this side of the linedef belongs to.
- VERTEXES: A list of each vertex in the map, using X and Y coordinates.
- SEGS: A list of line segments called "segs" that connect to form subsectors. Created by a node builder.
- SSECTORS: A list of subsectors, created by a node builder.
- NODES: The node tree which Doom uses to speed up the rendering process. Similar to a vismap in modern 3D games (such as Quake III Arena). Created by a node builder.
- SECTORS: Defines the floor and ceiling heights and textures, as well as light value, tag, and type of each sector in your map.
- REJECT: Optionally compiled by the node builder, this lump contains data about which sectors are visible from which other sectors. Originally, Doom used this to optimize the game speed by skipping AI routines for enemies whose target was in a rejected sector. Some modern source ports do not require this lump any more; ZDoom for example has been designed to work even without this lump present. For compatibility purposes, an empty (0-filled) REJECT lump should be included if nothing else. The REJECT lump can also be used to create certain special effects (sectors into which enemies cannot see, for example) if modified carefully.
- BLOCKMAP: Collision-detection information which determines whether objects in a map are touching.
- BEHAVIOR: Not originally a part of Doom, the BEHAVIOR lump was first used in Hexen and contains the compiled scripts that this map will use. Vanilla Doom and other ports designed for Doom only will crash when this lump is present because Hexen format levels are not compatible with Doom format levels. In Crispy Doom 2.3+ it is possible to load and explore maps in Hexen format, but all interactions with the environment are most probably broken. This lump must be present for Hexen format levels since it is the only way to tell if a map is in Hexen or Doom format.
Console port level lumps[edit]
Some of the console ports of Doom add additional lumps to levels in order to facilitate their extensions to the game engine:
- LEAFS: Present in PlayStation versions of Doom as well as in Doom 64. Indexes non-rendered extra segs added to subsectors by the node builder to track edges of subsectors which do not correspond to linedefs. Enables rendering floors and ceilings as polygons.
- MACROS: Present only in Doom 64. Stores compiled script bytecode for each map. Scripts may run at map start or be referenced individually by linedefs with a macro action flag.
Notes[edit]
The Doom alphas had noticeably different map formats.
- In alpha 0.2, there is only one map, which is defined by a series of lumps named M_AREAS, M_FNAMES, M_LINES, M_PLANES, M_PNAMES, M_POINTS, M_TEXTUR and M_THINGS.
- From alpha 0.4 to the beta, maps had to be within M_START and M_END markers.
- In alpha 0.4 and 0.5, a map's lumps are as such: map header, FLATNAME, POINTS, LINES, SECTORS, THINGS.
- With the press-release beta, the map lumps had nearly acquired their definitive names and formats, but the REJECT lump was not introduced yet.
Flats, Sprites, and Patches[edit]
These three resources must be located between special marker lumps so that Doom knows what it is looking at. Other than defining the beginning and end of a graphics section, these lumps contain no data and are 0 bytes long.
The markers consist of names X_START and X_END, where X is the first 1 or 2 letters of the appropriate resource. For example, sprites should be located between S_START and S_END markers. SS_START and SS_END are usually used for user WAD files.
These markers are required by DOOM:
Start marker | End marker | What | Used by |
---|---|---|---|
F_START | F_END | Flat delimiters | All versions |
S_START | S_END | Sprite delimiters | All versions |
These markers are found in the official WAD files, but are unused by known DOOM engines:
Start marker | End marker | Found in | What | Used by |
---|---|---|---|---|
P_START | P_END | Patch delimiters | All versions | |
P1_START | P1_END | inside P_ range | Shareware, Heretic or Hexen patches | Doom Shareware and up (but not Final Doom), Heretic, Hexen |
P2_START | P2_END | inside P_ range | Registered, Heretic or Hexen patches | Doom Registered and up (but not Final Doom), Heretic, Hexen |
P3_START | P3_END | inside P_ range | Doom2 patches | Doom2, but not Final Doom |
F1_START | F1_END | inside F_ range | Shareware, Heretic or Hexen flats | Doom Shareware and up (but not Final Doom), Heretic, Hexen |
F2_START | F2_END | inside F_ range | Registered, Heretic or Hexen flats | Doom Registered and up (but not Final Doom), Heretic, Hexen |
Patches are not required to have any markers. Some lump management utilities require P_START and P_END.
Console versions[edit]
Console versions based on the Atari Jaguar code base use an additional marker-defined namespace for storing textures:
Start marker | End marker | What | Used by |
---|---|---|---|
T_START | T_END | Textures | Jaguar, 32X, 3DO, GBA, PSX, Doom 64 |
Each texture in these versions of Doom consists of a single patch between these markers (in differing formats per platform), and the TEXTUREx lumps used by the DOS version do not exist.
Miscellaneous[edit]
Some lumps are known by their names and apply to the game as a whole. Some of these are:
- Sound effects.
- Music.
- PLAYPAL: Color palettes for various situations.
- COLORMAP: Map to adjust pixel values for reduced brightness.
- ENDOOM: Text displayed when vanilla Doom exits.
- TEXTURE1, TEXTURE2, PNAMES: Data defining the wall textures.
- DEMOs: Recorded games, auto-played before any level is started.
Uses outside the Doom engine[edit]
Due to Doom's popularity and the simplicity of the format, many third party tools were written to manipulate WAD files very early on. This encouraged other game developers to adopt it even for unrelated games. Some even made use of the map format, though the values associated to thing, line, and sector types are usually completely different.
- A.D. Cop and A.D. Cop: Overseas Missions (IWAD/PWAD signature replaced with PACK, additional lumps added to map format)
- Betrayal in Antara some of the ETC resource files are actually PWADs
- Amulets & Armor (only stores maps in WAD files, resources are in other containers)
- Birthright: The Gorgon's Alliance (only stores maps in WAD files, resources are in other containers)
- Doom 2D (stores resources in WAD, but with different image and sound formats. It also supports LMP and PLAYPAL lumps)
- Mars 3D (uses MAD files to store graphics and maps, function identically to WAD)
- Rise of the Triad (stores resources in WAD, but with different image and sound formats)
- Tank Wars (uses WAD as container)
The .wad extension has also been used for different formats. Quake uses the "wad2" format to store various graphic resources, this is an evolution of the Doom wad format with a modified directory entry structure. Half-Life's GoldSrc engine uses the "wad3" format, derived from and compatible with Quake's wad2 format, to store textures. Also of note is that the Nintendo Wii uses a format also called WAD for many files such as WiiWare games and system files, but this is coincidental; it is an entirely unrelated file format.
See also[edit]
References[edit]
- ↑ Hall, Tom (28 November 1992). "Doom Bible." Doomworld. Retrieved 8 October 2017.
- ↑ Romero, John (8 October 2017). "Castle Smurfenstein in 1983 was a mod of the original CW. Ppl modded 1992 Wolf3D and caused us to make DOOM open & moddable. Interesting." Twitter. Retrieved 8 October 2017.
- ↑ Kushner, David. Masters of Doom: How Two Guys Created an Empire and Transformed Pop Culture. New York: Random House, 2003.