Map format
From DoomWiki.org
| Doom level format |
|---|
Source port extensions: |
Map formats are used by the Doom engine, related games, level editors and source ports to describe the contents of a level's geometry, BSP tree, among others. Various official and community-made formats exist, most of which are primarily defined using binary structures in lumps.
Almost all binary-based formats use a unique and unbroken order of lumps, that can help determine the format in question. While some required lumps can be empty, and will be filled in by most source ports as needed, all required lumps must be present and in the specified order for correct data reading behavior.
Contents
Official formats[edit]
Binary-based formats are a compact, sequential structure, suitable for being read quickly by the port in question. However, they are also fragile to unstructured modification and hard to extend with additional features without turning to somewhat convoluted methods, or creating a new format entirely.
A common and easy limit-increasing technique used effectively by almost all source ports, is to address the vanilla Doom's source code use of signed 16-bit integers, when reading the indexes of certain structures, by modifying them to use unsigned 16-bit integers. While not a complete fix for the structural limitations of the binary formats, this fix does raise the maximum ceiling of certain properties, generally from 32768 to 65536 elements.
Doom[edit]
The original format used in the Doom engine is supported in almost all of its forks and derivative projects. It includes a series of lumps to describe basic level geometry (VERTEXES, LINEDEFS, SIDEDEFS and THINGS), as well as the BSP tree (NODES, SSECTORS and SEGS), and the BLOCKMAP and REJECT.
This format is the baseline from which all other binary formats are derived, and is supported on Doom, Doom II, Heretic, Strife, Chex Quest, Hacx, Sonic Robo Blast 2, and every related source port.
It stores most data in 16-bit integers, or eight ASCII characters in the case of names for textures and flats, storing data such as X/Y coordinates, ceiling/floor heights, texture offsets, sector light levels, line specials and line flags, among others.
Reality Engine[edit]
Randy Linden's Super Nintendo port of Doom features a bespoke map format slightly modified from Doom's original format, it optimizes existing binary structures for the size constraints of the console, as well as making certain dynamic elements, such as floors, stairs, lifts, doors and ceilings, into separate lumps.
Hexen[edit]
Developed by Raven Software, the Hexen map format comes with a few notable differences from the Doom format:
- LINEDEFS: Lines now contain parameterized action specials, sometimes colloquially referred to as "Hexen specials". Compared to the base Doom format, these offer a greater degree of customization, at the cost of total base line specials support, since these specials and their arguments are stored in 8-bit unsigned integers.
- THINGS: Similarly to LINEDEFS, it also contains the same parameterized action special structure, triggered under certain conditions, such as when a thing is picked up, or killed, etc.
- BEHAVIOR: A new level scripting lump is now present, storing bytecode from a compiled ACS script. Outside of user control, the presence of this lump is the sole indicator for differentiating between the Doom format and the Hexen format.
Unlike most other binary-based formats, the Hexen format is the only non-Doom format with notable support in Doom source ports, being adopted early on by ZDoom as its de-facto format—before being later superseded by the adoption of UDMF.
PSX Doom[edit]
The format used in the PlayStation port of Doom, developed by Williams Entertainment, makes a few notable changes over the base Doom format.
- VERTEXES: Vertices are encoded in full 32-bit fixed point coordinates, instead of 16-bit integers, due to the need for greater precision of the hardware renderer.
- SECTORS: Sectors have an additional "Color ID" property, pointing to an entry in the global LIGHTS lump. It defines a colored lighting effect for the hardware renderer to apply on all elements of the given sector. Sector brightness levels are stored in an unsigned 8-bit integer, instead of a 16-bit integer. These sectors also feature a set of flags, not previously included in the base game.
- LEAFS: Due to hardware renderer's need of for subsectors to be strictly convex, PSX Doom was the first official port to introduce a solution to this issue with the leafs data structure, which contains additional data to ensure the polygonal requirements of subsectors.
PSX Final Doom[edit]
The PlayStation port of Final Doom uses its predecessor's format as a baseline, with the following additions:
- SECTORS: Sectors do not store the name of a flat lump for ceilings and floors, opting for a 16-bit integer index instead of the 64 bits total needed for an eight character long lump name.
- SIDEDEFS: The structure used for sides was back-ported from Doom 64's development, which uses indices instead of names for the wall textures. Similarly to the aforementioned use of indices instead of names for flats in sectors.
Due to the lack of any new additional lumps, compared to its predecessor, there's no guaranteed way of automatically detecting, in isolation, whether or not a map uses the PSX Doom or PSX Final Doom formats—a trait otherwise present in almost all widely supported formats. While some engines are capable of seamlessly supporting multiple formats, the adoption of either one of these would, by definition, require some additional outside information, or some potential caveat to its implementation. Due to this, any implementing engine may need to make the assumption based on the currently loaded IWAD, and in practice is limited only to ports of the PlayStation versions, such as PsyDoom.
Doom 64[edit]
The format used in Doom 64 and its ports, is based upon the ones used on PSX Doom and PSX Final Doom, including the use of 32-bit vertices, the LEAFS structure and the inclusion of "Color ID" on sectors, etc. But also a few additional changes.
- MACROS: A new required lump, which is used in this format for simple scripting sequences.
- LIGHTS: A modified version of the previously global PSX Doom lump is now part of the Doom 64 map format, allowing for custom color setup on a per-map basis.
- SECTORS: Unlike the previous format, there are now five distinct "Color ID"s used in each sector—targeting floor, ceiling, things, top wall and bottom wall.
PSX Hexen[edit]
Being based on the original Hexen map format, the format used in the PlayStation port of Hexen includes the same parameterized action special system and ACS bytecode, but also has some of the same extensions as PSX Doom:
- VERTEXES: Same as the previous console port formats, uses 32-bit fixed-point coordinates.
- LEAFS: Same as the previous console port formats, includes additional information necessary for the hardware accelerated rendering implementation.
Derivative formats[edit]
Some games not directly related to Doom, or idTech 1, have chosen to use formats either based on, or pulled directly from, Doom. This can be mostly justified by the accessibility of the Doom map format, via the existing free idTech 1 level editors, and familiarity from developers who may have had prior experience working with the Doom engine.
Killing Time[edit]
Developed by Rebecca Ann Heineman, the map format used in Killing Time is a modified version of the Doom map format. It has the notable addition of X/Y offsets on floors and ceilings, and the use of indices for the level's textures.
The remastered release, Killing Time: Resurrected, went on to adopt UDMF as its primary mapping format.
Meridian 59[edit]
The map format used in Meridian 59 is notable for being a heavily modified version of the Doom map format, called .roo or room.
Optional extensions[edit]
Some feature sets are able to extend existing binary formats with completely optional lumps, but otherwise do not constitute their own format.
- The original Strife's conversation system, stored in separated lumps, SCRIPT01, SCRIPT02, etc, corresponding to each existing level.
- The Nintendo 64 port of Hexen uses additional FLOOR01, FLOOR02, etc, lumps to address hardware rendering requirements, similar in purpose to the LEAFS lump.
- The SCRIPTS lump used to store ACS source code, usually placed right after the BEHAVIOR lump.
- The historical GL nodes format, formerly used to address hardware rendering limitations related to the BSP tree.
- DLight-based lightmaps, used in certain advanced hardware renderers.
Community formats[edit]
Universal Doom Map Format[edit]
The Universal Doom Map Format, or UDMF for short, is a community-made, text-based map format designed for flexibility and extensibility. Instead of storing the basic map geometry data in structured binary lumps, it uses the TEXTMAP lump to describe the entire level and requires minimum support for, at least, the same properties stored in the original map lumps. The use of a text representation over binary data also comes with a trade off in storage space, and load time parsing speed, in exchange for the aforementioned benefits of not only being more flexible and expandable, but also being able to surpass the unsigned 16-bit integer limits present on the existing binary format—as such an upper ceiling can be easily reached with large enough, or detailed enough, levels.
Due to the nature of the UDMF lump order, it is also considerably easier to create additional map lumps, as they are simply stored locally with the core map data, thus superseding the aforementioned historical practice of adding separate lumps for such optional extensions.
Maps in this format require at least the following three basic lumps to be usable:
- The map's header lump. Usually named following the standard "MAPxy" or "ExMy" conventions.
- TEXTMAP includes all basic level geometry stored in the specific text-based format, and a UDMF namespace to indicate the expected feature set. Must come after the map marker.
- ENDMAP is used to indicate the end of the lump sequence included in the given level. All other lumps are stored in arbitrary order between TEXTMAP and ENDMAP.
The following lumps are commonly supported in much the same way as their binary format counterparts. While many UDMF-capable engines support the internal building of these data structures, almost all will load these lumps if present. However, there are some notable caveats caused by the need to support the increased limits of the text-based format—which are also possible on the binary formats, but harder to achieve based on their respective upper limits.
- ZNODES stores the BSP tree data in any of the ZDoom extended formats, due to the need for higher precision coordinates. This is the most widely supported lump, and is often a requirement for most UDMF-capable engines and namespaces.
- BLOCKMAP will still work the same on all supporting engines, if provided. However, due to the lack of an extended 32-bit format, the lump itself can only support maps so large, before needing to be omitted by a node builder.
- REJECT also behaves the same as it does on binary formats, if provided. However, due to the lack of a compressed, or "sparse" format, these may easily reach the excess of dozens of megabytes on increasingly larger levels.
Some lumps store information for additional features that are less widely supported.
- BEHAVIOR is the given map's included ACS bytecode, a feature originated in the Hexen map format.
- SCRIPTS is also commonly included alongside BEHAVIOR, being the corresponding original source code for the compiled bytecode. A practice originated in the ZDoom modding scene, due to the early adoption of the Hexen map format.
- DIALOGUE stores data for NPC conversations, a feature originated in Strife, in the form of separate lumps.
- LIGHTMAP enables certain advanced hardware accelerated engines to render world geometry with proper lighting information.
Storage space[edit]
One of the notable trade-offs when using UDMF's TEXTMAP over structured binary data, is that map data will comparatively take up significantly more storage on disk. This can be mitigated by the use of compressed file formats such as PK3, PKE, etc.
Naming convention[edit]
One common point of confusion regarding different map formats and their capabilities is the distinction between "formats" and "feature sets." Map formats themselves only dictate the structure of map data, but not necessarily the semantics, or behavior, of the data stored in them. This is particularly notable when comparing Doom, Heretic and Strife, all of which use the same map format, but use completely distinct groups of linedef specials, linedef flags and sector specials.
The terms Boom format, MBF format, etc, are a colloquial, shorthand way of referring to maps made using the Doom format, with vanilla Doom as a baseline feature set, extended with additional features originated in the Boom and MBF source ports. Thus, the use of such terminology is technically a misnomer.
Due to the "three-dimensional" nature of combining different map formats, feature sets and game types, there exists a triplet-based convention used to refer to each possible, valid combination of the three. This terminology is formatted as Feature set: Game type (Map format), made popular for their use in naming the configuration files used in GZDoom Builder—and still in use, as of Ultimate Doom Builder.
Map format can be any of the formats defined above. Game type refers to the basic game being targeted by the map, this defines the absolute most basic features supported, such as thing types, line specials, line flags and sector specials. Feature set is the complete set of features being used, being either the vanilla baseline for each game, or a group of extensions defined in a given engine's specification.
Not every possible combination of the three is considered valid, or even widely supported. Some feature sets are only defined for certain games, or map formats. The following are some common examples:
| Feature set | Game type | Map format | Vanilla definitions | Extended definitions |
|---|---|---|---|---|
|
|
|
|
|
Gallery[edit]
Doom Builder X's deprecated naming convention.
Ultimate Doom Builder allows to define which configurations are available to choose from.
Selecting from different configurations upon creating a new level.
