From DoomWiki.org

XymphBot is an account for use in MediaWiki API queries. It is owned by Xymph, and used via bot scripts written in PHP based on the Wikimate library. After a coding and testing period, it commenced automated edits in October 2016.

The overall goal of these scripts is to put as much computer-creatable content into map pages as feasible, and check that existing content is as complete and consistent as possible, so that human contributors can focus more on crafting creative prose like background information, walkthroughs, trivia and other map-specific content. Also, I like alliteration. :)

Configuration[edit]

The starting point for all scripts is a .ini file that specifies settings and a series of map pages. Typically the list includes all levels in an IWAD or megawad, but combining single-level PWADs in one list is also possible.

The settings for all maps include the various input and output paths, the relevant IWAD, an optional PWAD, the type of DMMPST output and any DMMPST flags, an optional pattern for map view filenames, an optional flag defining whether to put the "notally=1" parameter into the map spots template (for Hexen and Strife levels), the WAD ID for Compet-n and DSDA links, and the WAD's author(s).

The settings per map are the episode and mission, the path to the Doom Wiki page, and optional overrides of that map's PWAD, DMMPST output type and flags, author(s), and script-specific actions.

An optional navboxes section defines the number and style of navboxes (Doom-like, standard Doom II-like or Doom II episodic grouping), and the paths, optional episode names, and map ranges for each navbox.

The .ini file can easily be generated from a list of levels in wikicode (e.g. Doom II's) copy/pasted into a file, and passing it through the wiki2ini.php script. This extracts the list of required map settings and prefixes it with the general settings, so that only a few manual edits remain necessary to result in a .ini file usable by the scripts.

.ini scripts[edit]

skeletonBot.php[edit]

Checks map pages' section headers against the reference map skeleton and shows differences in ordering, spelling, and formatting.

conventionsBot.php[edit]

Checks map pages for a wide variety of spelling, linking and formatting conventions.

secretsBot.php[edit]

Compares DMMPST's list of secret sectors against those in map pages' Secrets sections (or Official subsections) and updates its formatting if necessary, or adds the list if such a section is empty.

pspawnsBot.php[edit]

Compares DMMPST's list of DM/CTF player spawns with facings (directions) against those in map pages' "Player spawns" sections and updates their formatting if necessary, or adds the list if such a section is empty.

dmmpstBot.php[edit]

Uses DMMPST's output to replace map pages' Statistics, Map data or Things sections.
Optionally provides interactive comparison between old and new contents of the pertaining section, prior to submitting the edit.
Alternatively creates new map pages with DMMPST's full skeleton output.

dmsectionBot.php[edit]

Uses DMMPST's output to add "Player spawns" and Things subsections (with a short introduction sentence) to the Deathmatch sections for separate arenas within larger maps. The things tables are generated from WADs with the pertaining levels (linedefs and things) cropped down to just the arenas.
For regular maps without separate arenas, only the "Player spawns" subsection is added, along with the Deathmatch section header (before the Statistics section) if not yet present.
Conversely, for maps without deathmatch starts, the Deathmatch header is removed (unless the section is not empty).
Optionally provides interactive comparison between old and new contents of the pertaining section or page, prior to submitting the edit.

extraCoopBot.php[edit]

Uses DMMPST's logging of additional Things existing in cooperative (for Boom or Hexen flags) or multiplayer mode when compared to single-player mode to add categories Levels with extra monsters in co-op and/or Levels with extra equipment in co-op.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

uploadmapBot.php[edit]

Uploads new or updated map view images with description text, of which the exact phrasing about scale or size and cropping is determined from image sets generated via Omgifol drawmaps. New images are uploaded according to the map view pattern.
Always provides interactive viewing of (old and) new contents of the description, prior to submitting the image and description.

uploadmapDmBot.php[edit]

Performs the same tasks as uploadmapBot.php, but for deathmatch (arena) map views with spawn points. The images are generated from WADs with the pertaining linedefs cropped down to just the arenas, but their things left intact (because cropping them in SLADE 3 changes their order, which would result in incorrect spawn points numbering).

mapviewBot.php[edit]

Checks and updates the map view (link, dimensions, caption, and map spots template) in map pages' Walkthrough sections. Newly uploaded images are linked according to the map view pattern.
Always provides interactive comparison between old and new contents of the map view lines, prior to submitting the edit.

mapviewDmBot.php[edit]

Performs the same tasks as mapviewBot.php, but in Analysis sections for deathmatch maps or in Deathmatch sections for arenas within larger maps, and without the spots template.

navboxBot.php[edit]

Checks and adds the navbox line in map pages' intro sections, updates or strips the nocat option in the map template, and strips the skeleton preamble comment.
Always provides interactive comparison between old and new contents of the intro lines, prior to submitting the edit.

navboxGen.php[edit]

Generates one or multiple navboxes for Doom/Doom II/Heretic map pages based on the navboxes settings, and adds or replaces them in the template space.
Always provides interactive viewing of (old and) new contents of each navbox, prior to submitting the edit.

galleryBot.php[edit]

Checks and updates the gallery in map pages' "Areas / screenshots" sections. Adds the gallery container (and section) if it's not yet present. Applies the standard parameters and trims unneeded image attributes, if any. Also cleans up some minor page issues, if necessary.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

demolinkBot.php[edit]

Checks and updates Compet-n and DSDA links in map pages' "External links" sections, or adds them if applicable. Also standardizes formatting of categories and H2 headers.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

authorBot.php[edit]

Adds one or more authors to map pages' intro sections (optionally omitting wikilinks), along with each corresponding author's levels category at the end. Optionally removes the default skeleton text about custom music.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.
If the page already deviates from the skeleton text, then instead shows the initial intro line and all trailing categories.

partimeBot.php[edit]

Parses par times from (separately extracted) DEHACKED and/or MAPINFO lumps and appends standard sentences to the first paragraph in map pages' intro sections. Mentions the origin lump unless the same time is specified in both lumps (they differ in e.g. D2TWID). Uses general lumps for the main (P)WAD, but can also handle lumps from map-specific PWADs.
Alternatively, if no par time is defined, removes the initial stub sentence.
Always provides interactive comparison between old and new contents of the intro, prior to submitting the edit.

comparisonBot.php[edit]

Checks and updates the map view gallery for comparing PC maps with console counterpart(s) on the latter's map pages: adds the PSX variant if not yet included, appends platform indicators to map captions, and redimensions the gallery per the aspect ratio of the PC image so that its area approximates 90000 px2.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

mapperMaps.php[edit]

In a mapper's article, finds the multi-level WADs and their pertaining maps that are covered in the .ini collection, checks consistency of their names (using approximate matching to detect typos) and authorship both ways, and (re)links them via the {{maplinkgen}} template.
If needed, italicizes a parenthesized string after an individual map entry (usually cooperating authors). Also, formats bullet indentation consistently, and erases trailing whitespace and extraneous empty lines throughout the page. Further, (re)formats DEFAULTSORT and any categories towards the end of the page, one per line.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.
Lastly, lists any maps attributed in the .ini collection that were not found on the mapper's page, suitably formatted for manual addition.
All this can performed on one named page, or on a list of pages loaded from a file. Auxiliary script listCategory.php generates such a list from the Mappers category.

createMapper.php[edit]

From the author registration in the .ini collection, generates the text for a manually createable stub page of a mapper, with preformatted name, alias, DEFAULTSORT (if needed) and category. Organizes their covered projects / contributions by year (optionally with annual section headers), with formatted lists of co-authors per map where applicable. The companion categoryMapper.php script quickly creates a mapper's levels category.

addCategoryBot.php[edit]

Appends a new category to all or selected map pages. Optionally inserts it before an existing category.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

pregreplBot.php[edit]

Performs a preg_replace on all map pages with the given search and replace expressions, for all or a limited number of matches.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

dsdaMapBot.php[edit]

For all map pages' "Speedrunning" sections, collects the current DSDA record via their API in all standard Doom, Heretic or Hexen categories plus the optional ones (NoMo, NoMo 100S, Stroller & Collector), and compiles the table with intro line for the "Current records" subsection -- similar to srUpdMapRecs.php for Compet-n map record tables. Links to each category are not duplicated.
If an existing "Miscellaneous demos" table contains entries, prompts whether to preserve at least one of them. If so, updates the download link(s) to {{dsdaftp}} for the old DSDA site; if not, drops the section.
An intermediate step checks the "Secrets" section or "Official" subsection for bulleted or bolded entries, and uses that for the next NM 100S check. But a secret exit or the absence of monsters & secrets is flagged for pertaining maps in the .ini file.
For a map without secrets, skips the NM 100S category -- but if a record exists in that category and not in NM Speed, then lists the former in the latter row. For a map without monsters and secrets, drops all categories except UV Speed -- and Pacifist if cross-listed into UV Speed. In both cases an explanation line is added at the start of the "Speedrunning" section (replacing an existing line if present).
If no records are in the table at all, an "(absence of)" attribute is added in the datestamp line, and the default edit summary is a Verify; otherwise it's an Update.
Always provides interactive comparison between old and new contents of the section, prior to submitting the edit.

Miscellaneous scripts[edit]

These scripts don't work on a .ini file and its series of map pages as such, but perform other convenient tasks. Many of these don't automatically update pages via the bot account, and only produce output for manual updates.

coveredWads.php[edit]

Collects and lists the names of all single- and multi-level WADs covered by the aforementioned .ini files and scripts, and totals the number of maps included in them. Also totals the number of navboxes created/updated by navboxGen.php, the number of custom things mapping files, and the number of distinct links covered by demolinkBot.php. Its output is the entire "WADs covered so far" section below.

articleSort.php[edit]

Finds article, category and template pages whose titles start with an article ("A"/"An"/"The") and shows whether they contain a DEFAULTSORT tag, named map template, category sortkey, or none. Its output is the entire "Pages starting with an article" page. Thanks to Ryan W for the idea.

textmapStats.php[edit]

Processes an extracted TEXTMAP lump with regular expressions and a little code to generate statistics in Template:Mapdata format, along with formatted lists of secret sector numbers and deathmatch spawn points (if any). Useful for UDMF maps, but obsolete now that DMMPST supports those.

textmapExtra.php[edit]

Processes an extracted TEXTMAP lump with regular expressions and some code to collect monster and equipment counts for single-player and cooperative modes, and lists those where the counts differ. Again, useful for UDMF maps.

textmapSector.php[edit]

For Doom-format maps, DeePsea allows searching sectors for a specific type (e.g. secret) or jumping to a specific sector number in the layout. Editors that support UMDF (like SLADE 3) appear to lack these features. textmapSector searches for a specified sector number in an extracted TEXTMAP lump and computes the approximate center position of the associated vertices, allowing to hover to that coordinate in the layout and thus assist in verifying secret descriptions.

listWikiLinks.php[edit]

Collects links to Wikipedia by templates, interwiki links and direct links, along with the DoomWiki pages they're linked from, in the following namespaces: main, Doom Wiki, Help, File, Category, and Template. Its output is the entire "List of Wikipedia links" page. Requested by Ryan W.

listInterLinks.php[edit]

Collects direct links to sites for which Interwikis are defined, along with the DoomWiki pages they're linked from, in the following namespaces: main, Doom Wiki, Help, File, Category, and Template. Its output is the entire "List of Interwiki links" page. Thanks to Gez for the idea.

listExternLinks.php[edit]

Collects direct links to sites for which external link templates are defined, along with the DoomWiki pages they're linked from, in the following namespaces: main, Doom Wiki, Help, File, and Category. Its output is the entire "List of templatable links" page. Thanks to Gez for the idea.

linkArticle.php[edit]

Performs a search for the specified string in the main and Category namespaces, then in each of the resulting articles wikilinks the first occurrence of the string, if not already linked.
Conversely, unlinkArticle.php removes any wikilinks.
Both always provide interactive comparison between old and new contents of the page, prior to submitting the edit.

destubMaps.php[edit]

From map pages transcluding {{level stub}}, removes the template if the article contains: a walkthrough description in the Essentials section, descriptions for bulleted Secrets (or no official secrets), and complete Map data & Things tables. Conversely, restubMaps.php adds back the template if any such section (typically Essentials) is empty or too minimalistic.
Both always provide interactive comparison between old and new contents of the page, for the actual de-/restub decision by the bot operator, prior to submitting the edit.

updateAuthor.php[edit]

After a mapper article and its associated levels category have been moved (typically to include or remove a full name), go over the map articles in the old category to rename the wikilinked author and levels category.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

addTemplate.php[edit]

Processes all articles linked from a template to append that template. If the template contains an includeonly category, this category is replaced by the template. Also, (re)formats any categories after the template(s), one per line.
Always provides interactive comparison between old and new contents of the page, prior to submitting the edit.

appearMaps.sh[edit]

From the archived things tables, collects/sorts the maps with the highest count of a given thing and emits a prettytable. Because Holy Hell MAP05 is a superset of MAP01-04, the latter are skipped from the table.

top100MemMaps.php[edit]

Processes the maps list of the Top 100 Most Memorable Maps and appends a sentence about the map's place to each page's intro section (taking a trailing 'level stub' into account). Also checks for the presence of the Top100 category.

rankMappers.php[edit]

From all the subcategories under category Levels by author, collects the number of pages, then sorts and prints the 50 highest entries. Its output is the entire "Mappers ranked by map count" page. Inspired by the Most Prolific Mappers topic on the Doomworld forums.

sortCodeptrs.php[edit]

A one-off script to traverse category hierarchy Code pointers, collect all pages, and add DEFAULTSORT tags with out the A_ prefix to support Fraggle's project.

onemanLinks.php[edit]

Processes the reviews index of ONEMANDOOM: WAD Reviews and adds/updates review links to the External links section of matching WAD articles. WADs are matched by title or filename in the .ini collection and otherwise by searching for title, or a mapping list for special cases (such as the Cabal series and Master Levels). The intro paragraph of the article is shown for visual verification that it's the same WAD, then existing review links (via https or template) are updated or a new link bullet is appended.
Always provides interactive comparison between old and new contents of the section, prior to submitting the edit.

doomugLinks.php[edit]

Processes the reviews index of Doom Underground and adds/updates review links to the External links section of matching WAD articles. WADs are matched by filename in the .ini collection or a brief mapping list for minor differences. The intro paragraph of the article is shown for visual verification that it's the same WAD, then a new link bullet is appended unless a templated entry is already present.
Always provides interactive comparison between old and new contents of the section, prior to submitting the edit.

waddsd2Links.php[edit]

Processes all .ini files for the old DSDA numeric ID, looks up the new wadname ID, verifies it exists (via a HTTP HEAD request), updates (sometimes adds) the External link in the corresponding article, and adds the new ID back to the pertaining .ini file.
Always provides interactive comparison between old and new contents of both update types, prior to submitting the edits.

srDsdaFeed.php[edit]

Parses DSDA's HTML feed pages from today back to a given start date to collect WADs covered on the wiki that got new/updated records, and lists the corresponding .ini files for updating the maps' record tables via dsdaMapBot.php.

collectDLlinks.php[edit]

Before March 2021, .ini files didn't contain a link for their pertaining PWAD. This one-off script processes a specific or all .ini files, fetching each article path and extracting the "|link=" option from the {{wad}} template. For an {{ig}} template, uses the "|file=" parameter as is, or for the "|id=" parameter runs a HTTP HEAD request on the Doomworld idgames frontend to catch the redirect to the file path URL. Use this relative idgames path (without .zip), or the full URL for other "|link=" options, to update the .ini file with a new 'link' field for each PWAD.

builtinDemos.php[edit]

From a PWAD's .ini file and previously extracted DEMOx lumps or .lmp files included in the .zip, collects the stats of the demo(s) via a separate LmpStats library, and constructs a 'Built-in demos' section with a table for multiple demos, and for a single-level PWAD also a 'Demo files' section. On a PWAD overview page the 'Built-in demos' section is appended to the 'Content' section if present, and otherwise prepended to the 'External links' section or earlier in the page, always showing a comparison between old and new contents of the page prior to submitting the automatic edit; while for a single-level PWAD the 'Demo files' section and table are merely shown for a manual edit.

pregreplGen.php[edit]

A generalized spin-off from pregreplBot.php, runs a search query (optionally different from the in-page search expression) and performs a preg_replace on all resulting pages with the given search and replace expressions.
Always provides interactive comparison between old and new contents of the section, prior to submitting the edit.

rmUTF8marks.php[edit]

Using the database replica that also powers the wiki growth charts, search page and file titles for UTF-8 BOM characters and list them for manual correction. Also search page content and remove such characters via the API. Alternatively, do the same for the left-to-right and right-to-left marks.

listDeadAnchors.php[edit]

Using the database replica, collect pages in the namespaces main, Doom Wiki, Help, File, Category and Template, and builds lists of anchors (section headers, CSS id's and {{anchor}} templates) as well as links to anchors (page#anchor), then shows pages with links to missing anchors for manual correction. Thanks to Quasar for the idea.

drawmaps.py[edit]

The enhanced version of Omgifol's demo script to draw map views that is used for nearly all maps on the wiki, adding scale, logging and spawn points support.

texpackstats.py[edit]

Collects statistics for a texture pack (or any WAD with textures/flats/etc.): totals for textures, patches, flats, skies (patches containing "SKY" in their names), switches, and animated textures and flats. Optionally also dumps lists of textures, skies, switches and/or animated. Thanks Redneckerz for the idea.

searchSpecial.py[edit]

Searches linedefs/things in one or multiple maps (Hexen or ZDoom format) for a given action special and ID. Primarily used in relation to SecretTriggers that are not directly invoked by another thing or linedef (DMMPST already lists those) but have a Thing ID that is indirectly activated via a script called by a thing/linedef. Another use is to find the linedef that gets assigned an ID by a (start-up) script which can then invoke the SecretTrigger. [How convoluted can one make secret invocations...?]

searchThing.py[edit]

Searches things in one or multiple maps for a given type and lists their positions. Primarily used to check where remapped or new custom things are located prior to compiling a custom mapping file.

searchSector.py[edit]

Searches sectors in one or multiple maps for a given sector and shows its type, tag, and the sidedefs and linedefs that construct it.

slotfile.py[edit]

PK3 archives can contain a maps folder with WAD files by slot, but then the slot lump in the WADs themselves is apparently ignored, and thus is often inconsistent with the WAD slot (RAMP takes this to the extreme). DMMPST and some other tools need consistent, distinct slots in a WAD series, so this script updates the slot lump(s) in extracted WAD file(s) to match their filename(s).

playpal_total.php[edit]

Process a PLAYPAL lump extracted from a Doom-engine WAD and count the number of unique colors, optionally skipping unused palettes and truncating 8-bits colors to 6-bits. Idea originated during a discussion of palettes and the game's color space.

listigf[edit]

A Bash script to search (most of) the .txt files in the idgames archive for a regular expression, show the matching context segments of each matching file, prompt whether to include the file, for the selected files fetch the title and ID from the pertaining page in the Doomworld idgames frontend, and output a bulleted list sorted by the files' timestamps and grouped by their years. At the prompt, a comma-separated list of map names can be entered (with optional co-authors), which is then included in a bulleted sublist for the pertaining file (with italicized co-authors). Used primarily to generate body-of-work sections for mappers.

And more?[edit]

Ideas for future scripts, anyone?

How can you help?[edit]

Create overview pages for all WADs to be covered (e.g. Top 25 Missed Cacowards, Cacowards, List of notable WADs) with information as accurate and complete as feasible, especially the episodic structure (if any), the list of levels, and the use of custom monsters/weapons (see below).

And after a series of skeleton pages has been created, edit them to add any music and other introductory information, and of course walkthrough and secrets descriptions, speedrunning records, and whatever else you can contribute.

Custom things mapping[edit]

For custom things, a description or table of thing types and the new monster/weapon names is particularly helpful, and can be used to provide a mapping file to DMMPST that automatically incorporates the customizations into the Things table. An initial discussion and example mapping was presented for Valiant.

The mapping file should link to description(s) of the custom thing(s) on the WAD's overview page. This can be a simple one-liner but preferably contains a brief entry per thing that mentions which original thing is replaced by which custom one, and may be even more elaborate.

So, to help, please provide such descriptions as well as a short or long mapping table (as the case may be, but it must be accurate and complete) on the WAD's talk page for the following WADs that I already processed previously:

Then I can (re)generate the Things tables on all pertaining map pages. I no longer have time to compile mapping lists, so these are required to make progress.

Going forward, the same is welcome for WADs I am skipping for now because the overview information does not appear to be sufficiently descriptive and complete, and/or the mapping list is missing. This includes:

and (m?)any more I haven't gotten around to yet.

WADs covered so far[edit]

The lists below show the single-level and multi-level WADs that have so far been covered (created and/or updated) via XymphBot and its .ini files. Map counts per WAD and various totals are included (state of 2022-06-05 10:59 CDT).

Single-level WADs[edit]

Multi-level WADs[edit]

Statistics[edit]

Total
Overall map pages 12618
Single-level WAD map pages 574
Multi-level WAD map pages 12044
Multi-level WAD overviews 638
WADs with a custom things mapping file * 233
WADs with built-in/included demos 279
Built-in/included demos 733
Navboxes for map articles 1521
Distinct map authors with articles 824
Distinct map authors w/o articles 907
Distinct links to WADs in idgames archive 1035
Distinct links to WADs on other sites 124
Distinct links to Compet-n 346
Distinct links to DSDA 764
.ini files 666