Doom Wiki:Central Processing

From DoomWiki.org

Revision as of 19:44, 23 January 2020 by Ryan W (talk | contribs) (Re: List of notable WADs)

This is the central discussion forum for wiki editing and administration activity on the Doom Wiki. Feel free to ask any questions or pose any concerns you have here, and you should receive a response shortly. Check the archived discussions for older threads. For extended discussion on long-range "to do" issues and project planning, please also visit our Request For Comment hub.

Archived discussions


Doomsday Wiki

The Doomsday Engine Wiki (https://dengine.net/dew/) is no more, replaced by a Doomsday Engine Manual (https://manual.dengine.net/). It's not just an address change, it's the entire thing rewritten from scratch with a different organization, so there's no 1-to-1 mapping between DEW articles and DEM articles. That means we have to find and update every dengine.net/dew links on our wiki. --Gez (talk) 13:39, 13 December 2018 (CST)

Sounds like a plan. Make it so. :) --Xymph (talk) 03:01, 15 December 2018 (CST)

Update: I believe I have fixed this.  Please somebody check my interwiki changes, which affect WikiNode and the template, probably the two most important cases.

I'm far from an expert but it seems our IRC conversation was correct: the number of existing links was small to begin with.  Hopefully that changes someday; there are certainly people in the community who considered it the best port, and even modded for it.    Ryan W (living fossil) 09:19, 5 April 2019 (CDT)

2019

Happy New Year everybody! --Quasar (talk) 00:13, 1 January 2019 (CST)

competn

Hi all, I see you managed to pull script for competn demos. Very nice! :) If you need some help in any kind I'll be pleased to help and give you info how to get info. I can even provide direct sql reads to database if anyone is interested for example to show only current records per wad/map/etc. --Fx

Hi, not sure why you think "pull script" applies to demos tables. To my knowledge, they were all created/updated manually over the years.
Anyway, scripting XymphBot updates of that information is on my to-do list and received prior discussion already (and for DSADA too). Your kind offer of direct SQL access (or just a, say, weekly export that I can load and process locally) would ease one major part of that project considerably. But currently two non-wiki projects serve as my time sink ;-) so it will be quite a long time before I would be able to turn my attention to this project. --Xymph (talk) 05:05, 23 January 2019 (CST)
I was thinking about "External links" category where you made auto linking like this: * MAP01 demos from the Compet-n database: UV speed • NM speed • UV max • NM100S • UV -fast • UV -respawn • UV Tyson • UV pacifist which is rather cool. There is incoming log file that can be parsed https://www.doom.com.hr/public/compet-n/admin/incoming.log but there can be multiple wrong uploads. In the end query to latest record is possible as I have column just for that (current_record). Anyway, if I could help you somehow, feel free to ask whatever is on your mind. --Fx (talk) 18:02, 23 January 2019 (CST)
Ah yes, those IDs are part of XymphBot's .ini system and the links are generated by the demolinkBot script. Thanks for the offer, once my plate is sufficiently cleared of ongoing and unfinished projects (wiki and otherwise), I intend to return to this topic. --Xymph (talk) 13:02, 23 January 2019 (CST)

Influence of certain maps

Would anyone be in favour of adding more info to certain map pages showing which other maps they have inspired as I have done for MAP24: Post Mortem? I think it would be good addition and would prevent the map pages becoming just a bunch of walkthroughs and map stats.

Off the top of my head other ones that could be added are:

Let me know if there are any more examples of this. —The preceding unsigned comment was added by Rootof2 (talkcontribs) .

Could be interesting. Only issue is maybe making sure we don't stray too far into speculation. An example I know of right off the bat is MAP21: Slayer (The Plutonia Experiment), which is obviously a "hard type" version of MAP11 from Doom II. Then in turn, many Plutonia 2 maps expand directly on the theme of a map from Plutonia. Sometimes this is mentioned and other times it is not. --Quasar (talk) 11:19, 31 January 2019 (CST)
I agree we should not stray too far into speculation but I think with the examples you and I have provided direct comparisons between the maps can be made. I think overall though it would provide valuable information to the reader about the history of mapmaking and how it has changed over time. --Rootof2 (talk) 11:30, 31 January 2019 (CST)
I like this.  Any topic benefits IMO by showing interconnections between individual events, and mappers are always saying they study the best of others' releases to learn from them.
Some speculation may be unavoidable, especially for older maps where the development documentation is gone, and there wasn't this community compulsion to give everyone proper credit like there is now.  Moreover, creative people may imitate without even realizing it, because the prior author's work affected them so much, and some of course try to keep it secret, thinking you won't download otherwise.  I agree with Quasar that it can get out of hand (our "music inspirations" threads/sections were such a lightning rod that eventually admins just started erasing content), so you'll need to be patient and perhaps try not to attract attention  :>  until you have a sense of where the balance lies.  FWIW, Ryan W (living fossil) 14:13, 1 February 2019 (CST)

OpenGraphMeta SEO additions

I've added some new features to our customized OpenGraphMeta extension:

  • Articles now have schema.org-compliant JSON-LD metadata - supposed to be good for SEO.
    • Also ties the wiki to corresponding social media profile(s).
  • Can define custom meta keywords on the page (will probably make a template for this and do some limited deployment eventually).
    • The default meta keywords are now defined on the wiki (message seo-default-keywords) instead of inside our LocalSettings.php file.
  • Support for meta itemprop tags in the page header (of limited use thus far).

Most of the code for this stuff came from Curse, which I found out open sourced their stuff under GPL not very long ago. --Quasar (talk) 03:07, 28 March 2019 (CDT)

Category for maps with extra monsters in co-op?

A little while ago a category was made for maps with a deathmatch area separate from everything else. I thought maybe if it's worth having that then it might be worth having this too. Is there an easy way of finding maps that fit the criteria? - SiFi270 (talk) 11:00, 28 March 2019 (CDT)

If there is support for this idea, then I could write a XymphBot script to accomplish it. --Xymph (talk) 09:52, 31 March 2019 (CDT)
Unlike some recent proposals, this one admits to an algorithm, so I'd support XymphBot carrying it out if resources allow.  I can see it being relevant to co-op devotees (FWIW given I'm not one, except in Gauntlet).    Ryan W (living fossil) 10:49, 31 March 2019 (CDT)
I've added the functionality to DMMPST to list the things where extra items occur in coop compared to single-player. This is generalized so the list(s) can be produced not just for monsters but also weapons, ammo, items, etc. The list(s) in turn will be used in a XymphBot script that modifies map articles to add the new category.
But before I move on to that, this raises the question whether an additional category, e.g. "Levels with extra equipment in co-op", would be worth adding too? MAP01: Outpost (Memento Mori II) has some extra monsters but also an extra shotgun. MAP17: No One (Memento Mori II) has heaps of extra monsters, weapons/ammo, health & armor.
Caveat is that on maps with a separate deathmatch arena with piles of stuff, such as MAP07: Frustration (Memento Mori II), DMMPST cannot determine which items are extra specifically to coop. (The DM things table on such articles was generated from a separate, specially cropped version of the WAD/map.)
And the second question raised is whether the extra item counts should actually be listed in a table in the article? And if so, where/how? --Xymph (talk) 16:33, 9 May 2019 (CDT)
IMO the equipment category is good; if we're already adding the monster category, we might as well be thorough (easy for me to say of course  :>
I'm mightily undecided about tables.  In theory, presenting more data is helpful, but based on previous thing table discussions, there seems to be a tipping point where the layout becomes cumbersome to read.  One could imagine alternate approaches, e.g. a co-op strategy subsection following the deathmatch part, where the important pickups would be pointed out within that narrative.  OTOH contributors aren't exactly lining up to do such writing and testing, and if they did, the table wouldn't prevent it.
This may be recentism, but my impression is that people just play with item respawning anyway, because they want to be able to choose any map with co-op starts, not just the tiny subset where co-op is integrated into the entire design.  HTH.  Ryan W (living fossil) 08:53, 19 May 2019 (CDT)

← ← ← ←
This little project is completed, except of course for map articles not yet covered by .ini files because help is needed. --Xymph (talk) 06:08, 10 June 2019 (CDT)

Hell Guard editor

Hi! I'm Shotgun2016, I just joined, and I religiously play DOOM 2016. I edited the Hell Guard page and many others before joining, and I'm looking for some feedback. I've took an oath to log in at least once a week, so I'll reply quickly to any feedback. — —The preceding unsigned comment was added by Shotgun2016 (talkcontribs) .

Attribution question

Moved from here. — Ryan W (living fossil) 16:33, 11 April 2019 (CDT)

I am a new user, and edited articles before creating an account, such as the Hell Guard. How do I identify these as being my works? —The preceding unsigned comment was added by Shotgun2016 (talkcontribs) .

You could edit the unregistered "user page" to state this, with a link to your registered name.  At least, that's been done before (I can think of more drastic approaches requiring admin assistance, but I have no idea if they would work, or be acceptable to the community).
Remember that the wiki database already considers you identified by your IP address.  It was your choice whether to create an account first, you decided not to, and the software respects that decision for increased transparency.  HTH.  Ryan W (living fossil) 10:24, 12 April 2019 (CDT)

Thanks. I apologize for my amateur wiki abilities; I hope to compensate for this by having a good knowledge of DOOM 2016. --Shotgun2016

latest archive.org wiki dump available

I do this roughly annually, this is the third dump. https://archive.org/details/wiki-doomwiki.org-20190408 -- Shambler (talk) 07:52, 15 April 2019 (CDT)

FlaggedRevs configuration change proposal

Dissatisfaction with the settings for autopromotion have been expressed since go-live back in 2011 but I never really figured out what to do with it. My current suggestion: cut the page edit requirements roughly in half:

Variable Current Target
edits 250 125
totalContentEdits 300 150
totalCheckedEdits 200 100
benchmarks 14 7

These feel like they give more than enough time, along with the other requirements like using unique edit summaries, having a max 3% rollback/revert rate, etc, to determine if somebody is trustable enough to get autochecks. --Quasar (talk) 21:53, 5 May 2019 (CDT)

To see what all the variables mean, go here and search for wgFlaggedRevsAutopromote.
Established community members should never be obstructed by this, because any admin can confer the permissions manually.  I promoted three people in 2012 to address the backlog, with zero drama (though none had time to help either sadly).  I can't judge in this case, as I don't recall any previous interaction, although one more contributor who knows disassembly can't be bad!  :>
Still thinking about whether a config change is worth the effort.    Ryan W (living fossil) 17:45, 6 May 2019 (CDT)
There is no effort, it's literally 4 lines to add in our LocalSettings.php. What I need ultimately is your opinion on the reasonableness of the amount of edits required. Manual promotion is neither really "here" nor "there" with respect to changing this. If what you're saying is that you'd prefer there be no autopromotion, then we can just disable it. But then people have to be on top of determining who gets it and when at all times. Currently I don't see that has been happening. If 2012 was the last time it was used, it's definitely not. --Quasar (talk) 09:17, 7 May 2019 (CDT)
I've opposed autopromotion in the past, which IIRC was on two grounds.  (1) User rights changed systematically without consensus or even announcement.  I suppose that's water under the bridge, and low overall participation is the most pressing issue, so having more (potential) reviewers can only help us.  (2) The filter can be, and has been, fooled by edits that are borderline helpful but also self-aggrandizing, plagiarized, full of markup errors, etc.  This is exacerbated by reviewers who interrupt their current task to "complete" the edit rather than reverting, thus decreasing the revert percentage.  In hindsight, however, such situations are too rare to build process around.
Your argument on IRC about the "fairness" of an automatic approach is persuasive — many community members are wary of subjectivity and groupthink in admin decisions — and you may have convinced me.    Ryan W (living fossil) 22:45, 7 May 2019 (CDT)
What is the objective of this system? Curb vandalism so it doesn't show up automatically? Have editors fact check stuff added in new edits? Have editors review the style of new edits? A mixture of the three? If it's mostly the first one, then the requirements can be cut even more. Also, someone who is not familiar with how this site operates might think that edits are rejected often and then be reluctant to contribute. --Kyano (talk) 07:24, 7 May 2019 (CDT)
All of the above. --Quasar (talk) 09:17, 7 May 2019 (CDT)
2nd is especially dangerous: it means that some edits will never be approved for fear of them being wrong. Fact checking whether Doom was made by either "id software" or "Sun Microsystems" is easy, fact checking whether a certain projectile in some obscure dehacked mod does 32 damage instead of 40 is hard. Many editors will avoid that kind of responsibility. --Kyano (talk) 16:22, 7 May 2019 (CDT)
In practice we haven't let it become that level of an obstacle. It did at one time, with trying to verify the hashes relevant to the Xbox and PS3 versions of IWADs, but we went ahead and accepted those eventually with the understanding that they are correct as far as anybody can reasonably know, and always subject to further adjustment in the future.
I feel like this is more a debate on the merits of the extension than on my suggestion to halve the autopromotion requirements. Unfortunately with regulatory things being the way they are now (SESTA, namely) that I'd prefer by far to keep things as they are right now. Having the ability to review content before it can be broadcasted out to search engines is critical at this point. --Quasar (talk) 19:54, 7 May 2019 (CDT)
(edit conflict) We don't know why so few contribute, but yes, I'd guess delays don't help.  New content is visible instantly on forums, social media, even Wikipedia where newbies are cannon fodder.  Last I looked, the most common complaint is not rejection, but an edit sitting in limbo for days until someone has time to verify all content, all technical data, all style rules, proper use of templates, disambiguation, page title conventions, cross-referencing, and oh yeah it's not spam or vandalism.  This is IMO not how the tool is meant to be used, and I plan to address that in a future proposal.    Ryan W (living fossil) 16:26, 7 May 2019 (CDT)
It's been a long time since there was a review delay that long. I check the site multiple times per day when it's physically possible to do so. If you haven't seen me do it in a day or two, it's because my work is pile driving me into the ground. Darn day jobs. --Quasar (talk) 20:10, 7 May 2019 (CDT)
TLDR: My gut feeling is that this is an improvement and should happen, but I don't have a sense of the exact number.
New users arrive with huge variation in how ready they are for a collaborative project.  User:Goyuken and User:Shidou were patient, thorough, and open to discussion from day one.  Conversely, I made about 2000 edits before I had a clue (some say I still don't).  Applying one value across the board is a judgement call as to which type of newbie should be harmed least, and whatever choice is made, edge cases will still need individual correction.
One argument for a lower number is that, in practice, harmful behavior is clear immediately rather than erupting suddenly after scores of good edits.  A high threshold alienates more good-faith newbies and increases Editor workload without really affecting security.
One argument for a higher number is that edit count is a very blunt instrument, compared to consultation with an experienced human moderator.  Therefore, if we use an algorithm, it should be somewhat conservative to prevent skilled trolls, hucksters, codependents, etc., who don't provoke immediate suspicion, from being promoted quickly.  (It already happened once at 300 edits!)  Also, permissions tend to be added to groups over time, so in effect we're evaluating users now for the more powerful "Editor" toolkit a decade hence.
Frankly I'm hoping we hear from more non-admins, who actually deal with these restrictions day after day.    Ryan W (living fossil) 22:45, 7 May 2019 (CDT)

Replace /idgames/ links with Doomworld download links

Should we consider changing all of the /idgames links to Doomworld Downloads? I assume the entire point of the Downloads is to eventually supersede /idgames entirely, and although it still seems to be a WIP it might be good to get people used to using them instead. Don't know how fast it would be to change each link though... Death Egg (talk) 18:14, 28 June 2019 (CDT)

Very, very, very slow.  It's a good question though, since the legacy pages could vanish overnight (again) and we'd like to have a plan for that.
IMO we should keep the gamers.org path format because it doesn't tie us to one front end with a unique URL scheme.  The path format allows a user to download from any mirror if DW is down, or too far away to be reliable, or the user's connectivity is so slow that any Invision site is unbrowsable.  It's useful to show the reviews and readmes as advance information also, but not as a substitute for a stable file link.
Once Linguica completes the reorganizations in progress, perhaps he would be willing (or accept code from someone who is) to implement URL redirection as was done for thread IDs.  Speaking as a non-programmer, this seems reasonable because the rendered page for each file already includes download links with paths.  The crosswalk exists, it only needs inversion.  But it's his choice and if he decides not to be wiki-friendly, I would rather obfuscate reviews than create 10,000+ links which might break with the next upgrade and require manual revision (we already have too many of those).
If someone thinks I'm totally wrong and has their heart set on implementing this, may I at least suggest waiting until the domain migration?  For all we know, the URLs will change again.
Thanks to Xymph (on IRC) for helping me sound less dumb.  :>  HTH.  Ryan W (living fossil) 17:27, 29 June 2019 (CDT)
The "new" download system on Doomworld is extremely broken and has been suffering from database corruption for several years now. I do not see it becoming a reliable replacement for the old idgames database any time soon. --Quasar (talk) 20:09, 29 June 2019 (CDT)

Deletion tracking, followup

I would like to implement this proposal at last.  The plan goes beyond what we discussed IMO, so I'll post it here first:

  1. ✔ Review the current category and note any threads that look "routine", but don't edit them yet.
  2. ✔ Create category Deletion precedents, a subcategory of Talk pages without articles (which describes the database situation only, irrespective of thread content) and Article management (analogous to the resolved RFCs category).
  3. ✔ Change the archiving template to point to the new category unless an optional parameter, trivial=, is true.
  4. ✔ Update the template instructions.
  5. ✔ Wait 24 hours for the backlog to enter the new category.
  6. ✔ When a talk page has no article, and the article has never existed, don't categorize it.  This is a corner case, but if information is so incomplete or uncertain that even a sub-stub benefits from consultation, so be it.
    • EDIT:  ✔ 6a. When a talk page has an article, don't categorize it.  Also a corner case but can occur, e.g. a term with multiple meanings exhumed as a redirect.  17:27, 23 October 2019 (CDT)
  7. ✔ Find uncategorized talk pages of deleted articles, and tag them with trivial=yes.  Note in passing any threads that do indeed appear substantive.
  8. ✔ Update the admin procedure for future threads.
  9. ✔ Wait 24 hours for the parent category to re-populate.
  10. ✔ Perform the tweaks in #1 and #7 (i.e. separated from the mass edit), and link them here in case anyone wants to double-check.

A thread can constitute a precedent even if the article was kept or later recreated (not sure why I didn't include that earlier).  I'll mention those cases in the documentation and categorize pages I remember, but filling in the backlog systematically is another matter; many older nominations didn't use templates at all.

Categorization tasks are never as simple as they appear!  :D  HTH.  Ryan W (living fossil) 22:17, 22 October 2019 (CDT)

I had forgotten that redirects were an exception to #6a, as described here.  IIRC they are rare, but my spreadsheet treated them as full-size pages, so I'll have to do an additional pass at the end.  Ryan W (living fossil) 17:39, 30 November 2019 (CST)

I haven't followed this process in detail, but in view of the "manually triaging 4000+ redirects" remark: would you need bot script assistance with that?
Btw, in item 7 above, instead of "=yes" you used "trivial=1" so far. --Xymph (talk) 03:11, 8 December 2019 (CST)
The offer is appreciated, but no, spreadsheet functions should handle it easily.  If any on-wiki task could ever be urgent enough that a script was required to bypass category links lag, this isn't it. :>
These steps may have been posted before actually looking up which boolean formats are supported currently.  0 and 1 are always safest.    Ryan W (living fossil) 14:47, 8 December 2019 (CST)

This should now be complete.  As noted above, here are my non-algorithmic taggings — the new parameter is often a judgement call, so I'm collecting these for easy review.

  • Move threads from non-trivial to trivial category (step 1): [1]
  • Move threads from trivial to non-trivial category (step 7): [2] [3] [4]
  • Tag non-trivial threads where the page was kept or later recreated (step 6a): [5] [6]

A very small number of cases were skipped because

  • the talk posts were so vague/tentative that I couldn't tell whether a change was seriously proposed;
  • only placeholder items like this were present, their fate having been decided elsewhere; or
  • they were in user space, where it's generally more trouble than it is worth to intervene.  (That said, the log seems to show every case to date being an obvious speedy anyway.)

I occasionally stumbled over case sensitivity and non-ASCII characters (which I don't recall being issues previously), so it's possible I didn't locate 100.00% of the threads, especially for redirects.

To be clear, this is a one-time job; I don't intend to chase everyone around waving the template.  For one thing that's irritating  :>  and quite far from mission-critical.  More importantly, people who close discussions are trusted to know the applicable process, even when they intend to deviate from it to serve common sense.

HTH/KUTGW everyone.  Ryan W (living fossil) 00:48, 9 December 2019 (CST)

Idgames template zoo, followup

I am starting to think about this task.  My first brainstorm is to leave IRC because it engenders rash promises.  :>

We currently have:

  • Template:Idgames (883 transclusions, 11 links)
  • Template:Ig (1117 transclusions, 4 links)
  • Template:Ig2 (2 transclusions, 5 links) Marked as deprecated in 2016.  Functionality is a subset of Template:Idgames.  Therefore, should be completely safe to replace.  Ryan W
  • (redirect) Template:Idgames2 (no transclusions, 2 links) This one is no longer used, so it can be a normal speedy deletion.  Ryan W

I don't have a proposal yet, but (per Death Egg's thread above) I tentatively believe that

  • a single template could support all features, using parameters like linkonly=
  • some mass edits will be slightly too large to do manually
  • we should excise Doomworld's ID numbers while we're here
  • listing affected pages is relatively easy with a maintenance category
  • any spidering must be heavily throttled to avoid pissing the webmaster off
  • after Doomworld's domain migration, the new template could be expanded to allow a front end URL (assuming those continue) and maybe even a /newstuff parameter
  • script development is a finite resource, so if it's not available right now, that's completely fine

Other contributors use these links far more often than I do, so hopefully they will share their thoughts here.  Thanks, Ryan W (living fossil) 23:51, 20 December 2019 (CST)

The first step should be to eliminate the little-used ones -- basically, everything that's not {{idgames}} or {{ig}}. This can be done manually, as there aren't that many of them. The {{idftp}} templates lost their reason to exist separately when the id FTP server went offline.
The second step would be to normalize on either idgames or ig. Since ig is the most used, I'd suggest using it as the baseline to shave off ~230 edits. This task would have to be made by a bot. Things to watch out for are the different default link text (pagename for idgames, "Doomworld/idgames" for ig), this behavior could be made conditional on the value of the linkonly parameter (if true, assume the default is dw/ig, if false, assume pn). --Gez (talk) 03:29, 23 December 2019 (CST)
Quasar listed some cons to IDs, which I agree are valid concerns, but having worked with the templates and their id=/file= parameters a lot, I feel some pros of IDs were underexposed.
Foremost is that ordering by ID results in approximate chronological order of files' initial releases. Approximate because this is not exactly by date, but when compiling Body of Work lists it does certainly help. And when a file is updated keeping the same name in the archive or with a version increment in its name, then its unchanged ID helps to keep it in order of original release – at the time of generating or verifying a works-list – while its updated timestamp does not. Conversely, a version increment in a name requires the file= parameter to be updated every time (as I do for every DMMPST release), which is less convenient than the static ID. These advantages are the reason I added the ID lookup in my listigf script rather than directly used the file paths.
But I can imagine that consensus is reached on the cons outweighing the pros. So if IDs are to be replaced then it would be helpful to keep some ordering ability with the alternative – although I have no idea how.
Anyway, id= parameters are (and were historically) used far more frequently than file=, and updates are thus needed on the majority of both idgames and ig template transclusions. To a script that processes both lists, if only to verify that some don't need updating, the number of entries is then the same. So I would suggest to use as template name the one that is the most general and clear. In that sense, I think idgames is a better choice than the somewhat cryptic ig (although there are the {{lc}} and {{wp}} precedents).
Ryan, what do you mean with "a /newstuff parameter"? --Xymph (talk) 11:15, 23 December 2019 (CST)
Ouch, please disregard /newstuff for now.  Structured data it is not.  :>
For chronological order, query the date field perhaps?  If it still works, it should be far more accurate, especially for old maps originally hosted elsewhere.
Short template names arose, at least in part, to reduce keystrokes during mass edits by humans.  Whether you agree or disagree with that goal, be aware it may be discussed again.  Ryan W (living fossil) 14:50, 28 December 2019 (CST)
For the ID parameter, I'd be tempted to keep it. I believe the template already prefers the file parameter if you give both, so the id parameter can remain as a historical note. Furthermore, ironically, the "modern" Doomworld interface uses it. For example, the Plutonia 2 path for the modern interface is <https://www.doomworld.com/files/file/15550-plutonia-2/> and, not coincidentally, that same number 15550 is its id in the old interface <https://www.doomworld.com/idgames/?id=15550>. Finally, there's the whole "/idgames protocol" thing, which again is based on numbers, and Plutonia 2's URL with that protocol is unsurprinsingly <idgames://15550>.
While we should definitely strive to put the real path in these templates, the numbers can be handy if we decide to update the output of the templates. For example, if we want to generate URLs for the "modern" Doomworld interface (perhaps as a consequence of the removal of the "legacy" interface). --Gez (talk) 06:50, 24 December 2019 (CST)
I tried it and file= indeed takes precedence if both parameters are present (in both templates), so that works for me too. Combining them in scripted updates should be pretty trivial.
Regarding "different default link text" which "could be made conditional on the value of the linkonly parameter", I don't think it's that easy. The two templates produce the following output on these parameter combos:
idgames w/o linkonly idgames linkonly=1 ig
title title at DW/IG title title
no title PAGENAME at DW/IG PAGENAME DW/IG
I'm not sure idgames linkonly=1 is ever used without a title parameter though. If not then emitting DW/IG in place of PAGENAME works, or if a few times then we could resolve those manually. If the PAGENAME output of idgames linkonly=1 w/o title parameter should remain available, then to incorporate the "no title" effect of ig, an archname=1 parameter could be used. Not very elegant, but anyway... food for thought? --Xymph (talk) 04:52, 28 December 2019 (CST)
A first incarnation of the idgamesLinks.php script written yesterday (on 1356 links in 884 pages) confirmed that idgames linkonly=1 is only used with a title (parameter). So the above table can be condensed to:
idgames w/o linkonly idgames linkonly=1
title title at DW/IG title
no title PAGENAME at DW/IG DW/IG
to combine both templates per Gez's original proposal. --Xymph (talk) 02:37, 30 December 2019 (CST)
I can revise conditional logic as needed, so please don't treat it as a design constraint.  ;>  I'm proceeding gradually because I want full backward compatibility, and for the script's actions to be deterministic.  It will take time to work through all cases, so I created this thread first, to ensure I didn't overlook essential conceptual points.
Regarding ID numbers: When the domain moves, ideally it should take at most one edit to unbreak all links.  That isn't possible with IDs alone if the legacy front end is dropped, as in 2017, so I agree with Gez about populating paths.  "Beta" URLs must be optional for the same reason: we can't expect to know their final format until they go live, and unless Ling shares his algorithm, I would be nervous about generating them automatically as Gez suggests.
I'm still pondering whether link format should have one parameter or several.  Multiple parameters would avoid conflations of logic like this, but might seem cumbersome to editors who use templates rarely.
FWIW, and thanks for spending some time with this.  Ryan W (living fossil) 14:50, 28 December 2019 (CST)
"That isn't possible with IDs alone if the legacy front end is dropped, as in 2017" – I don't get that concern. Consider what happens if you visit <https://www.doomworld.com/files/file/15550-nuts/>, <https://www.doomworld.com/files/file/15550-my-neighbor-totoro/>, or <https://www.doomworld.com/files/file/15550--/>. As far as I can tell, all that's needed to make a valid URL is a number, a separator dash, and then any other character. The server looks at the number part and redirects you to the "proper" address. --Gez (talk) 05:49, 29 December 2019 (CST)
That's true today, but immediately following the upgrade it was not.  Only after massive backlash were the alternate links enabled.  The webmaster is on record elsewhere as not prioritizing history preservation (example), so IMO we should plan for the next relaunch moving irreversibly to new URLs.  Ryan W (living fossil) 06:50, 29 December 2019 (CST)  Obviously I'd love to be wrong, and if you (or another with community standing) is feeling generous, you could end speculation by inquiring with the root users directly.  For most of us that's not an option and we have to treat browser output as a black box.  HTH.
What was broken was the old paths, but not the id. Knowing the id of a file and having only the modern interface lets you access it reliably, as I just demonstrated. The name doesn't matter for the new interface, only the id. For the id to no longer be valid, it'd take a nuking of the Doomworld database, in which case then yeah, they would become useless, but then the idgames would have to point directly to one of the mirrors instead of any Doomworld interface. --Gez (talk) 10:33, 29 December 2019 (CST)
Definitely agree on the last part.  Time permitting (which is for Xymph to decide of course), and after normalizing on one template, I suppose the bot could do something like:
  • If id= is populated and file= is not, retrieve the file path from the API and add it.  Then put the path back into the API to get the ID, and if it doesn't match, or if either call failed, set a parameter in the template to invoke a maintenance category for human review.
  • Same thing if file= is populated and id= is not.
  • If both are populated, put each into the API, and if both calls succeed and the returned values match, do nothing.  Otherwise, invoke the maintenance category (because a human must compare the package contents to the article's description).
  • Stipulating that the IDs have now been validated, follow redirects of the form https://www.doomworld.com/files/file/12345--/ to retrieve the beta URL for each reviews page, and populate that in the template call also (another new parameter).
  • Repeat all steps once a year in case of database hiccups, per the 2016 thread.
Regarding the future, again I hope you're right about everything, as you usually are.  The DSDA migration changed its backend and stopped assigning IDs to new mods.  Just saying.  Ryan W (living fossil) 17:23, 29 December 2019 (CST)
That should be feasible, except the part "to retrieve the beta URL for each reviews page, and populate that in the template call also". How do you mean this? --Xymph (talk) 02:37, 30 December 2019 (CST)

←←←←  At the risk of being pedantic, here's a real example:

* {{idgames|file=levels/doom2/a-c/cct}}

From the API, the ID comes back as

11694

(the documentation doesn't mention that ".zip" must be explicitly appended).  Per Gez above, it deterministically becomes

https://www.doomworld.com/files/file/11694--/

The bot pastes this into an imaginary browser location bar (it's possible in bash so I tend to assume a library exists :>  and observes it turn into

https://www.doomworld.com/files/file/11694-congestion-control/

therefore the current link is replaced by something like

* {{idgames3|file=levels/doom2/a-c/cct|id=11694|xaserurl=https://www.doomworld.com/files/file/11694-congestion-control/}}

which would possibly render as

Please don't feel you need to implement anything until a proposal is posted and undergoes a comment period.  For all I know, people consider the new URLs too tentative or redundant or navelgazing to include.  I am undoubtedly forgetting how to handle the first link intelligently (i.e. not sending all traffic to the same mirror).  Also the format in "body of work" lists is different and the regular contributors might want it to remain different after this update — apologies for totally overlooking that situation in the OP.  FWIW everyone.  Ryan W (living fossil) 05:30, 30 December 2019 (CST)

No worries, wasn't planning on steamrolling ahead until a good plan has been hashed out. :)
Re. your example, several observations/questions:
  1. I don't think a direct download link to any particular mirror (especially not one that died earlier this year) should be used. Even though the list of mirrors on the old interface site is outdated, at least it provides multiple working options.
  2. The old interface site shows the file path (here levels/doom2/a-c/cct.zip) as a string that can be directly copy/pasted on a file system level. Not sure how valuable that is to anyone else, but this field is invaluable to me because on my own Linux system I run a backup of gamers.org and thus an idgames mirror. I frequently use the direct path in commands like zipinfo/unzip and cat of the .txt file. On the new beta interface this Filename field is absent.
  3. The file= path should not have .zip appended, because that redirects to the legacy server with expired cert.
  4. What's Xaser's involvement with the new beta interface?
  5. Given that a properly constructed URL with ID already gets redirected, wouldn't it be easier to not require a parameter (like xaserurl) but just have the template produce the second link automagically? At least when manually using the template to make it less elaborate for the casual wiki contributor.
E.g. * {{idgames4|file=levels/doom2/a-c/cct|id=11694}}
which would possibly render as
For scripted updates to existing template invocations, the processed URL could indeed be included to show the official URL when hovering over, and save a redirect on the DW server. --Xymph (talk) 16:27, 30 December 2019 (CST)
(2) Wiki contributors have always been asked to template any external link that can be templated, so yes, that's just you.  :>  If those paths disappear during the domain move and we can't find an easy workaround, you will have to discuss it with the DW staff.
(4) (5) The parameter name was mostly a joke; I endorse the "automagic" approach for improved usability.  At the time, I interpreted statements such as this to mean Xaser had some authority over the URL changes.
(1) (3) These also sound like improvements.  I guess if I assume without direct evidence that arbitrary changes could occur during the domain move, it's equally valid to assume there is no timetable to start the move, thus we can fairly design our templates around the current setup for now.  (I wasn't suggesting ".zip" could be included in template calls idgames transclusions, as indeed it was never designed to be, only in API queries.)
HTH / KUTGW.  Ryan W (living fossil) 12:20, 18 January 2020 (CST)
P.S.  the processed URL could indeed be included to show the official URL when hovering over  — This seems of limited use to the reader if they can't follow the link without editing the page (but maybe with a modern mobile browser such things are easier).  It's also technical debt if the URLs again change non-deterministically.  But I don't feel strongly; if you really want hover text, I'll implement it.  Ryan W (living fossil) 12:30, 18 January 2020 (CST)

Please see merge logic (first draft).  It isn't finished: it only covers Quasar's initial request, without any of the refinements above.  I continue to assume that, before the bot does anything, I'll have to revise Template:Idgames to support one or more new options without corrupting any existing links.  Still thinking about how to do so.  Thanks, Ryan W (living fossil) 19:41, 22 January 2020 (CST)

The more I think about it, the less I like the idea of merging ig and idgames. Looking at some of the examples on that first draft, worry for the size of a link-template-intensive page like list of notable WADs. --Gez (talk) 10:51, 23 January 2020 (CST)
ig has only 16 uses there; the other links are generated by Template:Notable and thus need no markup updates.  If such "alternate" links are intended to occupy a standard position, you could even replace most/all with optional parameters to Template:Notable (thereby changing the page's nesting depth, which has a greater effect than byte count alone).
Regarding more typical cases like this, in a vacuum I agree that code shouldn't get longer and longer over time.  But we aren't in a vacuum because each scripted task has a specific goal: forcing every possible upstream data set into the same uniform rendered format.  My reading of Wikipedia's template history suggests that the inverse 80/20 rule is occurring: the more real events we document, the more the logic must expand to accept the associated diversity of input.  Ryan W (living fossil) 18:43, 23 January 2020 (CST)
Hi. I've only been peripherally aware of this and that's a lot of reading above so apologies if this has already been brought up. I'm in favour of keeping file IDs since as Xymph noted it's useful for approximating a chronological order of WAD releases on mapper articles and the like, however... Doomworld's beta interface was introduced in March 2017, files uploaded before then have the same ID on the beta interface as the legacy one, but that is not the case for files uploaded since. For example: MAYhem Purple, uploaded in November 2018 has ID 19233 on legacy, but 19249 on the beta. --Eris Falling (talk) 15:49, 23 January 2020 (CST)

Bypassing front end?

The functionality of the "id Software FTP" templates is not duplicated by the main templates, because the former hotlink to a file at a specific mirror, rather than a landing page with a selection of mirrors.  A live example is here.

Discussion so far (including previous threads) suggests to me that the landing page is superior, despite the extra click:

  • Any individual mirror can experience downtime, or may be too distant (via network topology) from the reader.
  • The archived reviews provide additional insight about the file which would be too "chatty" for our article.
  • For backward compatibility, the main template would have to begin supporting the hotlinking and the choice of mirror.  The link counts above imply this would be considerable effort for a rarely used feature.  After implementation, it would be tedious to keep all transclusions updated each time a server moved or died or lost its certificate or whatever.
  • The FTP protocol itself is deprecated [7].  Even if some mirrors retained it, hotlinks would eventually break for readers with newer browsers, and might harm our search rank by being non-secure.

I've never used these templates, so I don't want to unilaterally dump them.  Does anyone definitely agree or disagree?    Ryan W (living fossil) 23:47, 18 January 2020 (CST)

IMO they can be safely dumped. As noted, they originally served to link to an archive that no longer exists. There's no reason anymore to preserve this link to the past, they can use the same {{ig}} template as the others. --Gez (talk) 09:03, 20 January 2020 (CST)