DEHEXTRA discussion (split from Pr+ UMAPINFO thread)

[Altazimuth]

So apparently some stuff happened with extra states and mobjs and stuff? Can somebody fill me in on the technical details here (number of new states/mobjs, dehackednumes, etc)? What do I need to set aside/define for this stuff? I saw https://github.com/coelckers/prboom-plus/commit/923bb1b4e8d821f26a9b3a16e4427090dca6830b but I also notice stuff like the S_PLAY_GDIE1 frames which certainly don't exist in EE and I don't even know what they might be.

 

2021 Update: Example WADs provided below:

• Black Ops
• SMOOTHED
• Pact of the Damned

 

Edited June 7, 2021 by Altazimuth

[fabian]

Well, the initial implelentation of this can be found in Doom Retro and it was added there to be able to load both of VGA's "Smoothed" and "Black Ops" mods at the same time, IIRC. I adapted the changes to Crispy Doom a few weeks later.

 

The commit that you refer to is really all that is needed to get this implemented in any other port, I guess. The change is pretty self-explanatory: You add a bunch of extra states starting at 1089 up to 4000. The states before are not affected by this. For example, the S_PLAY_GDIE* states in PrBoom+ were used for the ominious "skull pop player death" easter egg that we already had entirely disabled in the UMAPINFO fork before. Similar for the mobj types, you simply add 100 dummies starting at 150.

 

The values are all taken from how it's done in Doom Retro, since this is pretty much the reference implementation of all this. This is also the cause for the only stumbling block that I remember: There is a mapping between sprite name strings and enums, and since Doom Retro had already introduced another "SPR_BLD2" sprite before, all other implementations will have to carry this as well, albeit unused.

[Ferk]

10 hours ago, Altazimuth said:

I also notice stuff like the S_PLAY_GDIE1 frames which certainly don't exist in EE and I don't even know what they might be.

 

This is something already existing in prboom-plus for years, added by entryway.

I believe it's for an alternate player death where the head of the player would fly off and be detached from the body.

See this thread:

 

[Redneckerz]

5 hours ago, fabian said:

Well, the initial implelentation of this can be found in Doom Retro and it was added there to be able to load both of VGA's "Smoothed" and "Black Ops" mods at the same time, IIRC. I adapted the changes to Crispy Doom a few weeks later.

 

The commit that you refer to is really all that is needed to get this implemented in any other port, I guess. The change is pretty self-explanatory: You add a bunch of extra states starting at 1089 up to 4000. The states before are not affected by this. For example, the S_PLAY_GDIE* states in PrBoom+ were used for the ominious "skull pop player death" easter egg that we already had entirely disabled in the UMAPINFO fork before. Similar for the mobj types, you simply add 100 dummies starting at 150.

 

The values are all taken from how it's done in Doom Retro, since this is pretty much the reference implementation of all this. This is also the cause for the only stumbling block that I remember: There is a mapping between sprite name strings and enums, and since Doom Retro had already introduced another "SPR_BLD2" sprite before, all other implementations will have to carry this as well, albeit unused.

One thing to note is to arrive on a specific namesake for these extra states offered by both Retro and Crispy. Its obviously not just plain DeHacked, Nor Boom, nor DEHSUPP.

 

Gez came up with EXTRADEH whereas myself suggested DEHADDIT (DeHacked Additional). DEHTOOMS was also suggested but Brad Harding suggested VGA do the honours.

 

So @VGA how to call these? :P

 

If this warrants a seperate thread since i have posted about this in a unrelated port, please tell me. I'd love to have a definition for these additional states since it provides for easier explanation and definition, and its a rather interesting feature to market both ports with.

Edited July 10, 2020 by Redneckerz

[Shadow Hog]

So to be clear, if I wanted to add extra sounds to this new definition so we could make the most of the new mobjs/states (or just wanted to back port that one ZDoom mod that gave all the vanilla monsters unique hurt sounds IDK I'm not a cop), I'd need to make changes to/pull requests for Doom Retro, Crispy Doom, PrBoom+/um and WhackEd4 at the very least, correct? Not that I'm sure how many. If we have 250 mobj definitions then it'd take 1250 sounds to ensure each and every one has a unique sound for each of the five possible soundtypes (see, active, attack, pain, and death), but I'd probably cap it at 1000 new sounds (DSFRE000-DSFRE999) at most. (I mean we could get cute and use hex instead, giving us DSFRE000-DSFREFFF for a total of 4096 new sounds, but that's way overkill, even if "frefff" is fun to say.)

[esselfortium]

Adding extra slots for sounds to go with the added sprites, states, and mobjs would definitely be wise. 1250 sounds feels like overkill to me, though -- how many different sounds are there in the base game?

 

Another thing I'd suggest adding is a new codepointer for generic projectile attacks. MBF's codepointers cover a lot of bases, but this is one that sticks out to me as an obvious omission that'd be really useful to have. Misc1 to define mobj type by dehackednum, Misc2 to optionally set a horizontal angle offset for doing custom spread attacks. Should automatically assign a tracer so that you can create additional homing missiles if desired, since by default only A_SkelMissile can do this.

 

There are definitely other features that could be incorporated and might be worth considering*, but I think keeping it as simple as possible would be best to help encourage ports to adopt it. I think just that one codepointer would already open a lot of doors for crossport-compatible modding, combined with the other additions that are already in Pr+ now.

 

*An additional text-based field for additional flags would be nice to allow customization of some hardcoded-to-specific-mobj behaviors like splash damage immunity, but I'm guessing that that's a bigger ask than what's already been proposed.

Edited July 10, 2020 by esselfortium

[Ferk]

Doom Retro refers to the new objects as MT_EXTRA, it might make sense to keep the "extra" namesake like the EXTRADEH Gez sugested, it's clearer.

 

And while adding that, it might also make sense to carry the "Dropped Item" Dehacked field Doom Retro also supports, so that it's possible for new mobjs to have drops.

This in combination with UMAPINFO already would make possible things like the old WolfenDoom to run (for Hans Grösse to drop a key and Mecha-Hitler to drop Hitler).

Edited July 10, 2020 by Ferk

[esselfortium]

11 minutes ago, Ferk said:

Doom Retro refers to the new objects as MT_EXTRA, it might make sense to keep the "extra" namesake like the EXTRADEH Gez sugested, it's clearer.

 

And while adding that, it might also make sense to carry the "Dropped Item" Dehacked field Doom Retro also supports, so that it's possible for new mobjs to have drops.

This in combination with UMAPINFO already would make possible things like the old WolfenDoom to run (for Hans Grösse to drop a key and Mecha-Hitler to drop Hitler).

Oh, that's interesting! I wasn't aware of that feature. I'm still wary of the potential for overcomplication, but some additional fields could be nice if the Pr+/EE/GZ devs are on board with it... I know GZDoom and Eternity Engine already support custom item drops via their own content definition languages, so I'd imagine that also allowing it to be defined via EXTRADEH should be feasible.

[Altazimuth]

@fabian OK thanks. Just to be 100% clear. 100 sprnames (w/ an extra for a blood splat), 100 mobjs w/ dehnum 150-249 (using a basic set of properties from the EMPTYMOBJ in Doom Retro), 2911 states w/ dehnum 1089-3999 and all those states' next state is themself and everything else is basically null (if we consider a duration of -1 to be null too at least)? Also I need to implement the 9 extra states dedicated to the easter egg?

 

EXTRADEH seems good. Let's not get too wrapped up in this topic though, lest we end up w/ the same awful bikeshedding that happened when discussing this project's name.

 

As for essel's suggestion I think that a simple generic projectile firing codepointer would be a good idea. Need to find an obvious-enough name that isn't used by any ports yet, but it's a solid suggestion.

 

More sounds is also a good idea, though if possible I'd like to have the extra sounds have nums that don't collide with anything EE-related already because rearranging existing sounds dehnums to all be 1000 or so higher sounds like a nightmare.

 

It's worth noting that Eternity already has a bunch of extensions to deh afaik that could be useful here. I'll need to reread https://www.doomworld.com/eternity/engine/dehref.html. They all seem to be labelled "Eternity Extension" at least though I'll check more.

Edited July 10, 2020 by Altazimuth

[Ferk]

20 minutes ago, esselfortium said:

Oh, that's interesting! I wasn't aware of that feature. I'm still wary of the potential for overcomplication, but some additional fields could be nice if the Pr+/EE/GZ devs are on board with it... I know GZDoom and Eternity Engine already support custom item drops via their own content definition languages, so I'd imagine that also allowing it to be defined via EXTRADEH should be feasible.

Yes, it's not very well documented I think:https://github.com/bradharding/doomretro/wiki/INFORMATION-FOR-MAPPERS#DEHACKED-LUMPS

 

But adding support for dropped items in the way Doom Retro does it (which is just using the Thing id of the thing to drop on death) would be relatively simple and would not mess with the Bits, compatibility with WADs not supporting the feature should be fine, I had already added that to the PrBoom libretro fork in the past (along with some other changes).

 

I don't know if devs would consider it worth of standard (I think the way it's done in EE and GZDoom is much more complex and featureful), but I can make a PR for PrBoom+ this weekend for "Dropped Item" to initiate discussion on that.

Edited July 10, 2020 by Ferk

[Redneckerz]

29 minutes ago, Ferk said:

Doom Retro refers to the new objects as MT_EXTRA, it might make sense to keep the "extra" namesake like the EXTRADEH Gez sugested, it's clearer.

 

And while adding that, it might also make sense to carry the "Dropped Item" Dehacked field Doom Retro also supports, so that it's possible for new mobjs to have drops.

This in combination with UMAPINFO already would make possible things like the old WolfenDoom to run (for Hans Grösse to drop a key and Mecha-Hitler to drop Hitler).

This exactly.

 

I found the page you linked below earlier aswell, but it just states (heh) ''DeHackEd support has been extended to allow for an additional 2,910 states (numbered 1,089 to 3,999), 100 additional things (numbered 150 to 249) and 100 additional sprites (numbered 145 to 244). Examples of this extended support being put to great use include the SMOOTHED and Black Ops mods by VGA.'' 

 

That's why i'd like a new name for these states, so that this can be documented properly. It would also be easier to market as a feature by Crispy and Retro, and who knows what else can be modified to support these things.

 

I know atleast one project that may consider this kind of thing useful for one. DEHEXT or EXTRADEH or DEHXTD or whatever could be a viable and powerful alternative if you want more elaborate DeHacked trickery (Linguica and his DeHacked magic could have a field day with these) but do not want to go for DECORATE (Because ZDoom) or ZScript (Because ZDoom)

 

@AltazimuthI have not checked yet, but the EE DeHackEd extensions are also useful to document, if need be.

And that reminds me, something like DEH_EDGE should be too.

[Altazimuth]

23 minutes ago, Redneckerz said:

@AltazimuthI have not checked yet, but the EE DeHackEd extensions are also useful to document, if need be.

Yeah OK so most of the 4 are worthless. There's the [HELPER] block which is only useful if you use -dogs so it isn't useful at all.

The other three are more interesting, as they appear to have been planned by Ty Halderman.

• [SPRITES]: You can change the 4-letter prefix of any sprite type. Seems possibly handy.
• [SOUNDS]: The SOUNDS block can change the name of any sound. Seems weird and kinda niche but maybe useful? I dunno the EE documentation mentions EDF sound mnemonics specifically, but it's worth noting the DOOM EDF sound names are all the exact same as their BEX counterparts.
• [MUSIC]: The MUSIC block can change the name of any music lump used by the game. This would have been useful in the early 2000s... but it's 2020 and UMAPINFO makes this pointless.

 

EDIT: Side-note: Would it be a good idea to perhaps split this DEH extension discussion into its own topic? I think it might be worthwhile, but dunno what everybody else thinks about the matter. It's related to this project but it's also related to a bunch of other ports.

Edited July 10, 2020 by Altazimuth
More thoughts.

[esselfortium]

37 minutes ago, Altazimuth said:

EDIT: Side-note: Would it be a good idea to perhaps split this DEH extension discussion into its own topic? I think it might be worthwhile, but dunno what everybody else thinks about the matter. It's related to this project but it's also related to a bunch of other ports.

That would probably be wise!

 

edit: Done :)

Edited July 10, 2020 by esselfortium

[Shadow Hog]

1 hour ago, esselfortium said:

Adding extra slots for sounds to go with the added sprites, states, and mobjs would definitely be wise. 1250 sounds feels like overkill to me, though -- how many different sounds are there in the base game?

Including the dog sounds, DSSECRET and whatever "sfx_gibdth" is, but excluding "sfx_none", 115. But again, not 1250, I was thinking 1000 as an absolute upper limit (so you could fully define 5 unique sounds for almost every object,  but would otherwise have to share some or omit them, as with decoration objects) and something like 128-512 as a more practical amount.

 

I also remembered that some sounds could he randomized in vanilla, but those instances were hardcoded. However, requesting anything genuine about that is waaaaay out of scope of this conversation, so I'mm'l let that that sleeping dog lie for now.

 

And yeah, we can continue this elsewhere.

Edited July 10, 2020 by Shadow Hog

[esselfortium]

1 minute ago, Shadow Hog said:

I also remembered that some sounds could he randomized in vanilla, but those instances were hardcoded. However, requesting anything genuine about that is waaaaay out of scope of this conversation, so I'mm'l let that that sleeping dog lie for now.

I suppose you can hack it with the existing RandomJump and PlaySound codepointers, at least, even if it'd be ugly.

[Ferk]

13 minutes ago, Shadow Hog said:

something like 128-512 as a more practical amount

Yes, maybe don't go too overboard.

The reason I didn't backport the extra states to PrBoom libretro just yet was because (other than not many WADs using them) I wasn't sure if it would bump the memory usage too much with just empty data structs, considering that port is meant to run in very constraint platforms like 3DS homebrew and even watches.

But I'm sure it makes very little difference.

Edited July 10, 2020 by Ferk

[esselfortium]

Alright, I've split this into its own thread, as suggested :)

[Altazimuth]

These new EDF thingtypes brought to you by: Microsoft Excel.

Microsoft Excel: Because I don't trust myself to not fuck up copying and pasting 100 lines while modifying numbers.

Spoiler

Anyway much of the heavy lifting is done w/ EE. I need to do the same thing for states. I'm still unclear what they point to. They seem to point to themselves by default as the next state, looking at the code, but I feel like it'd be more sensible if they pointed to the state after? I need somebody to clarify here.

Aside from that it's cleaning up any dehnum clashes, which no mods should have used since EDF came into existence fairly quickly.

[Da Werecat]

Just call it DEXTER. Dexter likes to hack!

[Gez]

48 minutes ago, Altazimuth said:

They seem to point to themselves by default as the next state, looking at the code, but I feel like it'd be more sensible if they pointed to the state after? I need somebody to clarify here.

I think the idea was that since they're only there as fodder for modders, their default values are not really important.

[esselfortium]

I can see where having states point to each other could be useful to save time in some situations if you're editing the states table directly, but I don't think it's necessary.

 

Rather than worrying too much about hand-editing the states table, which is going to still be a pain in the ass no matter what the defaults are, I think the ideal goal for this would be to have fraggle's DEH9000 python library expanded or forked to support BEX/MBF/EXTRADEH patches. The end result there would effectively be a crossport Decorate-lite, allowing modders to use a nice easily-readable text format that "compiles" to EXTRADEH like it already does for vanilla DEH. Would provide many of the benefits of a new cross-port-compatible content definition system without requiring ports to actually implement yet another new content definition system on top of the ones they have already. :)

[Ferk]

4 minutes ago, esselfortium said:

to have fraggle's DEH9000 python library expanded or forked to support BEX/MBF/EXTRADEH patches

This was also mentioned in an issue for deh9000: https://github.com/fragglet/deh9000/issues/5

[jval]

Instead of adding a constant number of new states, objects etc why this couldn't be dynamic?

And eg instead of 'FRAME' or 'THING' keyword the parser could recognize 'NEW FRAME' or 'NEW THING' as an extra new frame / thing.

[esselfortium]

11 minutes ago, jval said:

Instead of adding a constant number of new states, objects etc why this couldn't be dynamic?

And eg instead of 'FRAME' or 'THING' keyword the parser could recognize 'NEW FRAME' or 'NEW THING' as an extra new frame / thing.

They'd still need numbers so that they can reference each other, and advanced ports need to be able to fit them around their own existing expansions. Not saying that your suggestion is impossible, but I'm assuming that static additions like these are probably the path of least resistance for easy implementation across lots of ports.

[Ferk]

I guess in theory it could be possible to make it dynamic implementation-side by reallocating them whenever the Thing/Frame id given is higher than the current number of things, without introducing "NEW".

 

But that would be more troublesome for the ports. I think the main reason to not do it dynamic like that is to avoid complexity.

Edited July 10, 2020 by Ferk

[xttl]

Heh, if this stays simple enough, it might even be possible to implement support for it in vanilla by exe hacking. (possibly requiring having to run an external program which converts from text to binary format first, but hey, you need to run DHE to apply DEH files to vanilla anyway)

[plums]

I hope once this format gets finalized it can also be applied to Heretic. HHE was never fully developed and support for it in modern ports is basically limited to Chocolate/Crispy Heretic; having a DEH-like tool that works in Crispy Heretic, Eternity, and GZDoom would be fantastic.

[Shadow Hog]

You'd need to change a lot of pointers, whether to expand the size of the tables in-place or to move them outside their normal location to, say, the end of the executable's filesize, but I suppose it's doable, yes. It certainly would be awesome to be able to use a gameplay mod that uses all these extra states on a Pentium II running Windows 98SE in DOS Mode (or hell, a 486 running DOS 6.22), for sure, if more a huge novelty than anything practical.

 

Re: number of sounds, doing some math, I

think 140 is a good number - bumps the overall number of sounds to 255 (+ sfx_None), and should be plenty for most use cases (famous last words, a la the apocryphal "640K should be enough for anybody", but whatever, it's a start). Maybe I'll play with later...

[Altazimuth]

Another thing. S_PLAY_GDIE9 is 1084 but EXTRASTATES is 1089. What goes on from states 1085 to 1088?

[fraggle]

40 minutes ago, esselfortium said:

I can see where having states point to each other could be useful to save time in some situations if you're editing the states table directly, but I don't think it's necessary.

 

Rather than worrying too much about hand-editing the states table, which is going to still be a pain in the ass no matter what the defaults are, I think the ideal goal for this would be to have fraggle's DEH9000 python library expanded or forked to support BEX/MBF/EXTRADEH patches.

Before we start considering forks, I'll just say that I'd be open to including this in DEH9000.

 

I'd like to see DEH9000's state parser expanded to something like a DECORATE-lite so that people can write vanilla-compatible patches (Or Boom/MBF-compatible, etc.) in a nicer format while being able to take advantage of DEH9000's features like the state auto-reclaim. It's potentially tricky in some ways though and parts of it need to be carefully considered.