Doomkid Posted November 5, 2014 So I've looked all around and can't find an answer to this one. I've heard that if you replace every Doom2 sprite, that you can get custom sprites to run in vanilla Doom, so long as all sprite entries and rotations match up. I just did that, and can't get the damn thing to run! (About 80% of sprites were replaced, but I've ensured the number, order, and rotations all match up. I can provide a link if needed.) Why the heck won't this thing run in vanilla doom? I keep getting "Bad lump characters in lump 4110". I can't identify lumps, but I dobule and triple checked - Every sprite rotation is there. It gets a bit weirder, though. There's 1,381 sprite frames in Doom2. However, DWANGO11.wad uses custom sprites, only replacing 302 of them, greatly reducing filesize. What's the story here? Why does it work? I just tested it in vanilla Doom, custom sprites weren't an issue even though they're FAR from fully replaced. What gives? I copied all the sprites from Dwango11 into an empty wad, so it was only the sprites and the S_START and S_END markers. No luck - crashed, simply cut to black, to error message or anything. Can anyone please shed light on this? 0 Quote Share this post Link to post
Obsidian Posted November 5, 2014 I'm pretty sure you need to rename sprites using DeHackEd if you want to use them in vanilla. Download WhackEd4 and rename the sprites in the Strings table, rename the sprites in SLADE accordingly and you should be fine. If you need an example then look at Strain's DeHackEd patch. 0 Quote Share this post Link to post
Gez Posted November 5, 2014 Doomkid said:Why the heck won't this thing run in vanilla doom? I keep getting "Bad lump characters in lump 4110". I can't identify lumps, but I dobule and triple checked - Every sprite rotation is there. The message you got is "bad frame character". if (frame >= 29 || rotation > 8) I_Error("R_InstallSpriteLump: " "Bad frame characters in lump %i", lump); It means it found something in the sprite namespace that didn't fit the requirements to look like a sprite. The 29 accepted frames are A-Z, [, \, ], and the accepted rotation values are 0-8. So if it finds something like a flat named CEIL_A, the "frame" character will '_' and the rotation will be 'A', both of which will fail. I'm not sure how the dehacked trick works under the hood. From what I know, it looks for sprites between S_END and S_START (Doom engine always searches backward from the last) so either you have an S_END marker and it'll start looking there, eventually getting in the Doom IWAD's flats if you don't have a corresponding S_START first; or you don't have markers and your sprites should be ignored. Yet somehow it works and I don't know why. 0 Quote Share this post Link to post
Doomkid Posted November 5, 2014 Thankyou both for the responses! See, this is what's weird - DWANGO11 doesn't have a .DEH patch - It just has a select few custom sprites and runs in vanilla with no hassle. Supposedly, this is impossible. I've copied the exact sprites from the wad and placed them in another, and it didn't work Take a look, it's strange: http://soulsphere.org/random/dwango-wads/dwango11.zip (select bfg/plasma) Only 302 of the ~1,300 sprites are replaced, and it runs under DOS... What the hell did they do to get this to work, without a dehacked patch? Figuring this out could be a big key to making future vanilla-compat graphics wads easier to deal with. I'm really curious to find out why! 0 Quote Share this post Link to post
purist Posted November 5, 2014 I had similar issues replacing sprites in vanilla plutonia. Have a look, it might help you: http://www.doomworld.com/vb/doom-editing/69236-replacing-sprites-in-vanilla-plutonia/ 0 Quote Share this post Link to post
scifista42 Posted November 5, 2014 The theory is: In vanilla, you either replace exactly ALL sprites, or none. In the latter case though, you can add new sprites with new names into the wad (their number or rotations don't matter at all) and use DEHACKED to change a "stock" sprite name to the new one, then it should work even in vanilla. You need to put the sprites between SS_START and S_END markers - yes, really SS_START and S_END, there shouldn't be S_START. However, I've encountered instances when it mysteriously didn't work. Sometimes it was due to the subsprites (=letters A,B,C... after the sprite name) not being in order. On the other hand, in other instances this worked, so I don't know. The DWANGO thing is also a mystery to me. 0 Quote Share this post Link to post
Da Werecat Posted November 5, 2014 Doomkid said:Only 302 of the ~1,300 sprites are replaced, and it runs under DOS... As far as I can see, all sprites that are needed in deathmatch are replaced. I guess it's okay to leave the monsters out, as long as the engine won't come looking for them. 1 Quote Share this post Link to post
Doomkid Posted November 5, 2014 Thanks for the further info, fellas. So, I've done as instructed, and tried to replace the pistol, chaingun and chainsaw sprites using the DEHACKED trick. (CHGG was changed to CHAI, and so on, through strings) In all source ports, it works as intended, but not in Doom2.exe. Can anyone explain why this is happening? I feel like Ive followed the instructions on every thread as close as possible - http://www.mediafire.com/download/a2agx3qdbppwxxb/ugh_poop.zip If someone can help me out here, I'd be incredibly thankful. I'm also hoping about the mystery of DWANGO11's custom sprites will be solved, it's strange to think there may be something about the vanilla EXE that isn;t yet documented. EDIT: Great find, Da Werecat! I want to replicate this, yet for some reason, doing the same thing causes my WADs to hang in DOS. Hell, even doing the full sprite replace trick causes it to hang. Is there something special aside from renaming S_START to SS_START that's needed? Boy, Doom2 vanilla sprite editing is even more finnicky than I remember.. 0 Quote Share this post Link to post
Da Werecat Posted November 5, 2014 Works in Choco. Did you forget to apply the patch to the vanilla Doom executable? 0 Quote Share this post Link to post
Doomkid Posted November 5, 2014 Is starting up doom2.exe with the "-file pooper.wad pooper.deh" command enough? If not, I've found my problem. Now what I'd need to do is work out how to patch DOOM2.exe for dehacked stuff, and I'd finally have it figured out. 0 Quote Share this post Link to post
scifista42 Posted November 5, 2014 Vanilla Doom2.exe doesn't support DEHACKED patches being loaded along with a wad. That's a feature of source ports. You need to make a copy of your "Doom2.exe" and use dehacked.exe to actually hack and rewrite it. That's the correct DOS-compatible way. 0 Quote Share this post Link to post
Doomkid Posted November 5, 2014 Thank you so much scifista, Da Werecat, purist, Obsidian and Gez. I now know how to replace vanilla sprites both through DEHACKED, and the "replace every sprite" way. (I tried replacing all sprites in XWE, but there was an oddity - It renamed all VILE\ sprites to VILE_! obviously Doom didn't recognize the "_" and that's why this method wasn't working for me.) Thanks a million. I'm now having problems getting DEHACKED itself to run, even after telling it where my Doom2.wad is through the INI. I feel like I've asked so many damn questions, though. Is using the old version the only way of making a patched doom2.exe? Can't be done through WhackEd, I assume? 0 Quote Share this post Link to post
scifista42 Posted November 5, 2014 Doomkid said:Can't be done through WhackEd, I assume? No, WhackEd can merely make and edit DEHACKED patches, not apply them to hack an exe. At least, I'm convinced it is so. Did you follow the enclosed README.txt of Dehacked? You should specify editname, normalname, wadname, pathname and patchdir in the .ini. Then you manually make the copy of Doom2.exe (calling it as you've specified in editname) and then simply launch "dehacked -load mypatch.deh". 0 Quote Share this post Link to post
Gez Posted November 5, 2014 SS_START means nothing to vanilla, though. Okay, so the frame character check is done in R_InstallSpriteLump(), which is called by R_InitSpriteDefs(). The way R_InitSpriteDefs() works is that it has a list of name, a first sprite lump, and a last sprite lump. It works through its list of name and searches through the lumps for lumps which match the sprite names from its list, so it overlooks those that don't fit. Interestingly, this function searches forward. Spoiler // // R_InitSpriteDefs // Pass a null terminated list of sprite names // (4 chars exactly) to be used. // Builds the sprite rotation matrixes to account // for horizontally flipped sprites. // Will report an error if the lumps are inconsistant. // Only called at startup. // // Sprite lump names are 4 characters for the actor, // a letter for the frame, and a number for the rotation. // A sprite that is flippable will have an additional // letter/number appended. // The rotation character can be 0 to signify no rotations. // void R_InitSpriteDefs (char** namelist) { char** check; int i; int l; int intname; int frame; int rotation; int start; int end; int patched; // count the number of sprite names check = namelist; while (*check != NULL) check++; numsprites = check-namelist; if (!numsprites) return; sprites = Z_Malloc(numsprites *sizeof(*sprites), PU_STATIC, NULL); start = firstspritelump-1; end = lastspritelump+1; // scan all the lump names for each of the names, // noting the highest frame letter. // Just compare 4 characters as ints for (i=0 ; i<numsprites ; i++) { spritename = namelist[ i ]; memset (sprtemp,-1, sizeof(sprtemp)); maxframe = -1; intname = *(int *)namelist[ i ]; // scan the lumps, // filling in the frames for whatever is found for (l=start+1 ; l<end ; l++) { if (*(int *)lumpinfo[l].name == intname) { frame = lumpinfo[l].name[4] - 'A'; rotation = lumpinfo[l].name[5] - '0'; if (modifiedgame) patched = W_GetNumForName (lumpinfo[l].name); else patched = l; R_InstallSpriteLump (patched, frame, rotation, false); if (lumpinfo[l].name[6]) { frame = lumpinfo[l].name[6] - 'A'; rotation = lumpinfo[l].name[7] - '0'; R_InstallSpriteLump (l, frame, rotation, true); } } } // check the frames that were found for completeness if (maxframe == -1) { sprites[ i ].numframes = 0; continue; } maxframe++; for (frame = 0 ; frame < maxframe ; frame++) { switch ((int)sprtemp[frame].rotate) { case -1: // no rotations were found for that frame at all I_Error ("R_InitSprites: No patches found " "for %s frame %c", namelist[ i ], frame+'A'); break; case 0: // only the first rotation is needed break; case 1: // must have all 8 frames for (rotation=0 ; rotation<8 ; rotation++) if (sprtemp[frame].lump[rotation] == -1) I_Error ("R_InitSprites: Sprite %s frame %c " "is missing rotations", namelist[ i ], frame+'A'); break; } } // allocate space for the frames present and copy sprtemp to it sprites[ i ].numframes = maxframe; sprites[ i ].spriteframes = Z_Malloc (maxframe * sizeof(spriteframe_t), PU_STATIC, NULL); memcpy (sprites[ i ].spriteframes, sprtemp, maxframe*sizeof(spriteframe_t)); } } Edit: Haha, the code accidentally contained valid BBCode and I had to add spaces around any array access on the i index. This function is responsible for making sure sprites have all their rotations. Note that it has a nested loop construction: it searches for all names, for all lumps. R_InitSpriteDefs() is called by R_InitSprites(), which is called by P_Init() -- you may have noticed the message "P_Init: Init Playloop state." in vanilla startup. At no point do these function care about S_START or S_END directly. However, R_InitSpriteDefs() use global variables: start = firstspritelump-1; end = lastspritelump+1; These variables are set by R_InitSpriteLumps() which is what scans through S_START and S_END.// // R_InitSpriteLumps // Finds the width and hoffset of all sprites in the wad, // so the sprite does not need to be cached completely // just for having the header info ready during rendering. // void R_InitSpriteLumps (void) { int i; patch_t *patch; firstspritelump = W_GetNumForName ("S_START") + 1; lastspritelump = W_GetNumForName ("S_END") - 1; numspritelumps = lastspritelump - firstspritelump + 1; spritewidth = Z_Malloc (numspritelumps*4, PU_STATIC, 0); spriteoffset = Z_Malloc (numspritelumps*4, PU_STATIC, 0); spritetopoffset = Z_Malloc (numspritelumps*4, PU_STATIC, 0); for (i=0 ; i< numspritelumps ; i++) { if (!(i&63)) printf ("."); patch = W_CacheLumpNum (firstspritelump+i, PU_CACHE); spritewidth[ i ] = SHORT(patch->width)<<FRACBITS; spriteoffset[ i ] = SHORT(patch->leftoffset)<<FRACBITS; spritetopoffset[ i ] = SHORT(patch->topoffset)<<FRACBITS; } }It is called by R_InitData(), which is called by R_Init(), which is the famous "R_Init: Init DOOM refresh daemon" line. R_Init() is called just before P_Init(). So the real question here is, "how does it get the namelist?". Well it's just a hardcoded global array:char *sprnames[NUMSPRITES] = { "TROO","SHTG","PUNG","PISG","PISF","SHTF","SHT2","CHGG","CHGF","MISG", "MISF","SAWG","PLSG","PLSF","BFGG","BFGF","BLUD","PUFF","BAL1","BAL2", "PLSS","PLSE","MISL","BFS1","BFE1","BFE2","TFOG","IFOG","PLAY","POSS", "SPOS","VILE","FIRE","FATB","FBXP","SKEL","MANF","FATT","CPOS","SARG", "HEAD","BAL7","BOSS","BOS2","SKUL","SPID","BSPI","APLS","APBX","CYBR", "PAIN","SSWV","KEEN","BBRN","BOSF","ARM1","ARM2","BAR1","BEXP","FCAN", "BON1","BON2","BKEY","RKEY","YKEY","BSKU","RSKU","YSKU","STIM","MEDI", "SOUL","PINV","PSTR","PINS","MEGA","SUIT","PMAP","PVIS","CLIP","AMMO", "ROCK","BROK","CELL","CELP","SHEL","SBOX","BPAK","BFUG","MGUN","CSAW", "LAUN","PLAS","SHOT","SGN2","COLU","SMT2","GOR1","POL2","POL5","POL4", "POL3","POL1","POL6","GOR2","GOR3","GOR4","GOR5","SMIT","COL1","COL2", "COL3","COL4","CAND","CBRA","COL6","TRE1","TRE2","ELEC","CEYE","FSKU", "COL5","TBLU","TGRN","TRED","SMBT","SMGT","SMRT","HDB1","HDB2","HDB3", "HDB4","HDB5","HDB6","POB1","POB2","BRS1","TLMP","TLP2" }; That's what you can modify in DEHACKED. From what I can guess, it's safe to change sprites if none of the non-sprite lumps in vanilla after S_END (included) and in your own mod have their first four characters in common with the sprites you are not replacing, which is why replacing all sprites is safe. In fact I've just made a small PWAD which only contains one lump: the S_END marker. Loading it in vanilla within DOSBox, it didn't cause any error during loading... Doomkid said:(I tried replacing all sprites in XWE, but there was an oddity - It renamed all VILE\ sprites to VILE_! obviously Doom didn't recognize the "_" and that's why this method wasn't working for me.) SLADE can rename lumps to VILE\ without problem. :) 0 Quote Share this post Link to post
Doomkid Posted November 6, 2014 Wow, very informative Gez. I always wondered what was the deal with the "DOOM Refresh daemon" line, it's interesting to know after all these years. So I have just one final question. Is there any other way I can tell vanilla doom2.exe to load a DEH file? What I mean is, so long as I have DEHACKED.exe, Doom2.exe, doom2.wad, something.DEH, and something.wad in the directory, is there and easy way for me to make a simple BAT textfile that loads it properly in dosbox? Or do I have to use the way built into dehacked.exe? It would be great if whacked was somehow able to create a patch for vanilla D2. I got dehacked booted up through dosbox, but everything kept getting corrupted and garbled on screen, as if it was having trouble dealing with my IWAD. It's just the commercial doom2 1.9 iwad though, not sure why it causes all the garbled text. The strings were just showing up as smiley faces and stuff, unless there's another way, I think my .deh making will be limited to choco-doom and up ;) All these old TC's like STRAIN come with .BAT files, but they're a bit beyond my understanding. It does work placing it in my doom2 directory and just loading the BAT though - This is what I want to achieve. 0 Quote Share this post Link to post
scifista42 Posted November 6, 2014 It's perfectly possible to create a DEHACKED patch using WhackEd, then put the patch in Dehacked's directory and just use "dehacked -load mypatch.deh" to generate the proper .exe in a brief moment, as I said. You won't even see Dehacked's interface at all. I do it this way, there's no special effort connected to it. 0 Quote Share this post Link to post
Doomkid Posted November 20, 2014 (Sorry for the slight bump) Alrighty scifista, I think you might be able to help me. I ran dehacked just as you said, "dehacked -load dehthing.deh" and it just said "Line 9:unkown line, Line 10: unknown line. Patch file read, one or more errors detected" and so on. I then tried to run DOOM2.exe, which had apparently been edited by this. It crashed with the message "fatal error 1313: can't resolve external references". Any ideas what I can do here? I made sure it was in ultimate doom DEH format and not BEX format when in WhackEd. 0 Quote Share this post Link to post
dg93 Posted November 20, 2014 scifista42 said:It's perfectly possible to create a DEHACKED patch using WhackEd, then put the patch in Dehacked's directory and just use "dehacked -load mypatch.deh" to generate the proper .exe in a brief moment, as I said. You won't even see Dehacked's interface at all. I do it this way, there's no special effort connected to it. This is a bit off topic, but are there any good tutorials on how to use WhackEd? I've tried using it before, but I had absolutely no clue on what the Hell I was doing. 0 Quote Share this post Link to post
scifista42 Posted November 20, 2014 the_miano said:This is a bit off topic, but are there any good tutorials on how to use WhackEd? I've tried using it before, but I had absolutely no clue on what the Hell I was doing. Enjay's tutorials (here) describe DOS Dehacked, but it should be sufficient for you, because the principle is absolutely the same. @Doomkid: Make sure that you've set up Dehacked's ini file correctly (according to instructions in README.txt), because Dehacked was supposed to edit a copy of your Doom2.exe (which you previously had to make yourself), and not the original file. If that's not the problem, post your dehacked file here. 0 Quote Share this post Link to post
dg93 Posted November 20, 2014 scifista42 said:Enjay's tutorials (here) describe DOS Dehacked, but it should be sufficient for you, because the principle is absolutely the same. Awesome, thanks man 0 Quote Share this post Link to post
GreyGhost Posted November 21, 2014 Doomkid said:I ran dehacked just as you said, "dehacked -load dehthing.deh" and it just said "Line 9:unkown line, Line 10: unknown line. Patch file read, one or more errors detected" and so on. I think the problem is that WhackEd4 now uses a verbose text format for the Bits field that DeHackEd doesn't understand, it expects to find a numeric value. I don't know if there's any way to force WhackEd to use numeric Bit values for backward compatibility, but exl might add that as an option if asked nicely. In the meantime your only option is to use an earlier version of the editor. 1 Quote Share this post Link to post
AD_79 Posted November 28, 2014 Ok, so I found this thread really helpful; I was having issues when I replaced the baron sprites, and the baron ended up being invisible for many of its frames in Chocolate Doom. Thanks to whoever said "replace all the sprites". 0 Quote Share this post Link to post
scifista42 Posted November 28, 2014 AD_79 said:Ok, so I found this thread really helpful; I was having issues when I replaced the baron sprites, and the baron ended up being invisible for many of its frames in Chocolate Doom. Thanks to whoever said "replace all the sprites". In Chocolate Doom, there's even better solution: use "-merge" parameter instead of "-file" when launching the wad, then you won't need to replace all sprites. Vanilla itself doesn't support this option, though. You can, however, tell the user to use a program DEUSF to add all non-replaced sprites to the wad on his end. But that's a 90's method. I strongly prefer using the DEHACKED trick nowadays. 0 Quote Share this post Link to post
fraggle Posted November 28, 2014 Doomkid said:Thankyou both for the responses! See, this is what's weird - DWANGO11 doesn't have a .DEH patch - It just has a select few custom sprites and runs in vanilla with no hassle. Supposedly, this is impossible. I've copied the exact sprites from the wad and placed them in another, and it didn't workI checked the WAD - it uses the "replace all sprites" method. It's just that as a deathmatch WAD it doesn't need to bother including the monster sprites. If you try loading a single player WAD on top of it, the game crashes as soon as you see a monster. Here's how sprites actually work in Vanilla: First off, SS_START and SS_END don't actually do anything. That's just a convention invented by tools like deutex to make sprite replacement more convenient. When you use the -file parameter, the PWAD is (internally to Doom) appended to the end of the IWAD directory. So for example: doom2.wad has 2919 entries. If you run doom2 -file foo.wad, the first lump of foo.wad is lump #2920, following right on from F_END (the last lump at the end of doom2.wad). When Doom has to look up a lump by name, it searches backwards from the end of the directory. So suppose that you have a PWAD that includes a TITLEPIC lump. When you load the game and the title screen is displayed, the game searches backwards from the end of the combined directory. Because it searches backwards from the end, it finds the TITLEPIC from the PWAD file first, before TITLEPIC from the IWAD. And your modified TITLEPIC is shown. This works for almost everything: graphics, sounds, music, levels etc. - because most lumps are looked up by name. The two exceptions are flats and sprites. Flats and sprites are special because they have to be enclosed in F_START..F_END and S_START..S_END. The game uses these markers to delineate ranges of lumps that can be used as flats or sprites. You can't just put flats or sprites into PWADs because they won't be in these ranges. Vanilla mappers back in the '90s figured out a trick to include replacement flats in PWADs. They include their replacement sprites in FF_START..F_END. The FF_START is actually unnecessary and just a convention. It's the F_END (with no F_START) that's the important part. Let's remember that the PWAD directory is joined to the end of the IWAD directory. So it looks like this:... lumps ... F_START FLOOR0_1 ...bunch of flats ... F_END ... more lumps ... [end of IWAD directory] [start of PWAD directory] FF_START ... bunch of replacement /additional flats ... F_END When Doom looks up F_START and F_END it gets the F_START from the IWAD and the F_END from the PWAD. So everything within that range is a valid flat, even if it isn't a flat (could be sounds, music, levels etc.). Fortunately the engine doesn't do any kind of checks on the lumps it finds, so it isn't a problem if you have random junk in there that isn't really a flat, and so the trick works. The sprite code isn't so forgiving - it has some strict checks in place that make sure the directory has the right number of sprites in place for each frame. So if you try to include a replacement sprite in an SS_START..S_END range, you get an error like this: R_InitSprites: Sprite POSS : A : 1 has two lumps mapped to it Because you now have POSSA1 twice instead of just once, and that's a no-no. It seems that Id did try to include some hacky code apparently intended to support sprites in PWAD files. But it doesn't work - I assume it was never tested: if (modifiedgame) patched = W_GetNumForName (lumpinfo[l].name); else patched = l; To translate: this means "if we're running with a PWAD, re-lookup the sprite name again to see if it's been replaced in the PWAD". But it doesn't work - there's other code elsewhere that requires all sprites to be within the S_START..S_END range or they don't work. But you can see the effect of this broken code if you try to include a replacement sprite bare in a PWAD: make a PWAD that just contains POSSA1 and nothing else, and when you run it in Vanilla you'll see that that particular frame just disappears. So PWAD sprite replacements just don't work in Vanilla. The ways to work around it are: (1) replace all sprites in the PWAD - in which case the IWAD sprites are completely ignored, (2) use a tool like deusf to modify the IWAD file and merge in the replacement sprites (lots of mods used this technique) or (3) use dehacked to change the sprite names. The dehacked technique is kind of clever. As we saw above, replacing sprites by extending the S_END range doesn't work, because now you have two POSSA1 lumps. But if you use dehacked to rename POSS to eg. XOSS, now you only have the XOSS from your PWAD, and the original POSS just gets ignored. But in the end, hacking up either doom2.exe or doom2.wad is pretty ugly. Personally I think the deusf approach is the best one, just because it's the most common, accepted approach and most source ports support it in some way or other. 3 Quote Share this post Link to post
Doomkid Posted February 16, 2016 Okay so, I'm really sorry for the extreme necrobump, but this question is relevant: I'm currently using WhackEd4, and in the "Strings" editing section, I don't see anything where I could change SKEL to SMGN, for example, to alter what sprites it uses. What am I doing wrong, where in WhackEd4 can I edit the sprite strings? Here's where I'm expecting to be able to find and edit sprite definitions yet can't: 0 Quote Share this post Link to post
scifista42 Posted February 16, 2016 You can't change sprite strings in Boom-extended-format DEHACKED, only in vanilla-format DEHACKED. You better rename the actual sprites in your wad to match the default names. 1 Quote Share this post Link to post
Doomkid Posted February 16, 2016 Oh, damn! That's a major setback, but thanks for the info scifista. 0 Quote Share this post Link to post
plums Posted February 16, 2016 Can't you just open the dehacked file in a text editor and change the string anyhow? Or will Boom ports reject that? (If it does work, no idea if WhackEd will preserve the change if you edit it more.) Text 4 4 SKELSMGN 0 Quote Share this post Link to post
VGA Posted February 16, 2016 It's better to replace the original sprites directly imo. In the source ports, you don't have to replace everything or jump through any hoops. SLADE is great, it supports mass renaming with asterisks to help you, too. 0 Quote Share this post Link to post
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.