E6M4 and beyond in Vanilla Heretic

[plums]

I was going to post this in the "Obscure fact" thread, but decided it might warrant it's own post. It's not something I've seen documented anywhere before, so maybe no one else knows about it. Exciting!

 

In Heretic SotSR, there's a hidden mini-episode of E6M1-E6M3, maps that were meant for deathmatch, that you can only access with -warp or the ENGAGE cheat (Heretic's version of IDCLEV). This is not the obscure part.

 

In Vanilla (DOS) Heretic, but not Chocolate Heretic, you can access more levels in E6, if they exist.

 

Normally E6M3 has no exit, but if you make a replacement E6M3, you can exit to E6M4. E6M4 plays the title screen music; trying to exit will crash the game with the message "Bad V_DrawPatch". (What is it trying to draw/read? Can adding the right lump prevent this?) You can't seem to access E6 maps higher than E6M3 with -warp or ENGAGE, just with level exits.

 

Here's the fun part: you can go to E6M9 with a secret exit. In DOSBox this will produce a whole bunch of "Illegal read from" errors but it will still work. I presume this is from the exe trying to read an invalid index or something for the music. (Will it even work in actual DOS? I don't know! Anyone have a real DOS computer going that wants to try?)

 

If you exit E6M9 with a normal exit, you go to E6MT. (What?) E6M9+ has no music, and exiting to E6MT and further will have no music on the intermission screen even, probably because of the illegal read business.

 

From there, you go to E6MU, V W X Y Z [ \ ] ^ ... Those of you that are familiar with ASCII character codes will know this pattern. I had to hex-edit the wad to get E6M^ because SLADE didn't want to save a lump with that name in it. After that you go to E6MA through E6MZ again, which is the lower-case letters, but even if map lump names with lowercase letters like E6Ma exist (again through hex-editing), the full uppercase E6MA gets used. Then on to E6M{ | } ~ then E6M(delete key, 0x7F).

 

I didn't go any further than this but exiting E6Mdel will crash saying it can't find E6MÇ, so presumably you can go through the full extended ASCII range. After that, probably some kind of exciting overflow crash! edit: Updated! See below.

 

Anyhow here's a wad if you want to play with it. The left exit is the normal exit, and the right one is the secret exit.

 

 

htic_e6.wad.zip

 

Make sure to get fontb from xttl as well!

Edited June 27, 2020 by plums

[DynamiteKaitorn]

This is rather interesting! I'm guessing episode 6 wasn't coded very well or the secret exit of episode 6 can be exploited. Honestly, I'd love to see all ASCII character maps appear mostly because I wanna reach the final character to absolutely nuke the engine. XD

[Dark Pulse]

3 hours ago, plums said:

Normally E6M3 has no exit, but if you make a replacement E6M3, you can exit to E6M4. E6M4 plays the title screen music; trying to exit will crash the game with the message "Bad V_DrawPatch". (What is it trying to draw/read? Can adding the right lump prevent this?)

Probably some sort of graphic for E6M4 is trying to be loaded by the intermission screen, and since it doesn't exist, boom.

Edited June 26, 2020 by Dark Pulse

[Gez]

4 hours ago, plums said:

I had to hex-edit the wad to get E6M^ because SLADE didn't want to save a lump with that name in it.

Go to Edit->Preferences->Editing and check the percent encoding box. Default behavior is to translate \ in lump names as ^ in other archive formats; which is a ZDoom convention, but it has the side effect that ^ in a lump name in other archive formats gets turned into a \ in a WAD, so it doesn't expect WADs to actually contain the caret as a character in entry names.

 

You can also uncheck "Force uppercase in WAD archives" there.

Edited June 26, 2020 by Gez

[plums]

2 hours ago, Dark Pulse said:

Probably some sort of graphic for E6M4 is trying to be loaded by the intermission screen, and since it doesn't exist, boom.

Maybe, but what? Heretic's levels don't use a CWILVxx graphic for level names like Doom. It has a bitmapped font and the level names are just text strings in the exe. And none of the other E6 levels have names, and none of them cause crashes.

[Dark Pulse]

That I couldn't tell you. All I know is that it's trying to find some graphic to draw, failing to find it, and bombing as a result.

[plums]

So, after reaching E6M 0xFF, the game will not in fact crash, but just loop around to E6M 0x00 (which is just "E6M" because of null-terminating strings) and then go to E6M 0x01 etc, until you get back to E6M1 again! From there going to E6M5 will not crash with Bad V_DrawPatch this time, and you can keep going around -- I bombed out this time at E6M: because I hadn't made that level lump before. I've added E6M: through E6M@ now, too, but haven't run the wad with those additions yet.

 

Looping around past E6M 0xFF is probably overflowing/overwriting something in memory but I'm not going to try figuring out what.

 

I've updated the download in the first post. Maps past E6Mdel, before E6M1, and from : to @ don't have the mapname drawn in the map, and put you right in front of the normal exit.

Edited June 26, 2020 by plums

[plums]

3 hours ago, Gez said:

Go to Edit->Preferences->Editing and check the percent encoding box.

Hey, that's pretty cool. It seems that SLADE won't save characters in wads above 0x7F though, replacing them with ? instead. (Not that I expect anyone else will ever need such a feature...)

[xttl]

Illegal read EIPs:

 

180:1ACEE0 = CS:1DEE0, which is in IN_DrTextB

180:1AD0E0 = CS:1E0E0, which is in MN_TextBWidth

 

I think they're happening while it's trying to draw intermission level names.

 

It also seems E6M1-E6M3 intermission screens only work without any errors by pure chance. :P

 

The LevelNames[] string array ends on E5M9, there are no entries (even empty ones) after that, and there are no special E6 checks in IN_LUDE.C that I see...

 

Do the original E6M1-E6M3 maps even have any exits?

[plums]

2 minutes ago, xttl said:

Do the original E6M1-E6M3 maps even have any exits?

E6M1 and E6M2 do, but not E6M3. E6M3 was E4M1 in Heretic before the SotSR expansion.

[P41R47]

So, if i understand it right, E6 could potentially have more than 256 maps?

DAMM! Fate's Path sure is long!

[xttl]

Since LevelNames[] is defined in AM_MAP.C, right after it in memory come initialized variables used by the automap code.

 

E6 map name pointers:

 

E6M1 = cheating state (affected by ravmap, 0-2)

E6M2 = grid state (0/1)

E6M3 = leveljuststarted  (0/1)

E6M4 = automapactive (0/1)

E6M5 = finit_width (SCREENWIDTH = 320)

E6M6 = finit_height (SCREENHEIGHT-42 = 158)

E6M7 = scale_mtof (INITSCALEMTOF = 0.2*FRACUNIT = hex 0x3333 = 13107)

E6M8 = followplayer (0/1)

E6M9 = cheat_ravmap (ascii "ravmap", first four letters reversed because little endian: 0x6D766172)

 

So E6M5 is the first map for which the name pointer has a slightly larger value, and E6M9 points to a very high (with high probability invalid) memory address.

Edited June 26, 2020 by xttl

[Chip]

let's rename the Episode 6 levels to something more appropriate! 

[plums]

@xttl That's absolutely hilarious, thanks for that.

 

Does that make the canonical names for those maps: cheating, grid, leveljuststarted? lol (or perhaps 0, 0, 1)

 

edit: So the same thing is happening for E6M9 -> E6MT. The afterSecret array in G_GAME.C only has 5 entries so it's just getting T (decimal 84) from.. something else?

Edited June 26, 2020 by plums

[xttl]

Actually I don't get why E6M5 is the first one to crash, because in DOSBox there seems to be the same pattern of bytes (60 10 00 F0) in memory at both address 0 and 320 (0x140) which the game tries to draw as text with the large green font). Maybe I'm missing something...

 

Anyway, attached here's a wad with more letters defined for the big green font (I just copy/pasted 'x' many times after 'z' in SLADE), load it and the game will no longer crash trying to draw E6M5's name.

fontb.zip

 

edit: instead, it'll crash entering E6M7 but in W_GetNumForName

Edited June 26, 2020 by xttl

[plums]

17 minutes ago, xttl said:

instead, it'll crash entering E6M7 but in W_GetNumForName

Possibly trying to read an invalid name for a music lump? In SOUNDS.C the next thing defined after the S_music[] array is S_sfx[], with the first entries being a bunch of nulls.

 

edit: :D :D :D

 

Edited June 26, 2020 by plums

[Gothic]

1 hour ago, P41R47 said:

So, if i understand it right, E6 could potentially have more than 256 maps?

DAMM! Fate's Path sure is long!

Fate's Path community project, coming 2070.

[xttl]

10 hours ago, plums said:

Possibly trying to read an invalid name for a music lump? In SOUNDS.C the next thing defined after the S_music[] array is S_sfx[], with the first entries being a bunch of nulls. 

 

That's it, I overwrote those nulls with MUS_E1M1 and it works now.

 

Which map's name is in the pic btw? For me E6M7 looks like:

 

(maybe it has something to do with this)

Edited June 26, 2020 by xttl

[plums]

12 minutes ago, xttl said:

Which map's name is in the pic btw?

That's me exiting E6M6. Are you running in DOSBox?

[xttl]

10 hours ago, plums said:

That's me exiting E6M6. Are you running in DOSBox?

 

Yeah, official 0.74-3 Windows build. But: since E6M7's name pointer is scale_mtof, the name will actually depend on whether automap was opened and what was the zoom level. Going straight for the exit in E6M6 I get "entering: ... xxx vcpi xxxx ...". Just opening/closing automap makes it crash due to Bad V_DrawPatch, but opening map and zooming in a little bit makes the level name empty or just a single 'y'. :-)

 

edit: well, I tested it on my Linux box which has 0.74-2 I think I might have built from source myself a long time ago with possibly some patches.

There, I just get "xx xxxxxx" by default (and again a bit different results by opening automap and zooming)

Edited June 26, 2020 by xttl

[Can't play on Nightmare]

Episode 6. Seriously?

[plums]

36 minutes ago, xttl said:

Just opening/closing automap makes it crash due to Bad V_DrawPatch

Trying to find a font patch for a null character, maybe? Too bad.

 

Time for someone to make a maze level, where you might crash the game if you open the automap! (No one actually do this please.)

Edited June 26, 2020 by plums

[Redneckerz]

@plums Very neat discovery!

 

Stupid question: Does this behavior also exist in Heretic-plus?

[plums]

4 minutes ago, Redneckerz said:

Does this behavior also exist in Heretic-plus?

It does. In fact that's what I first tried it on, since I haven't had a proper copy of heretic.exe on a computer for a while. (I first found out about this a long time ago, and then kind of forgot to explore it further until lately.)

 

Just to be sure, I went and bought the original on Steam today since it's on sale for < $2 :p

[xttl]

Memory in DOSBox 0.74-2 Heavydebug (selfbuilt) and DOSBox 0.74-3 Debug (Win32 build posted by one of the devs) near 0x3333 (default value of scale_mtof without opening map)

 

So that's where the VCPI was coming from, and it is indeed some string related to that VCPI. All lowercase letters become 'x' using fontb.wad because the game doesn't bother to convert strings to uppercase despite the fonts missing all lowercase glyphs.

 

 

edit: those are strings from inside DOS4GW (near beginning of heretic.exe)

 

Edited June 26, 2020 by xttl

[xttl]

I figured out why the earlier maps never seem to cause any problems for the intermission code despite first bytes at DS:0 looking like they should cause some:

 

It's because of the

x = 160-MN_TextBWidth(LevelNames[(gameepisode-1)*9+gamemap-1]+7)/2;
IN_DrTextB(LevelNames[(gameepisode-1)*9+gamemap-1]+7, x, 170);

+7 in that calculation.

 

So in reality, it is looking for the map name string at DS:7 instead of DS:0, DS:147h instead of DS:140h, etc. At DS:7 there is a null straight away which terminates the string.

 

Why is there even a +7 there? Because it's re-using the automap names array but skipping "ExMy:  " from the beginning.

 

Also explains why DS:3333h does not exactly line up with the beginning of what it shows as the name of E6M7 (it's coming from DS:333Ah instead)

Edited June 26, 2020 by xttl

[plums]

12 minutes ago, xttl said:

I figured out why the earlier maps never seem to cause any problems for the intermission code despite first bytes at DS:0 looking like they should cause some:

Nice. I knew about the +7 string offset, and the lack of characters for lower-case strings, from using HHE and @ETTiNGRiNDER's documentation on it, since you have to put your level names in ALL CAPS and you can't omit the ExMy to save space for longer map names, like you can for Doom.

 

Also FONTB actually just starts at 33d (exclamation mark) so that still leaves 31 control characters that can possibly cause a Bad V_DrawPatch error I guess? I tried putting more font characters before FONTB_S in the wad you posted, trying to not get a crash on E6M6 after opening the map, but no luck.

 

I'm also surprised just bringing up the automap doesn't cause any errors since the map names should also be displayed there.

Edited June 26, 2020 by plums

[xttl]

10 hours ago, plums said:

Also FONTB actually just starts at 33d (exclamation mark) so that still leaves 31 control characters that can possibly cause a Bad V_DrawPatch error I guess? I tried putting more font characters before FONTB_S in the wad you posted, trying to not get a crash on E6M6 after opening the map, but no luck.

 

There's a check for char<33 in both MN_TextBWidth and IN_DrTextB (it skips these characters and just increments current x coordinate by 5 or 8), but no such check for char>x. Long strings themselves can cause a problem even with no missing glyphs because V_DrawPatch* wants to make sure that all drawing stays within screen boundaries (it doesn't even allow overflowing to the next row with low y values but x>=320), and you never know how long it takes to get to the first null byte when it's reading garbage as level names.

 

10 hours ago, plums said:

I'm also surprised just bringing up the automap doesn't cause any errors since the map names should also be displayed there.

 

They actually thought to check for episode<6 in AM_MAP.C line 1346 (or episode<4 if not using the Shadow of the Serpent Riders wad).

[plums]

1 hour ago, xttl said:

They actually thought to check for episode<6 in AM_MAP.C line 1346

Ah, I missed that.

 

So, probably the level name of one of the E6 levels (or old E4M1) was crashing when viewed in the automap, and instead of adding map names or doing error checking or anything, they just... didn't display that string on the map. Problem solved!

 

Thanks for taking an interest in this! your posts have made this thread 10x better imo :D

[P41R47]

Now now now, please @plums. Explain this to the uneducated us that are following this thread.
We, theorically, can make -insert number- maps for Episode 6 and access them through a pwad even on vanilla? Shortcomming that it wouldn't be listed on the Episode list, right?

Edited June 26, 2020 by P41R47