Spechits, Reject and Intercepts Overflow Lists

[Hitherto]

REQUIEM.WAD MAP06
Bonus demo rt06fail.lmp inside rt063746.zip by vdgg, "All ghosts" effect at ~15:30, not detected by PrBoom+, crashes doom2.exe under win9x, playbacks correctly only in doom2.exe under "plain" DOS environment (or DOS box as I suppose).

Strange thing. I don't see there any obvious reason for an intercepts overflow.

[myk]

Yeah, intercepts can go wild in some situations. Maybe flaws in calculations that make the shot traverse the map more than once? Also, see the linedef 0 comments earlier in the thread and this post by entryway.

[kimo_xvirus]

Doom2 Map02, all ghosts while trying UV Max back in May 2010
plays back ok in PrBoom+

map02gho.zip

[Andy Olivera]

Not sure if the list is still being updated, but I found another E4M1 Spechits example.

e4ns1246.lmp - lines 256, 262, 263, 266, 268, 269, 270, 272, 273.

Works fine in PRBoom+ with emulation enabled.

[Grazza]

That one is already listed in the table and a comment made in the final column, since it fails on the next map without emulation.

[Vile]

Doh, I had one of these happen to me in my pl32 sessions. I would have saved the demo if I remembered this thread. :p

[Andy Olivera]

Grazza: Oops. Duh! I guess I should pay closer attention next time (I thought the list was in alphabetical order, but I could've sworn it wasn't there)...

[Andrea Rovenski]

Not sure where to post this, but I think thread is appropriate. If not, I apologize. I was doing Memento Mori MAP23 -respawn attempts, and then I got a strange bug, which I think is the "all ghosts" bug. If I've read this thread correctly, then I would assume this demo would be relevant. Was recording with prboom+ cl2.

wtf.zip

[Andromeda]

Sorry for the bump, but recently I encountered a Intercepts Overflow while playing MAP18: Lake Poison of Alien Vendetta (2nd release) while clearing out the north side room of the big central arena, that resulted in the "all-ghosts" bug. This was on PrBoom+ 2.5.1.4 -complevel 2 (the version found here), but sadly I wasn't recording a demo, so I'm afraid I can't provide more details.

[RjY]

How to fix yaotzin.wad's reject overrun to play yao-750.lmp

 

Summary: Zero-extend the map's reject table to the correct size. Or build a "true" reject table, either works. Then flip three bits, to block line-of-sight from sector 636 to sectors 249, 277, and 289. If done correctly the demo should play back fully and exit at 7:50 with 100%/62%/100% on intermission.
 

I am not sure how to share this: I do not believe I am allowed to distribute a modified wad.
 

If zennode's RMB options support worked I could suggest to use the following as an options file, but it seems to be broken (as I am not the only one to find).

EXCLUDE 636 249
EXCLUDE 636 277
EXCLUDE 636 289

Regardless, assuming zennode (with -n- -b- to suppress node and blockmap rebuilding), one may either use -rz to fill the proper-sized table with zeroes, or -rg to build a true reject table, as long as the three bits are set afterwards. Either worked for me: I tested playback in both PrBoom+ and Chocolate Doom. But I would like others to verify it if possible. Then, the community can cross one more troublesome desync off the list :).
 

Some more-or-less technical notes follow.
 

________________

 

yaotzin.wad has a short reject table. The map has 640 sectors thus requiring 640^2 = 409600 bits = 51200 bytes reject table data. But it only supplies 50404 bytes = 403232 bits. [Note that this is the correct size for a table of 635 sectors: 635^2 = 403225 bits rounded up to the next multiple of 8. Evidently the author added 5 more sectors and forgot to update the table afterwards.] This means any line-of-sight check either starting in a sector >= 631, or starting in sector 630 and ending in a sector >= 32, will overrun the table when testing to see if the full line-of-sight calculation can be skipped.
 

A short reject table means Doom will read random data "past the end of the array" as reject bits for line-of-sight checks. This was whatever was in Adam Hegyi's computer's memory after the place where Doom loaded the reject table. For Andy Olivera's yaotzin manufactured demo it's sufficient to assume all zeroes but that doesn't work in Adam's case. Nevertheless all else being equal we should be able to get the demo to play back correctly if we can guess the state of enough of the bits of random data that happened to be in Adam's computer that day.

 

Now there are 409600-403232=6368 out-of-range bits. It would take longer than the age of the universe to try every possible combination. However, observe that Doom should return the same result (sight blocked) regardless of what the reject table says if a sector pair is provably invisible. [Just for fun: when could this observation fail to be true? Fortunately it is vanishingly rare.] It will just take a bit longer to reach that answer if the reject table has a zero in that position. So if we use a reliable reject builder on the map, we can ignore all sector pairs it proves are invisible to each other. This reduces the search space to 335 bits; still far too much for an exhaustive search.

 

So we start by looking for the first line-of-sight check that's out-of-range of the short reject table, but is between a proven-visible sector pair, and the result is a success. If the demo visibly desyncs immediately after, then it is likely that line-of-sight check failed during recording. All other things being equal, this most likely means the corresponding bit in the overflow of the reject table in Adam's computer's memory was set. So, we try flipping that bit in our reject table, and see if it improves playback.

Restarting the demo after each bit-flip yields the next sector pair to try. Really this is a tedious process of trial and error. It needs intuition for sensing that a watched demo has desynced, and modifying the engine to dump lots of information about the results of line-of-sight checks.

The first candidate is given by a trooper, thing 169, who had already been woken and wandered up the stairs to the landing, sector 636, and who looks at the player in sector 277, on tic 5218. To be fair flipping that bit did not make a great deal of difference visibly; if anything the player went even more off-course. But it gave the next candidate, supplied by the same trooper on tic 5286: 636 could not in fact see 249, halfway up the stairs.

These two bits flipped took Adam a long way; after disposing of a revenant with plasma, he was no longer missing the trooper, but hitting him dead-on with the super shotgun. It also seems the trooper is meant to shoot him first. Furthermore, he made it all the way across to land another successful super shotgun hit on the chaingunner (thing 163) in the northeast, by the BFG secret. But the next monster, the spectre (thing 172) was too far away to die when Adam shot it, and remained at the top of the lift waiting when he returned from below.
 

I remained stuck here for a while because the next candidate was the aforementioned chaingunner sighting Adam from the top of the stairs on tic 5320. But blocking sight of sector 247 from 636 made things noticably worse; the chaingunner evidently needs to see the player and be woken at that point. That bit must have been a zero on Adam's computer.

The next candidate was 636 itself, but 636 can obviously see into itself or nothing would attack the player up there. 636 must not be able to see 277 or things go wrong much earlier; nothing else seemed relevant.

I floundered around for a while, making no progress, too intent on when monsters were being woken up by their first successful line-of-sight to the player. Also labouring under the misapprehension that the demo was still synced at the point when Adam kills the chaingunner, because it looked like an accurate shot.

Finally after a bit of shotgun debugging (not literally! The metaphorical kind, flipping bits at random) I noticed in the output that when Adam briefly passed through sector 289, on tic 5563, the chaingunner checks he still has Adam sighted to know to keep attacking.

I had earlier believed I was on the right track because Adam's shot had killed the chaingunner regardless, but if 636 could not see into 289, the chaingunner loses sight and stops firing a bit early. That change perturbed the random number sequence favourably to bring the spectre into the deadly range of to Adam's shotgun before he went down for the BFG secret.
 

Then Adam jumped off the landing onto the yellow key, after which, although there were some movements I mistook for desyncs at first, evidently they were part of the recording, because playback continued, and continued, and continued; until, to my delight, it reached intermission.

[vdgg]

On 11/1/2021 at 1:03 AM, RjY said:

yaotzin

So what is the best approach if I wanted to record a new demo? Just an advanced port (PrBoom+, DSDA Doom) with complevel 2 so it will playback with these ports? Is it the same map issue as bldnight MAP05?

[RjY]

16 hours ago, vdgg said:

So what is the best approach if I wanted to record a new demo? Just an advanced port (PrBoom+, DSDA Doom) with complevel 2 so it will playback with these ports?

 

This is probably the best way to proceed, short of fixing your copy of the wad and having your viewers do the same. Which would be highly unconventional. Relying on source ports is most likely the better trade-off.

 

This thread has shown it does not seem possible to achieve demosync consistently on maps with short reject tables in vanilla doom2.exe. In vanilla I believe memory "past the end of the array" consists of 16 bytes of z_zone header followed by an array of (in yaotzin's case, 4924) pointers to linedefs. Not fully random, but still, which pairs of sectors have their sight blocked is highly dependent on how Doom and DOS decided to lay out memory on the running system.

PrBoom+ and Chocolate Doom both seem to pad reject overruns in exactly the same way. I guess that's as close to a standard as one can hope for. Furthermore the padding is unlikely to affect the map's gameplay, as follows. The ports emulate the z_zone header but leave the following pointers all zero, so only a few bits are set, limiting the effect to sector 630 being unable to see into sectors 35, 40, 42, 43, 46, 97, 100, 101, 128, 132, 137, 139, 142, 144, 146, 147, and 148. Of these seventeen, the only one where a line-of-sight is possible is from 630 into 142. 630 is a 64x64 teleporter pad. So for the effect of the ports' padding to matter, a monster would somehow have to skip the teleporter lines, then check sight to its target in the surrounding blood lake. I'm not saying that's impossible, but you'd have to be deliberately trying to do it.

 

16 hours ago, vdgg said:

Is it the same map issue as bldnight MAP05? 

 

I haven't looked into it fully but a cursory glance suggests so. The map has 485 sectors requiring a 235225 bit = 29404 byte table, but only supplies 21218 bytes = 169744 bits which I note is the correct size for 412 sectors. Lines of sight from sectors >= 350 or from sector 349 into sectors >= 479 are out-of-range.

An extra complication with bldnight map05 is the lump size: 21218 isn't a multiple of 4.