ZDBSP output incorrect on ARM processors

[Csonicgo]

I've been playing various Doom source ports on the Raspberry Pi 2. For some reason, ZDBSP compiled for ARM targets cannot build nodes correctly, as the results of nodebuilding are chaotic, void-reaching walls, broken flat drawing, and HOM peppered everywhere.

The link below is E1M1 of DTWID built with the ARM compiled ZDBSP. Play this with ZDoom and turn "Enable Textured Automap" off. I don't know where it is in the menu, sorry. It might work with other source ports, I don't know.

Anyway, if you walk to the right, the wall warps and stretches into a total mess. A friend of mine said that it crashed GZDoom outright.

DOWNLOAD

The reason for the 2 MB download is that ZDBSP has no options whatsoever for exporting single maps from a compilation into a separate WAD. I didn't have access to a lump editor at the time!

I've heard explanations from floating point issues to GCC being dumb; all I know is that I'd like to get this resolved, as some maps for ZDoom are unplayable thanks to this.

[VGA]

Endianess?

[Gez]

Csonicgo said:

The link below is E1M1 of DTWID built with the ARM compiled ZDBSP. Play this with ZDoom and turn "Enable Textured Automap" off. I don't know where it is in the menu, sorry. It might work with other source ports, I don't know.

Main menu → Options menu → Automap options

By the way, the reason for turning that off is that the textured automap relies on GL nodes, so when it's on ZDoom will build GL nodes if needed and use them instead of normal nodes.

[Csonicgo]

Gez said:

Main menu → Options menu → Automap options

By the way, the reason for turning that off is that the textured automap relies on GL nodes, so when it's on ZDoom will build GL nodes if needed and use them instead of normal nodes.

That's good to know.

VGA said:

Endianess?

I think this may be something pertaining to floating point, given the observations by Blzut3 when I showed him.

[Csonicgo]

Here is a test file with five custom maps containing very simple structures. E1M1-E1M5. GLBSP found no warnings.

TEST1.WAD is the GLBSP version.
ZDBSP2.WAD is the ZDBSP version.
The debug output for ZDBSP is also included.

E1M3 is particularly interesting as the ceiling appears to rock back and forth depending on the view angle. The Donut sections just have some angles clipped off a bit.

Downloab

[Altazimuth]

After looking at the wads CSG posted I can confirm that map E1M3 acts really weird and E1M5 isn't too normal either. Comments are provided in the album: http://imgur.com/a/DQREs

[Csonicgo]

Wow.

[Edward850]

I can see through time!

[Csonicgo]

Sorry for necrobumping my own thread, but this still is a problem, and apparently has been fixed on the Android ports of GZDoom, so it'd be great if some collaboration would take place to fix bugs like this.

Of course, since I cant see the Android GZDoom source, I can't fix this myself!

ZDoom Forums bug report continues to gather dust because I'm the only one that uses ZDoom on not-x86, besides eevee, and he can't figure it out either.

At my wits end here because this breaks mods that request zdoom to build nodes, which is becoming more and more common. Any ideas on where to start debugging this?

[Quasar]

Csonicgo said:

At my wits end here because this breaks mods that request zdoom to build nodes, which is becoming more and more common. Any ideas on where to start debugging this?

The first thing to do is determine what is wrong with the data structures. Then through examination of the code, you try to determine what would cause those deviations.

Of course we already discussed how similar this is to EE's problem with float->int conversion so that's the first thing that should be examined IMHO.

[Graf Zahl]

Since you are the only one having some system to test this on, you'll have to run any tests yourself.

ZDBSP contains lots of debug output options, so I'd suggest to compare what they produce with an Intel system's output.

About Android: If it works there, can it be that your compiler is the culprit, not ARM itself?

Just saying 'it doesn't work' repeatedly won't fix this as none of the ZDoom devs have a system to run such tests.

[Blzut3]

Csonicgo said:

Of course, since I cant see the Android GZDoom source, I can't fix this myself!

Assuming you're referring to the Beloko port, then yes you can. You just have to email him.

Csonicgo said:

because I'm the only one that uses ZDoom on not-x86

I have PowerPC Macs which ZDoom runs on, and I don't recall running into any node builder errors there.

[RestlessRodent]

ARM may have differing alignments from x86, does ZDBSP and/or GZDoom directly read/write structures as they appear in memory?

Virtually all modern CPUs use IEEE 754 floating point so math operations should not cause trouble unless either of the two require processor specific results.

Another possibility is that ZDBSP and/or GZDoom rely on the strong memory order of the x86 CPU, whereas ARM is weak similar to PowerPC.

[Quasar]

GhostlyDeath said:

Virtually all modern CPUs use IEEE 754 floating point so math operations should not cause trouble unless either of the two require processor specific results.

It's not a matter of processor specific issues. It's a matter of undefined behavior. In particular, the conversion of a float to an unsigned int is either undefined or is implementation-defined, by the C and C++ standards, if the value would be out of range when converted to an unsigned int. The behavior of GCC and clang backends for ARM is completely different when dealing with this situation than for x86(-64). Whether or not that's due to the instruction set or a choice in the compiler is irrelevant, because the compilers are not in error in doing this - the standard states the behavior is not defined so the compilers can do whatever they want.

Now whether or not this has anything to do with the problem at hand, I don't know. I just know Eternity had a similar problem caused by such a conversion in its renderer which would cause sidedness decision reversal during BSP traversal. So it's worth looking into as a potential vector for incorrect math results.

[RjY]

Csonicgo said:

Here is a test file with five custom maps containing very simple structures. E1M1-E1M5. GLBSP found no warnings.

Could you re-upload this?

[Csonicgo]

RjY said:

Could you re-upload this?

Sure, hold on.... http://speedy.sh/txsy3/ZDBSP2.7z

There you go! This is the version of the map before the "fix" to ZDBSP that didn't seem to work at all in this situation...

[beloko]

Blzut3 said:

Assuming you're referring to the Beloko port, then yes you can. You just have to email him.
I have PowerPC Macs which ZDoom runs on, and I don't recall running into any node builder errors there.

Hey, actually I put all the code on Github now:

https://github.com/emileb/OpenGames/tree/master/opengames/src/main/jni/Doom/gzdoom-g1.8.6

There was a really weird issue where all the walls were messed up and sometimes missing, if this is the same issue it's due to some difference on the ARM compiler, never bothered to find the exact technical reason.

Anyway for a fix search for __ANDROID__ in the above code and copy the bits which involve angles such as:

#ifdef __ANDROID__
angle_t angle = angle_t((int)(v * ANGLE_90/90.));
#else

Should fix it.

[Gez]

beloko said:

Hey, actually I put all the code on Github now:

https://github.com/emileb/OpenGames/tree/master/opengames/src/main/jni/Doom/gzdoom-g1.8.6

Interesting. Going up a level I see there's also GZDoom 2 and Crispy Doom. Does D-Touch now offer a choice of five ports?

[beloko]

Gez said:

Interesting. Going up a level I see there's also GZDoom 2 and Crispy Doom. Does D-Touch now offer a choice of five ports?

No Cripsy Doom is not built in to the build any more, it was replaced with Choc Doom v2.
It has 2 version of GZDoom 1.8.6 and a pre release of 2 from last year, going to replace it with 1.8.10

[Danfun64]

Pity. I would have had Crispy Doom replace Chocolate.

[Gez]

beloko said:

It has 2 version of GZDoom 1.8.6 and a pre release of 2 from last year, going to replace it with 1.8.10

GZDoom just released versions 1.9.0 and 2.1.0.

[Blzut3]

beloko said:

There was a really weird issue where all the walls were messed up and sometimes missing, if this is the same issue it's due to some difference on the ARM compiler, never bothered to find the exact technical reason.

The technical reason is that the conversion from negative double to out of range integer is undefined (in these cases negative double to unsigned). In the case of ARM the conversion is clamped. In the case of PowerPC or x86 it appears to be the same as if converting a signed int to unsigned int.

Your changes are helpful, but are the cases in p_acs.cpp observed cases? That's entirely integer math and STACK(x) is already a signed integer.

[beloko]

Blzut3 said:

The technical reason is that the conversion from negative double to out of range integer is undefined (in these cases negative double to unsigned). In the case of ARM the conversion is clamped. In the case of PowerPC or x86 it appears to be the same as if converting a signed int to unsigned int.

Your changes are helpful, but are the cases in p_acs.cpp observed cases? That's entirely integer math and STACK(x) is already a signed integer.

Ahh thanks for the explanation, very interesting. Something to watch out in the future.

Yes actually those in p_asc.cpp may well not be needed, they were probably added when I was getting desperate trying to find a related bug.