The Play Station Doom Source Code Released! (Reverse Engineering)

[fenderc01]

I am able to compile this and run it in an emulator, but the CD audio is messed up. I verified it's not the emulator by playing the tracks back in UltraISO. Am I doing something wrong?

[Gerardo194]

Hmmm, that's weird... We have no problems so far. Let's expect an answer to this problem...

We use XEBRA emulator...

[fenderc01]

59 minutes ago, Gerardo194 said:

Hmmm, that's weird... We have no problems so far. Let's expect an answer to this problem...

We use XEBRA emulator...

I get the same problem when running with XEBRA. I think it may be an issue with when I run the MAKE_CD.bat batch file.

[Gerardo194]

Have you modified the code or something?? My brother is working now, he will see what's happening when he comes back home. If anyone else can solve this problem, it's very welcome...

[Gerardo194]

@fenderc01 the issue you're having is the music track is another one that isn't the correct one but it instantly changes to the one the must be played, right??

If so. try to compile it again... 

[fenderc01]

Edit: The left picture is the unmodified CD image and the right picture is the CD image created by MAKE_CD.

 

31 minutes ago, Gerardo194 said:

Have you modified the code or something?? My brother is working now, he will see what's happening when he comes back home. If anyone else can solve this problem, it's very welcome...

I haven't modified anything. The only possible variable I can think of would be if I'm using the wrong version of the game. I've tried the US release 1.0 and 1.1.

 

Quote

@fenderc01 the issue you're having is the music track is another one that isn't the correct one but it instantly changes to the one the must be played, right??

If so. try to compile it again... 

The problem I'm having is the CD tracks are playing back too fast and have a crackling sound. When I create the image, the file sizes are smaller too.

 

Track02.zip

Edited February 13, 2020 by fenderc01

[Gerardo194]

Hmmm, this is a complicated situation... Let's wait for Erick or anyone else to help us... I don't know anything about it,

This version is 1.0 so far, there have been updates adding code from 1.1 as far as I know. 

[intacowetrust]

Is it something to do with cd sector encodings and whether the .RAW data is interpreted to include the CD sector header and error detection information or not?

 

A raw CD sector (for mode 1) has 2,352 bytes of information and encodes 2,048 bytes of data. The ratio between the two (2048 / 2352 = ~ .8707) looks similar to the skew you are seeing in the file sizes above. For example for DOOMRAVE.RAW 19204/16722 = 0.8707.

[fenderc01]

Edit: I updated the image to show a better way of extracting the audio tracks. Once extracted, you will need to rename the files to match those in the CDAUDIO folder.

 

20 hours ago, intacowetrust said:

Is it something to do with cd sector encodings and whether the .RAW data is interpreted to include the CD sector header and error detection information or not?

 

A raw CD sector (for mode 1) has 2,352 bytes of information and encodes 2,048 bytes of data. The ratio between the two (2048 / 2352 = ~ .8707) looks similar to the skew you are seeing in the file sizes above. For example for DOOMRAVE.RAW 19204/16722 = 0.8707.

 

You are correct.

 

I ended up extracting the individual audio tracks from the bin/cue using ISOBuster instead of extracting the raw files from the CDAUDIO folder using UltraISO.

 

Edited February 14, 2020 by fenderc01

[Gerardo194]

Thank you @intacowetrust you are great!! 

[riderr3]

On 2/9/2020 at 8:14 PM, Gerardo194 said:

@intacowetrust I just found an error in this sentence... What my brothet meant was: are you interested in working on both PSX Hexen and Quake 2?? Because it's something he is interested in...

 

I apologize for interfering with your discussion, but I could recommend creating a discord channel for GEC Team projects and indicating the links in the "about" section of your profiles so that we can communicate independently of the forum. As for Hexen and Quake 2, this would be interesting, especially if you plan to convert The Reckoning and Ground Zero which were not included in the Q2 PSX release, as well Deathkings of the Dark Citadel (as for mapping and optimization - here I can also help if tools will be available, like I did for Master Edition). Mention @Erick194 as well.

Edited February 14, 2020 by riderr3

[Gerardo194]

@riderr3 that's n awesome idea!!! I will discuss this with Erick, I'm pretty sure he will like the idea, thanks!!

47 minutes ago, riderr3 said:

The Reckoning and Ground Zero

Actually, we have never played the Quake 2 expansion packs but having rhem available for the PSX version of the game would be awesome as well.

[Gez]

It's not PSX so the hardware is totally different, but the SNES Doom and GBA Doom II would be very interesting to reverse engineer because their engines are not derived from the Doom engine at all.

[Immorpher]

This is a blessing to the community! :D

I am a total newb to reverse engineering and compiling (only really have worked with interpreter languages). But I remember hearing with Mario 64, after being reverse engineered, nintendo didn't apply compiler optimizations when they compiled. So fans recompiled the game with compiler optimizations and there is a clear FPS boost in a lot of its areas on original hardware. Now ID probably had midway enable them for sure.

Also I am not sure how much reversing you guys did for DZDoom with Doom 64, but that would be a blessing too. Oh and the offer still stands for me to test any hacked/recompiled Doom 64 roms with my flashcart (Everdrive 64) still stands :)

[taufan99]

2 hours ago, Gez said:

It's not PSX so the hardware is totally different, but the SNES Doom [...] would be very interesting to reverse engineer 

Reminds me of two people who attempted to hack the SNES DOOM.

 

I wonder if somebody will continue bringing their torch(es) anytime soon...

[Job]

1 hour ago, taufan99 said:

Reminds me of two people who attempted to hack the SNES DOOM.

 

I wonder if somebody will continue bringing their torch(es) anytime soon...

They got pretty far (IMO), promised some updates, then disappeared. As I recall, they even discovered how to play custom maps within the game. 

[Erick194]

@fenderc01 thanks for informing us about that audio error, the problem from what I see was already known by the creator of the mkpsxiso program, but that error exists in version 1.21 which is the one that I have modified, but that error was corrected later In version 1.23, I have updated the program and modified it so that I can generate the files (PSXCDABS.C and PSXCDABS.H).

 

https://github.com/Erick194/PSXDOOM-RE/commit/a1d98ec6a5ebf75570dbfceef29e6a416bf9f2a1

[Erick194]

@intacowetrust thanks for that information from the sector CD Audio.

[Erick194]

@riderr3 it would be interesting to create the [GEC] channel for discord, I will keep it in mind for this weekend.

[Erick194]

5 hours ago, Immorpher said:

Also I am not sure how much reversing you guys did for DZDoom with Doom 64, but that would be a blessing too. Oh and the offer still stands for me to test any hacked/recompiled Doom 64 roms with my flashcart (Everdrive 64) still stands :)

Rest assured, I am working on reverse engineering of Doom64 for the real hardware as well.

I will consider you for real hardware tests. ;)

[Immorpher]

Indeed a discord server would be great! I would have to refrain from posting constant praise on it though. haha

[intacowetrust]

23 hours ago, Immorpher said:

I am a total newb to reverse engineering and compiling (only really have worked with interpreter languages). But I remember hearing with Mario 64, after being reverse engineered, nintendo didn't apply compiler optimizations when they compiled. So fans recompiled the game with compiler optimizations and there is a clear FPS boost in a lot of its areas on original hardware. Now ID probably had midway enable them for sure.

 

Yeah there's many places in the PSX Doom machine code where optimizations can be spotted. There's plenty of instances of inlining, and also code de-duplication where the instructions for similar code blocks are collapsed into one. Other tricks commonly used by the compiler in this binary include reducing multiplies (by a constant factor) to a series of adds and bit shifts, reducing division (by a constant factor) to multiply by a reciprocal, and eliminating branches (for values that depend on them) by using bitwise operations if possible.

 

Having that said there seems to be many instances where the optimizer seems to make odd choices for inlining, like inserting very large functions into smaller ones. There also seems to be a lot of dead/unused code in final executable, both for functions that are inlined everywhere (and never called explicitly anymore) or for library functions like from LIBGPU or LIBGTE that are simply never used at all.

 

I often wonder if this was recompiled with a more modern toolchain how much more smaller & efficient the machine code could be made, given the advances in compiler technology over the past 25 years.

[taufan99]

Speaking of which, I wonder if this source code can be used to further analyze and compare with the Sega Saturn port, as they are mostly identical.

[Quasar]

20 hours ago, intacowetrust said:

I often wonder if this was recompiled with a more modern toolchain how much more smaller & efficient the machine code could be made, given the advances in compiler technology over the past 25 years.

The bad state of the binary codegen was one of the things that continually put off both Kaiser and I from doing a more thorough project targeting the PSX version. I am certain a modern compiler and linker could do absolute wonders and still result in more readable assembly :>

[fenderc01]

@Erick194 @intacowetrust I'm curious what the differences are between the original version and the greatest hits version.

[Erick194]

@fenderc01 The differences are minimal in a way, the Greatest Hits version updates version 1.0 a little, some examples are the skull keys, in 1.0 it only shows the normal cards, also the password was not kept in version 1.0 which was deleted when accessing the next level, right now I do not remember more, @intacowetrust

could give a little more detail about it.

[intacowetrust]

5 hours ago, Erick194 said:

@fenderc01 The differences are minimal in a way, the Greatest Hits version updates version 1.0 a little, some examples are the skull keys, in 1.0 it only shows the normal cards, also the password was not kept in version 1.0 which was deleted when accessing the next level, right now I do not remember more, @intacowetrust

could give a little more detail about it.

 

Yep, those were two differences that were found. The 'keycard needed' flash was fixed in Greatest Hits so that it shows a skull key instead of a keycard if the map uses skull keys. And the game remembers your password in Greatest Hits if you want to restart the level with the password that was given to you upon entering the level.

 

In addition to those there were also the following changes:

• Stop any playing chainsaw sounds immediately upon pausing in the Greatest Hits edition, rather than waiting for them to end naturally
• Small tweaks to X position of status bar messages (move more to the right, by 5 pixels for Greatest Hits)
• Point the game automatically at Doom 2 in Greatest Hits after finishing Doom 1, when returning to the main menu
• Passwords: ammo and health amounts are validated in the Greatest Hits edition (must be in a valid range for the password to be accepted)

Those are the main ones if I recall correctly. I think there were some other very minor code differences, but nothing that would result in any observable difference in the game.

[fenderc01]

@intacowetrust Is the original release the one with the GT Interactive logo on the title screen? I’ve always played the version without the logo and it has the changes you’ve listed with the greatest hits version.

[Gez]

8 hours ago, intacowetrust said:

Passwords: ammo and health amounts are validated in the Greatest Hits edition (must be in a valid range for the password to be accepted)

So you could make yourself a password with, say, 900 health/armor/bullets/shells/rockets/cells, and before Greatest Hits it'd work? Pretty clever way to cheat back then.

[intacowetrust]

10 hours ago, fenderc01 said:

@intacowetrust Is the original release the one with the GT Interactive logo on the title screen? I’ve always played the version without the logo and it has the changes you’ve listed with the greatest hits version.

 

The GT logo is specific to the PAL version I believe. See: https://tcrf.net/Doom_(PlayStation)#Regional_Differences

 

9 hours ago, Gez said:

So you could make yourself a password with, say, 900 health/armor/bullets/shells/rockets/cells, and before Greatest Hits it'd work? Pretty clever way to cheat back then.

 

In theory, if you knew how to encode passwords. Not sure at what point that system become known/cracked.