PsyDoom 1.1.1 - PSX Doom port (reverse engineered) for PC

[intacowetrust]

Time for another new build - 0.2.0!

https://github.com/BodbDearg/PsyDoom/releases/tag/releases%2F0.2.0

 

Here is the list of changes in the new version:

• The music sequencer and sound system is now entirely converted to native C++.
• The game can now load .LCD (sound sample) files using the current modding/overrides mechanism. This makes it possible to play new PSX maps with sound, such as those found in "[GEC] Master Edition PSX Doom".
• The game can play now custom maps without having to provide 'MAPSPR--.IMG' and 'MAPTEX--.IMG' files. If these files are not present, the required lumps are loaded from PSXDOOM.WAD instead.
• The music in Club Doom now loops correctly.
• Fixed the first 2 seconds of CD audio tracks being skipped.
• Fix for pause/unpause of cd audio not resuming playback from the previous position.
• Automap: fix a PSX DOOM bug where lines flagged with ML_DONTDRAW would draw when the computer map powerup is obtained.

I'm now done with audio and turning my attention finally to gameplay (the last category of code to be made native), since rendering, audio, ui, base engine and most low level PsyQ stuff is now all done.

 

First immediate goals (aside from converting the gameplay code itself) will likely be to figure out how to record more demos against the original binary, for verification. If I get a lot of demos gathered also it might be nice to automate the demo verification and run the game in headless mode.

 

Another task is to try and figure out the multiplayer stuff, and get that running over local tcp/ip. By doing that also I can implement the last few remaining PsyQ SDK functions (mostly link cable related) and finally remove the BIOS dependency once and for all. I'm VERY close to being able to run without a bios - I did a few quick temp hacks and the game was able to get by just fine without it.

 

Will post up more new builds as interesting stuff is added.

[intacowetrust]

Managed to get some demo recordings against the original game out of Avocado - turned out not too difficult in the end. Did a playthrough of map 01 and 02 and tested the resulting demos in PsyDoom. So far it's all looking good - no demo desync! My plan is to build up a library of these demos for every map in the game, that should build confidence in the accuracy of the port. Here is the patch/hack I made for Avocado as well for those who are interested in how it was done:

 

https://github.com/BodbDearg/PsyDoom/blob/master/docs/avocado_demo_record_hack.patch

 

And now that I have control the music sequencer, here's something I did just for fun to hear a fresh twist on the old tracks:

 

 

[seed]

This is getting pretty insane now. It's all the more fantastic if demos do not desync, at least from the testing that has been made thus far.

 

We're getting close to the "Choclate PSX Doom" goal :D .

[CoTeCiO]

HO LY SHIT!!!!! I'm loving this!! This project, at this point, has already surpassed the expectations I had when I first heard of it, and it's about to get even better. You are an insanely talented programmer! I was chatting with a friend the other day and I commented to him about this and we're definitely trying that multiplayer feature once you get it working and whenever the quarantine is over.

 

I got one question at this point though. Do you plan on implementing options regarding the video output? Like resolution and aspect ratio? I'm a sucker for playing stuff in their native resolution and I do that with Phoenix Doom, and I'd like to do that with this one as well. I know the aspect ratio of this port is a bit weird in that the console output stretches it horizontally, and if I remember correctly, this port ran in the "real", non-stretched aspect ratio at first and then you switched it to the output aspect ratio.

 

And another question, do you plan on porting or working on something else after this is done?

[intacowetrust]

On 4/14/2020 at 8:07 PM, CoTeCiO said:

HO LY SHIT!!!!! I'm loving this!! This project, at this point, has already surpassed the expectations I had when I first heard of it, and it's about to get even better. You are an insanely talented programmer! I was chatting with a friend the other day and I commented to him about this and we're definitely trying that multiplayer feature once you get it working and whenever the quarantine is over.

 

Thanks! Because that part of the game is currently off limits - I'm very much motivated to get it working again ;-) It's not going to be a very lag tolerant thing at all and will basically be synchronous multiplayer like the original DOOM. That means it'll just be sending across inputs, and no prediction etc. - which is basically all the link cable did. Hopefully though that should be OK over a LAN at least...

 

On 4/14/2020 at 8:07 PM, CoTeCiO said:

I got one question at this point though. Do you plan on implementing options regarding the video output? Like resolution and aspect ratio?

 

Yes indeed, eventually I will add those at a later stage. At the very least I'm going to make the scaling/stretching configurable. I'm considering (slightly) higher resolution support in this port (like Crispy Doom) as well while using the PSX rasterizer, but that mainly depends on how messy it gets. If there's too many code changes then I might keep original PsyDoom to vanilla resolution and do the high res stuff in a fork with a rewritten renderer.

 

On 4/14/2020 at 8:07 PM, CoTeCiO said:

And another question, do you plan on porting or working on something else after this is done?

 

Not sure about that - I'll probably be iterating on this for a while at least. My ambition for all this is to make some new maps and music with the PSX engine, and allow for fun stuff that wasn't possible on the original hardware due to memory constraints. That means some tooling (for music) and limit removing work.

[Lollie]

Quick question, from your video description:

Quote

This is made possible because PSX DOOM uses a tracker/sequencer internally and just a few sound samples to produce most of it's in-game music. Eventually the hope is to create tools so that new songs for the game (to accompany new maps) will be possible.

This is more of a far-future thought, but would it be worth looking into a tracker library like Libxmp-lite? Or maybe a tool for converting tracker modules to PSXDoom's music format?

[intacowetrust]

1 hour ago, Lollie said:

This is more of a far-future thought, but would it be worth looking into a tracker library like Libxmp-lite? Or maybe a tool for converting tracker modules to PSXDoom's music format?

 

Had a quick look at that library and it looks promising - thanks for the suggestion! Something like that would definitely help with handling the specifics of known tracker formats like .xm and .mod etc. I don't think any tool would exist to convert to the PSX DOOM formats though unless someone has cooked up something already based on Erick's source release. That's ok though - I'm familiar with the sequencer now so hopefully should be able come up with a tool given a little time.

[Erick194]

18 hours ago, intacowetrust said:

Had a quick look at that library and it looks promising - thanks for the suggestion! Something like that would definitely help with handling the specifics of known tracker formats like .xm and .mod etc. I don't think any tool would exist to convert to the PSX DOOM formats though unless someone has cooked up something already based on Erick's source release. That's ok though - I'm familiar with the sequencer now so hopefully should be able come up with a tool given a little time.

 

I have in mind to create a program like Williams did to package and unpack the WMD (both for PSX and N64), and create a MIDI to WSEQ converter ("Williams Sequences"), but you have to take into account the ADSR, both in PSXDOOM and DOOM 64, Aubrey Hodges wrote the midis and the WESS system does play all the notes but the hardware makes them sound different, it would be great to create an independent player with the PSYDOOM for those who want to create new music and verify the result .

[Lollie]

2 hours ago, Erick194 said:

I have in mind to create a program like Williams did to package and unpack the WMD (both for PSX and N64), and create a MIDI to WSEQ converter ("Williams Sequences"), but you have to take into account the ADSR [...] it would be great to create an independent player with the PSYDOOM for those who want to create new music and verify the result .

 

So, one of my bigger concerns about this approach... I use tracker programs to make music. When making sequenced music, it's pretty normal to constantly listen to the parts you're working on over and over. It's important to be able to hear what you make as you make it, and know that the end result will sound the way you expect (or at least, very similar). If you have to use an independent player to hear what you're working on, it can slow down the music-making process by a lot.

 

This is a big reason why I would suggest a converter for tracker modules like the XM or IT formats. Tracker modules contain sound files that are used for instruments, and XM & IT modules support instruments with volume envelopes — they can be more complex than ADSR, but it IS possible to create ADSR-like envelopes. Basically, trackers already make it easier/faster to hear what sequenced music should sound like.

 

That said, tracker programs like OpenMPT can export to MIDI, and there are tools for making music with MIDI Soundfonts... So my concerns might just be me being overly-cautious (+ me wanting the music-making process to be as simple as possible, lol). You would know better about PSXDoom's limitations for its music and audio, and what it can & can't support.

[intacowetrust]

5 hours ago, Erick194 said:

I have in mind to create a program like Williams did to package and unpack the WMD (both for PSX and N64), and create a MIDI to WSEQ converter ("Williams Sequences"), but you have to take into account the ADSR, both in PSXDOOM and DOOM 64, Aubrey Hodges wrote the midis and the WESS system does play all the notes but the hardware makes them sound different, it would be great to create an independent player with the PSYDOOM for those who want to create new music and verify the result .

 

Yeah I was thinking of eventually doing a pack/unpack utility for the module (.WMD) file also - you'd certainly need that to add new patches and sequences. Your point is valid, a tracker might not be able to reproduce the sound exactly as it would be on the hardware, especially with the hardware reverb that the SPU provides.

 

Another approach might be to create tools to edit the module file and the patches/instruments contained within, as well as importing music sequences from midi like you suggested. A specialized VST plugin could also be created which uses Avocado to emulate the PSX SPU and which exposes the instruments defined in the Williams module file to a music sequencer program. Once you had that you could use a DAW application of your choice (like FL Studio etc.) to compose the music and test against the actual instruments as they are in the game, and hear pretty similar results. Once you are happy with that then your sequence can be exported to midi and ultimately back into the module file and played back in game. A more complicated and fragmented workflow than the tracker approach perhaps, but potentially more powerful and accurate?

 

This might be an interesting little project for me actually... Many years ago I was very much into making electronic music, and always thought it would be cool to create my own VST instruments. At the time though I was only just beginning my programming career and learning Visual Basic 6 (shudder!) so such a thing was far beyond my abilities.

 

I'm really curious though does anyone know anything about the process or workflow that @Aubrey Hodges used for creating music in the game? Has such a question been raised to him and answered before? Not necessarily saying it's what we should do now (VST instruments did not exist back in the early 90s), but perhaps it might give some insight into how to get into the 'spirit' of making music for this game. Curious how it was done...

[Redneckerz]

@intacowetrust i have made a page for PsiDoom.

 

;)

 

Is there a settlement on the logo?

[Impboy4]

 

On 4/18/2020 at 2:44 PM, Redneckerz said:

i have made a page for PsiDoom.

Ps"Y" not Ps"I" dude.

[intacowetrust]

On 4/18/2020 at 11:44 AM, Redneckerz said:

@intacowetrust i have made a page for PsiDoom.

 

;)

 

Is there a settlement on the logo?

 

Haven't got a final one yet - have mainly been focusing on getting the fundamentals of the project done first before polishing up stuff like that. However if you want to go ahead and use the concept that I made earlier then that would work for now. When it changes and is polished we can always update later. It's in SVG format so you can export to whatever size you need: https://github.com/BodbDearg/PsyDoom/blob/master/docs/PsyDoom Concept Logo.svg
 

InkScape can be used to convert to PNG. You need the 'ZRNIC' font installed to render correctly though: https://www.dafont.com/zrnic.font

 

55 minutes ago, Impboy4 said:

Ps"Y" not Ps"I" dude.

 

@Redneckerz was just mentioning in jest, since PsiDoom (the Amiga port) came up earlier during the naming discussion and also because said he would do a wiki entry if I provided him with a logo :P

[Redneckerz]

2 hours ago, Impboy4 said:

 

Ps"Y" not Ps"I" dude.

Was there any reason to react like that given the clear nature of the post?

[Impboy4]

12 hours ago, Redneckerz said:

Was there any reason to react like that given the clear nature of the post?

[intacowetrust]

Good news everyone, the demo compatibility of PsyDoom with the original game is looking pretty solid at this point! I recorded playthroughs of all 59 maps using the original game in Avocado - every single demo played back correctly in PsyDoom. Even bugs such as the lost souls occasionally slipping through walls were replicated correctly :P

 

I now also have saved the high level expected results for each demo, which can be used in automated testing. To leverage this, I added a 'headless' mode to the game so it can run a demo without any display or sound and verify the result against what is expected. A small python script I created also does a playthrough of the entire game and verifies everything behaves as expected. The entire playthrough completes in about 8-9 seconds, which is awesome and means I can run it very frequently to sanity check everything.

[fenderc01]

When running the latest dev build, I get this error after letting the title screen run and right before the first demo plays:

 

[intacowetrust]

11 hours ago, fenderc01 said:

When running the latest dev build, I get this error after letting the title screen run and right before the first demo plays:

 

 

Thanks for checking out the build and reporting this issue!

 

Looks like a problem introduced while adding support for demos larger than the previous 16 KiB limit. I allocate the demos now outside of the Avocado PlayStation's 2 MiB of RAM (to allow for very long demos and not exhaust memory) but some low level CD-ROM handling code was still expecting the demo buffer to reside in the PlayStation's memory. I should have a fix for the issue shortly.

[intacowetrust]

@fenderc01 just added a fix for the issue you encountered in commit 7956f33256a39db905cae5032a1ccc0ce1f42a60. Should work now if you pull the latest code.

[intacowetrust]

Hey everyone just to let you know that I've put a new build up - 0.3.0:

 

https://github.com/BodbDearg/PsyDoom/releases/tag/releases%2F0.3.0

 

New in this version:

• PsyDoom no longer needs a PlayStation BIOS file to run.
• PsyDoom now runs 100% natively and does not emulate the MIPS cpu at all.
• Added support for deathmatch and co-op play, with an emulation of the original link cable functionality over TCP. See the main README.md file for instructions on how to use.
• Verified demo compatibility with the original PSX Doom for a full playthrough of the game. Demos recorded against the original binary play back the same in PsyDoom.
• Password system: nightmare mode now has it's own unique set of passwords.
• GTE (geometry transformation engine) code is now implemented using regular C++, no emulation used. This removes dependencies on the Avocado MIPS CPU.
• Reduced CPU usage if waiting during the framerate cap.
• Some frame pacing and input lag improvements.

 

Here is a short video of the new 'link cable' re-implementation in action:

 

For instructions on using the multiplayer functionality, see:

https://github.com/BodbDearg/PsyDoom/blob/master/README.md

 

Please note that it requires a very low latency connection in order to be playable; for best results Ethernet is preferable.

 

Lastly with this update one very minor bug is introduced; you may notice some seldom flicker or jitter of some pixels depending on the player's viewpoint. Unfortunately an update of Avocado introduced this minor precision issue in the rasterizer. I've already flagged the problem with the author and hopefully there will be a fix for that eventually.

 

That's all for this version. Getting rid of the BIOS dependency has now been achieved and that is a very big milestone passed for the project! Next up will be removing the dependency on the original .EXE so that PsyDoom can truly be it's own standalone replacement for the PlayStation executable. Once that is done I will be able to focus on nice stuff like controls, configuration and other quality of life improvements... Much work to be done still, but the project is definitely starting to take shape! :)

[taufan99]

Congratulations! I can see some PSX DOOM multiplayer tournament being held in a nearby future.

[Dark Pulse]

44 minutes ago, intacowetrust said:

Hey everyone just to let you know that I've put a new build up - 0.3.0:

 

https://github.com/BodbDearg/PsyDoom/releases/tag/releases%2F0.3.0

 

New in this version:

• PsyDoom no longer needs a PlayStation BIOS file to run.
• PsyDoom now runs 100% natively and does not emulate the MIPS cpu at all.
• Added support for deathmatch and co-op play, with an emulation of the original link cable functionality over TCP. See the main README.md file for instructions on how to use.
• Verified demo compatibility with the original PSX Doom for a full playthrough of the game. Demos recorded against the original binary play back the same in PsyDoom.
• Password system: nightmare mode now has it's own unique set of passwords.
• GTE (geometry transformation engine) code is now implemented using regular C++, no emulation used. This removes dependencies on the Avocado MIPS CPU.
• Reduced CPU usage if waiting during the framerate cap.
• Some frame pacing and input lag improvements.

 

Here is a short video of the new 'link cable' re-implementation in action:

 

For instructions on using the multiplayer functionality, see:

https://github.com/BodbDearg/PsyDoom/blob/master/README.md

 

Please note that it requires a very low latency connection in order to be playable; for best results Ethernet is preferable.

 

Lastly with this update one very minor bug is introduced; you may notice some seldom flicker or jitter of some pixels depending on the player's viewpoint. Unfortunately an update of Avocado introduced this minor precision issue in the rasterizer. I've already flagged the problem with the author and hopefully there will be a fix for that eventually.

 

That's all for this version. Getting rid of the BIOS dependency has now been achieved and that is a very big milestone passed for the project! Next up will be removing the dependency on the original .EXE so that PsyDoom can truly be it's own standalone replacement for the PlayStation executable. Once that is done I will be able to focus on nice stuff like controls, configuration and other quality of life improvements... Much work to be done still, but the project is definitely starting to take shape! :)

For some reason I have a strange desire to eat tacos.

 

Alas... tomorrow is Monday.

 

But Tuesday... DEFINITELY!

 

Would be great to make use of this for some multiplayer testing for the GEC project. Maybe this way @Jimmy and/or @Dragonfly can play the project eventually, too. A little more exposure never hurts, and PSX Doom definitely FEELS different from the usual Doom. :)

 

Maybe Jimmy could even try his hand at composing in the Aubrey Hodges style... wouldn't that be a treat?

[seed]

This is stellar news, the project is shaping up to be better and better 👊.

[Redneckerz]

10 hours ago, intacowetrust said:

Hey everyone just to let you know that I've put a new build up - 0.3.0:

 

https://github.com/BodbDearg/PsyDoom/releases/tag/releases%2F0.3.0

 

New in this version:

• PsyDoom no longer needs a PlayStation BIOS file to run.
• PsyDoom now runs 100% natively and does not emulate the MIPS cpu at all.
• Added support for deathmatch and co-op play, with an emulation of the original link cable functionality over TCP. See the main README.md file for instructions on how to use.
• Verified demo compatibility with the original PSX Doom for a full playthrough of the game. Demos recorded against the original binary play back the same in PsyDoom.
• Password system: nightmare mode now has it's own unique set of passwords.
• GTE (geometry transformation engine) code is now implemented using regular C++, no emulation used. This removes dependencies on the Avocado MIPS CPU.
• Reduced CPU usage if waiting during the framerate cap.
• Some frame pacing and input lag improvements.

 

These are the real important bits :P So, is all rendering related code moved to software rendering or are you using OpenGL?

Also, could you provide a screenshot of the game demonstrating it being run in native mode showing off things like the colored lighting? It is for Realm 667.

It now feels right to write some more about this. Hugely impressive work, the PSX renderer ported to Windows is a great achievement indeed.

Edited June 1, 2020 by Redneckerz

[Lollie]

Lord, this port has progressed much faster than I expected. Awesome work removing BIOS and MIPS dependencies, excited to see PSXDOOM.exe follow suit.

 

Multiplayer support is wild! Even if it's currently only suitable for LAN, it's very cool to see here. Do you see PsyDoom's multiplayer being expanded upon in the future, to make it more lag-tolerant? (assuming PSX/PsyDoom could support it - I was thinking something in the vein of GGPO's rollback netcode, which relies on saved game states and input prediction. https://www.ggpo.net/ Obviously in a simpler form though lol)

 

The only visible issue I was going to point towards was the pixel flicker that you already mentioned, always a good sign. Also nightmare mode having passwords is a genuine mercy lol, thank you.

Edited June 2, 2020 by Lollie

[Dragonfly]

16 hours ago, Dark Pulse said:

Maybe this way @Jimmy and/or @Dragonfly can play the project eventually, too.

 

We could consider it. :)

[intacowetrust]

21 hours ago, Dark Pulse said:

Would be great to make use of this for some multiplayer testing for the GEC project.

 

Absolutely! As it stands right now PsyDoom can be used to test the original PSX Doom format maps in deathmatch/co-op mode, using the file overrides mechanism:

 

 

Hoping to add support for the Final Doom map format soon (as a prelude to supporting FD fully), so hopefully that will open up the entire mapset for testing in PsyDoom.

 

19 hours ago, Redneckerz said:

These are the real important bits :P So, is all rendering related code moved to software rendering or are you using OpenGL?

 

It's entirely software rendering right now, OpenGL is only used for blitting and scaling the finished frame onto the window via SDL.

 

I'm using Avocado's GPU implementation to do all the rasterization of the actual draw primitives. It's basically just being used as a glorified triangle renderer at this point though, not as a full device within an emulated PlayStation. For example I no longer use interrupts at all or DMA transfers to fill VRAM. The layer between the application and triangle rendering functions is now extremely thin and I have a few more changes to make to make it even thinner for better performance.

 

20 hours ago, Redneckerz said:

Also, could you provide a screenshot of the game demonstrating it being run in native mode showing off things like the colored lighting? It is for Realm 667

 

Sure thing, is there any particular scene you would like to see? Here is a screenshot that I like of the Underhalls, at 3x native resolution:

 

 

 

18 hours ago, Lollie said:

Multiplayer support is wild! Even if it's currently only suitable for LAN, it's very cool to see here. Do you see PsyDoom's multiplayer being expanded upon in the future, to make it more lag-tolerant? (assuming PSX/PsyDoom could support it - I was thinking something in the vein of GGPO's rollback netcode, which relies on saved game states and input prediction. https://www.ggpo.net/ Obviously in a simpler form though lol)

 

 

It's possible - depends on if there is enough of a demand for it :) It's a huge task but I like the sound the approach from GGPO, kinda reminds me of a similar thing I saw about Mortal Kombat's networking in a GDC presentation:

 

 

[Dark Pulse]

Well, it could well be that there's possibly two sets of netcode. I mean, while having the original system-linked netcode is great and all, functionally speaking it's probably better to have something a bit more tolerant for actual online play.

 

I remember someone asking in the GEC channel about how it'd be neat to take the PS1 link cable and turn the other end into an ethernet cable to hook into a modem, and I think I kind of disappointed them when I told them that it's all serial communication and not likely to be latency-resistant at all - bit of a difference between two units within a very short distance of another and the possibility of literally hundreds/thousands of miles of separation :P

[Dark Pulse]

@intacowetrust Something that came to mind when I remembered it...

 

Can you explain why large, spanning floors will occasionally introduce some breakup in the floors? Can anything be done about that to improve the rendering on the PSX?

 

You should be able to test it using my GEC Master Edition conversion of E3M8: Dis, but it's noticeable in other maps too (I just know that since most of the work was done on the Final Doom engine, PsyDoom can't play those maps yet).

 

You should be able to see what I mean in this video someone recorded of the level, but I can't be 100% sure if it's an emulation bug, or something wrong with the PSX Doom code. It's up to you to find out!

 

 

Edited June 4, 2020 by Dark Pulse

[intacowetrust]

7 hours ago, Dark Pulse said:

Can you explain why large, spanning floors will occasionally introduce some breakup in the floors? Can anything be done about that to improve the rendering on the PSX?

 

You should be able to test it using my GEC Master Edition conversion of E3M8: Dis, but it's noticeable in other maps too (I just know that since most of the work was done on the Final Doom engine, PsyDoom can't play those maps yet).

 

You should be able to see what I mean in this video someone recorded of the level, but I can't be 100% sure if it's an emulation bug, or something wrong with the PSX Doom code. It's up to you to find out!

 

Challenge accepted! :P

 

PsyDoom also had the same artifact (it has pretty much all the OG rendering artifacts) but I hadn't spent time studying it until now. Here's a fix that I made after investigating the issue:

 

@Erick194 here's the changelist for it if you're interested:

https://github.com/BodbDearg/PsyDoom/commit/2a0250ca6b43d920706849d0d9018cc4656cb1f1

 

One tweak that could be made is maybe unrolling the last loop iteration so that the if() check isn't done inside of the loop each time. Not sure if it's worth the trouble though... I did it this way since it's cleaner and modern compilers might decide do make optimization anyway if it's advantageous.