"Runworthiness" of the linuxdoom code?

[Maes]

It may sound like a rhethorical question, but other than the usual unfixed Doom bugs, how "ready to run" is the linuxdoom 1.10 source code considered to be, seeing how it's used as a base for many "proof of concept" Doom ports?

OK, I never hid the fact that I used it as a base for Mocha Doom (which might have been a bad decision after all, and in fact I'm trying to move away from it), but I encountered many parts of the code that are, well, too broken to really make a viable source port out of it by just throwing a video/sound library at it and slapping it on a generic C compiler (or cross compiler to Javascript/Actionscript, as the recent trend seems to be).

Some of the stuff I needed to fix by hand once I realized they were really broken:

• Proper IWAD support for anything other than Shareware, non-ultimate Doom (registered) and Doom II. There are literally dozens of places where TNT/Plutonia and even Ultimate (retail) Doom quirks are not properly taken into account. Perhaps one of the weirdest was that Episode 4 switches didn't animate because the initializer didn't discriminate between "retail" and "registered", while the end texts etc. are utterly broken. Also, TNT/Plutonia are not treated uniformly as "commercial" where appropriate, with proper support being left as somewhat of an "exercise for the reader".
  
• Sound, but that's a no-brainer. Still, I discovered a few things that would be broken even with a fully functional sound driver added (e.g. sound handle numbers start out from 0, increase to 100, but never roll back to 0 from 100, there's only code to roll back 0 to 100 :-S
  
• Gamma settings/palette code also seems like an "exercise for the reader" to guesstimate how to properly implement.
  

There are probably others, but you get the drift. Is there any sort of "fixed linuxdoom" out there which is actually "almost" ready to use/plug in/convert with those outstanding issues fixed?

[chungy]

I don't think the LinuxDoom code can actually be run as-is anymore, but I haven't tried in a while. In general, yes, it's a mess of sloppy code.

Maes said:

Is there any sort of "fixed linuxdoom" out there which is actually "almost" ready to use/plug in/convert with those outstanding issues fixed?

Shame on you.

[Maes]

chungy said:

Chocolate Doom

Close but no cigar. Chocolate Doom actually carries a lot of other enhancements, advanced source port functionality and comes prepackaged and pre-bound to specific libraries, so it doesn't count. Also, many of those "proof of concept" projects are based off linuxdoom, not Chocolate Doom. So...nope, that's not the answer, sorreeeee :-p

Which implies that, unless their authors went into the pain of fixing the aforementioned bits, those ports will never be usable with anything but the Shareware IWAD and maybe 3-episode Doom and standard Doom II (and in fact, many are hardwired in such a way that you can't really put this to the test).

[fraggle]

Maes said:

Close but no cigar. Chocolate Doom actually carries a lot of other enhancements, advanced source port functionality and comes prepackaged and pre-bound to specific libraries, so it doesn't count. Also, many of those "proof of concept" projects are based off linuxdoom, not Chocolate Doom. So...nope, that's not the answer, sorreeeee :-p

Well, chungy is right really, it's sort of what the point of Chocolate Doom was ... originally, at least. Unfortunately it's grown quite a bit since then :-)

I still think though that if you're starting a new port and want something basic to build on, Chocolate Doom may be the best option. Maybe not the source "as-is", but it's probably easier to strip out the bits you don't want than it is to fix up the linuxdoom source. Plus, the svn history goes all the way back to the start, so if you wanted you could cherry pick just the bits you want.

EDIT: With some quick shell script hackery, I put this together. A chronological list of all the commits that changed the original source files. I left out the i_* files as they've been pretty much completely replaced. You'll also find references to some non-original files occasionally where a commit changed an original file. However, this should be a useful list if you ever want to cherry pick all the compatibility fixes.

[SaladBadger]

I once got the Linuxdoom code to compile on a Ubuntu 8.04 virtual machine one time, with a limited amount of difficulty (ie: things I could fix with my limited programming experience :P), and it would even (start to) run but then stop when trying to start the graphics stuff complaining that X needs to be in 256 colors mode, and I'm not familiar at working with X at all, so I just kinda stopped there.

[Maes]

Anyway, my initial point was that even if you rig it just so it starts -and nothing more than than that-, you will soon run into problems with IWADs and other general brokenness. So I'm surprised it's still used as a base for proof-of-concept ports (see my comments above about Mocha Doom...I thought it would be educational to start from the very root, that's why I didn't jump on Boom or even Chocolate right from the start).

To take some recent examples, the "Flash Doom", "Javascript Doom" and (I think) even the recent TI Doom (the ARM one, not the TI-99 one) were based off linuxdoom. Which means that if you throw Plutonia or TNT at them, or even Ultimate Doom, they will very soon crap out, if their authors didn't fix certain specific bits.

In Mocha's case, I did that by trial and error and added a deeper discrimination between Ultimate/Non-Ultimate and Doom 2/TNT/Plutonia/XBLA.

[hex11]

There's always the original SDL Doom port:
http://www.libsdl.org/projects/doom/

It compiles easily. Sound volume and gamma correction work. All the official IWADs are in d_main.c. There's no music though. It also runs in a 320x200 window, unless you use -2 or -3.

I was able to play the v1.9 shareware doom1.wad, but it doesn't like the Freedoom IWAD:
Error: P_InitPicAnims: bad cycle from BFALL1 to BFALL4

[hex11]

InsanityBringer said:

X needs to be in 256 colors mode, and I'm not familiar at working with X at all, so I just kinda stopped there.

startx -- -bpp 8

Well, the -bpp is deprecated and it's now -depth, but both should work. man Xorg will show the gory details. Oh, it was a lot of fun getting these games to run back in the day, using scripts to start a second X server at 8 bpp just to run that one client. :) Well there was always the svgalib version (but not for Quake).

[Csonicgo]

hex11 said:

startx -- -bpp 8

Well, the -bpp is deprecated and it's now -depth, but both should work. man Xorg will show the gory details. Oh, it was a lot of fun getting these games to run back in the day, using scripts to start a second X server at 8 bpp just to run that one client. :) Well there was always the svgalib version (but not for Quake).

IN UBUNTU AND DEBIAN THIS WILL NOT WORK.

Sorry for the caps but this has been an issue with Linux "Tech-support forums" for a long time. People don't run root anymore (they should NEVER EVER RUN ROOT at ALL) and startx REQUIRES ROOT in these distros to run. You'll have to do some xorg.conf trickery and even in Ubuntu 8.04 the xorg conf is pretty much uneditable, it's all script driven.

Just had to clear that up before we void someone's distro trying to get X to work.

[chungy]

Uh. startx doesn't require root, not in Debian anyway (and I seriously doubt Ubuntu changed anything in this regard).

[Csonicgo]

chungy said:

Uh. startx doesn't require root, not in Debian anyway (and I seriously doubt Ubuntu changed anything in this regard).

Well I know it won't work in Ubuntu 8.04 and in later releases Xorg.conf was so well hidden and useless that I couldn't add another screen if I wanted to.

That nonsense has to stop, by the way. I love auto-configuration but hiding the config from me is not very nice. >:(

[chungy]

I don't know about Ubuntu but it should always be at /etc/X11/xorg.conf like pretty much every distro in the world. :P

[Csonicgo]

chungy said:

I don't know about Ubuntu but it should always be at /etc/X11/xorg.conf like pretty much every distro in the world. :P

It's there, but it's not read since 8.04, I know, unless forced.

[Maes]

hex11 said:

Freedoom IWAD:

That cycle only exists in TNT/Plutonia but I had to take no special measures with Mochadoom: it was already hardcoded in the non-SDL linuxdoom code, so it's surpising that SDL Doom doesn't include it. However the Freedoom IWAD has a major brokenness in that it's not vanilla compatible (requires Boom) so it's not really a good ground for comparison.

hex11 said:

All the official IWADs are in d_main.c

So they are in linuxdoom, too, but scattered along the whole codebase there are countless spots where they will fail: e.g. missing HELP2 lump for Doom II, Ultimate Doom's Episode 4 doesn't get animated switches in P_InitSwitchAnims (or somesuch) because of a broken check etc.

And the f_finale end and intermission texts for anything other than Shareware/Doom (non-ultimate)/Doom II are just plain broken and there's no way they will function without reimplementing them (What I call "left as an exercise for the reader").

[hex11]

You must be thinking of something else man. startx is just a wrapper shell script that invokes the suid-root X server binary for you:

$ ls -l /usr/X11R6/bin/X{,org}
lrwxr-xr-x 1 root wheel 4 Nov 5 2010 /usr/X11R6/bin/X@ -> Xorg
-rwsr-xr-x 1 root wheel 1719388 Mar 15 2010 /usr/X11R6/bin/Xorg*

Anybody who runs that is going to run it as root. Without root privs, there is no X11, plain and simple. OpenBSD tries to limit the damage a bit, using privilege separation (you end up with two Xorg processes, a small one @ UID 0, and the other main one @ your UID). That's a fairly common strategy in many other system daemons also, all to limit the amount of code that's running as UID 0.

This explains it a bit, if you're interested:
http://en.wikipedia.org/wiki/OpenBSD_security_features#X11

[Gez]

Why not use Boom 2.02 as the baseline, instead of LinuxDoom? Sure you'll have some bugs to fix there too; but at least you'll have a mostly-working codebase with support for standard advanced modding features.

[Maes]

Gez said:

Why not use Boom 2.02 as the baseline, instead of LinuxDoom?

As I said, it was a design decision which I had to weigh carefully, and it was not easy to take. In the end I went with linuxdoom with the rationale that I was doing foundation/groundwork while getting myself acquainted with the codebase, so I needed to keep close to the basics.

The result, is a mixture of linuxdoom, Boom, Eternity and original elements which will hopefully spawn its own development branches. However the next incremental milestone will surely be copying Boom functionality (or even prBoom+, now that the pesky aspect of actually producing a viable non-C mapping of the Doom source code is out of the way).

[Csonicgo]

hex11 said:

Without root privs, there is no X11

And still that is bad design! Theoretically I could exploit X11 somehow with some crazy x11 acceleration commands from a program and take over the system through registers if I wanted....doesn't that bother people? Not even WINDOWS does this!

[hex11]

Maes said:

That cycle only exists in TNT/Plutonia but I had to take no special measures with Mochadoom: it was already hardcoded in the non-SDL linuxdoom code, so it's surpising that SDL Doom doesn't include it. However the Freedoom IWAD has a major brokenness in that it's not vanilla compatible (requires Boom) so it's not really a good ground for comparison.

Well the weird thing is that if you rename freedm.wad to doom2.wad, you still get the same result (yet FreeDM is meant to be vanilla-compatible).

And, sdldoom will hapilly use tnt.wad as its IWAD (you can even run Doom II PWADs that way). The intermission screens and end-of-episode texts seem okay. I don't know about the animated switches you're talking about though.

Btw, if you decide to try this, be aware of one limitation: there is no way to specify the IWAD on the command line. It'll pretty much use the first one it finds in its list. This is exactly the behavior that linuxdoom had, so I used to keep all IWADs in their own separate subdirectory.

There's a pretty detailed Changelog file in the source tarball, with explanations. Maybe that will shed some light as to what the major changes are between linuxdoom and sdldoom.

[hex11]

Csonicgo said:

And still that is bad design! Theoretically I could exploit X11 somehow with some crazy x11 acceleration commands from a program and take over the system through registers if I wanted....doesn't that bother people? Not even WINDOWS does this!

Well, it seems to bother Theo de Raadt quite a bit:
http://marc.info/?l=openbsd-misc&m=114738577123893&w=2

Now that message is from 2006, and I don't know if the current Xorg maintainers are trying to work on a solution or not (back in 2006, XFree86 was still widely used). I'm not holding my breath too much though...

One thing you can do (at least on OpenBSD) is to use the VESA X server, which allegedly doesn't require machdep.allowaperture set above 0. The downside is you won't benefit from your video card's accelerated features. I will have to try this and see just how different it feels. It might not even affect me at all, since I mostly run old games and no advanced Doom ports beyond prboom or odamex (both with software rendering).

On Linux or FreeBSD, you might be able to use the framebuffer console to run most of your games (anything SDL-based should just work once you set the environment variables right). I think there's even an X server that runs on top of the FB, which also probably doesn't require any root privs.

[Maes]

hex11 said:

Well the weird thing is that if you rename freedm.wad to doom2.wad, you still get the same result (yet FreeDM is meant to be vanilla-compatible).

I don't remember if the BFALLx cycle only exists in Plutonia and not TNT. If that's the case, renaming it won't do much good.

I don't know about the animated switches you're talking about though.

Well, in linuxdoom the P_InitSwitchList has this nasty bit left in:

//
// P_InitSwitchList
// Only called at game initialization.
//
void P_InitSwitchList(void)
{
    int		i;
    int		index;
    int		episode;
	
    episode = 1;

    if (gamemode == registered)
	episode = 2;
    else
	if ( gamemode == commercial )
	    episode = 3;

"episode" is used to initialize (or less) specific lists of switches.
A value of 1 means shareware only. A value of 2 means registered (but NOT retail) and 3 means Doom II switches.

The problem here is that ultimate Doom is identified as "retail", which is a proper superset of "registered", but is not checked for occurence here. In layman terms: if you are using the Ultimate Doom IWAD, the "registered" switches won't be initialized, and won't animate when flicked. Switches that belong to the shareware Doom are initialized in ANY case, even with Doom II IWADs.

In Mocha Doom, I fixed this manually by making the distinction between "retail" and "registered" less severe, so that "isRegistered()" is true for BOTH "retail" and "registered" cases (while isRetail() is only true for Ultimate Doom).

        public void InitSwitchList() {
            int i;
            int index;
            int episode;

            episode = 1;

            // MAES: if this isn't changed Ultimate Doom's switches
            // won't work visually. TODO: are there any episode 4-only switches?
            if (DM.isRegistered())
                episode = 2;
            else if (DM.isCommercial())
                episode = 3;

...


    /** Registered is a subset of Ultimate 
     * 
     * @return
     */

    public boolean isRegistered(){
    	return (gamemode== GameMode_t.registered || gamemode== GameMode_t.retail );
    }

Also, "isCommercial()" includes TNT/Plutonia (another distinction that is NOT accounted for in the linuxdoom code, and many Doom II-specific things just won't work unless you fix that broken gamemode/mission pack distinction.

My original question remains: how come many projects are still based on linuxdoom (and NOT SDLDoom or ChocoDoom) when it obviously can't work with most IWADs without extensive modifications to the whole codebase, even when it's obvious their authors didn't want to get their hands dirty with the code directly?

[fraggle]

Maes said:

My original question remains: how come many projects are still based on linuxdoom (and NOT SDLDoom or ChocoDoom) when it obviously can't work with most IWADs without extensive modifications to the whole codebase, even when it's obvious their authors didn't want to get their hands dirty with the code directly?

Most such projects are created by people who want to do "Doom on (x)" - "Flash Doom", "Javascript Doom", "PSP Doom" etc. They therefore do the logical thing of downloading "the Doom source" and adapting it to do what they want. Basically these things are usually being created by people who don't know anything about the Doom community or source ports.

[hex11]

Maes said:

I don't remember if the BFALLx cycle only exists in Plutonia and not TNT. If that's the case, renaming it won't do much good.

Ah, well sdldoom successfully loads every official IWAD. It only bombs with the BFALL error if you try to use Freedoom or FreeDM, whether it's named doom2.wad, plutonia.wad, or anything else...

The problem here is that ultimate Doom is identified as "retail", which is a proper superset of "registered", but is not checked for occurence here. In layman terms: if you are using the Ultimate Doom IWAD, the "registered" switches won't be initialized, and won't animate when flicked. Switches that belong to the shareware Doom are initialized in ANY case, even with Doom II IWADs.

It looks like sdldoom also has the same switch bug. I warped to -episode 4 and found that the two demon-head switches near the start don't animate when you active them. Maybe tnt & plutonia also have some issues, but I never cared enough about them to know what to look for...

[shadow1013]

well, I actually saw that linuxdoom was ruined by the guy who "cleaned it up" especially when it came to system dependant code, and furthermore, unfortunately, that guy (forget his name) screwed tons of things up in an attempt to (once again) "clean" the code

[natt]

shadow1013 said:

well, I actually saw that linuxdoom was ruined by the guy who "cleaned it up" especially when it came to system dependant code, and furthermore, unfortunately, that guy (forget his name) screwed tons of things up in an attempt to (once again) "clean" the code

Let's not be too harsh; after all, at the end of the day, we got the source code to Doom.

[shadow1013]

oh, and i think i can dig up an improved version of the linux doom that i wrote based on the original source code. but id have to make some changes as im embarrassed by my old code

[shadow1013]

@natt I guess thats true. But oh how I wish (just remembered his name) Bernd Kermier didn't screw so much stuff up

[_bruce_]

shadow1013 said:

@natt I guess thats true. But oh how I wish (just remembered his name) Bernd Kermier didn't screw so much stuff up

What things did he screw up(Kreimeier)?

[hex11]

This article talks about BK's involvement with the Doom source code:
http://www.doomworld.com/10years/ports/ports01_1.php

And that also means I was wrong about the sdldoom Changelog file... It only contains entries by bk@gamers.org, so it must the exact same file that shipped with the 1997 Doom src release. It also means SDL Doom doesn't have a proper changelog. :(

Anyway, in the article it says the Boom team was temporarily given access to the original DOS Doom source, so they could correct any discrepancies. But that also means it's probably not a good idea to use only the Linux Doom source release as a base if you want to avoid compatibility problems...

And man, that history of source ports reads like a real epic tragedy, with flamewars galore, trojan horses, abandoned codebases, etc.

[shadow1013]

ive looked at both the sdldoom and linuxxdoom sources and honestly i could never find a difference other than the system dependent code. which in usability terms means that sdldoom runs at all screen modes, because sdl i assume does the 8-bit to 24-bit (or whatever other color depth you have) in the library instead of the actual program. other than that theyre identical. and Bernd Kermier screwed up alot of things. like for example, intermission screen animation was broken by him.

EDIT: sdldoom was ported in one day. I think i read that somewhere...