DOS Doom Code Execution

[ketmar]

i am pretty sure that somebody on DW has a working time machine in their basement. wait a little more, and send this back in time, in 1996 or so, please. thank you!

Edited November 14, 2020 by ketmar

[zokum]

I love this thread. So much cool stuff is possible, but also a lot of really evil stuff can be done :) We'll have to run antivirus software on wads now :D

A wad running arbitrary code COULD modify other executables that you usually run outside Dosbox. Ideally run DB inside a 'jail' with only the bare essentials, no other executables than the ones you would run inside dosbox anyway. Perhaps give Dosbox permission to only read files, not write to them nor make new ones. This is not that hard to do on Linux-like OSes, might be a bit more complex for windows users as the idea of multiple users is not as ingrained.

Dosbox has support for mounting stuff imitating optical drives. Maybe this can be used to jail things efficiently? There wasn't much to find in the Dosbox docs.

There's been enough petty shit in the Doom community with backdoors, cheating, various attacks and general fuckery that I suggest a cautious approach to wad files to be run with doom2.exe inside Dosbox from now on.

[xttl]

On 11/13/2020 at 5:06 PM, ketmar said:

and send this back in time

 

I sure would love to do this if time travel was possible. :P Though as has been mentioned before in this and/or the other exe hacking thread, it might have led to less interest in the source code release which would have had some downsides in the long run.

 

Also, something like LE loader would have been really nice to have in the early Windows NT era (2k, XP) when real Doom didn't run well anymore (without dualbooting anyway which is always an annoying pain in the ass), yet DOSBox and Chocolate Doom didn't exist either or weren't as fully developed and usable as nowadays, and PrBoom's vanilla complevels & netplay in general had some limitations and bugs.

 

On 11/13/2020 at 7:22 PM, zokum said:

A wad running arbitrary code COULD modify other executables that you usually run outside Dosbox. Ideally run DB inside a 'jail' with only the bare essentials, no other executables than the ones you would run inside dosbox anyway

 

Ah yeah, this is true. Personally, I've always put all my DOS stuff in a separate "DOS root" directory, then kept that mounted as drive C: in DOSBox. I wonder how many people keep vanilla Doom executables in the same dir with their native ports then mount that dir in DOSBox? Lack of IWAD selection in vanilla (without hacks anyway) makes that approach worse I think.

 

On 11/13/2020 at 7:22 PM, zokum said:

Perhaps give Dosbox permission to only read files, not write to them nor make new ones. This is not that hard to do on Linux-like OSes, might be a bit more complex for windows users as the idea of multiple users is not as ingrained.

 

On Windows you could always just set the read only attribute for all files which will prevent all users and programs from modifying them. Alternatively, NT-based Windows OSes have always had proper user accounts and access control (unlike the DOS-based lineage including Win3.1, Win9x, etc), though the GUI seems to lack a way to run an executable as another user that is not the adminstrator account via right-clicking except from the start menu, at least on Server 2019 (Win10 v1809). Weird. So you need to go to the command line and use runas or create a shortcut which calls that program.

 

holding shift while right clicking shows "run as different user" anywhere, but that's probably convoluted enough to say it's better to use the command line or link to runas anyway :P

 

It seems DOSBox does some strange things when it doesn't have write permissions though. Typing eg. echo aa > bb in a protected directory shows the message on screen instead of failing. Trying to save a file using the MS-DOS 5 text editor says "file not found" after "saving" it. At least trying to overwrite an existing file fails with a more sensible "path/file access error" message

 

On 11/13/2020 at 7:22 PM, zokum said:

Dosbox has support for mounting stuff imitating optical drives. Maybe this can be used to jail things efficiently? There wasn't much to find in the Dosbox docs.

 

Yeah, mounting a directory as CD-ROM (-t cdrom parms to mount command in DOSBox) prevents writing to it regardless of permissions on host. It seems to also cause the same quirks as mentioned above with echo and edit.

 

In any case, I wouldn't be surprised if DOSBox itself had some bugs that could be used to escape from the emulated environment. :P Like this. So if somebody is truly afraid of malicious WADs they should probably run DOSBox under a separate limited user account, not just restrict write access in mounted directories.

Edited November 16, 2020 by xttl

[Gez]

Run DOSBox inside a sandbox. DOSBoxie!

[ketmar]

On 11/16/2020 at 9:49 AM, xttl said:

I sure would love to do this if time travel was possible. :P Though as has been mentioned before in this and/or the other exe hacking thread, it might have led to less interest in the source code release which would have had some downsides in the long run.

afaik, Carmack wanted to open-source his works anyway, so other people could learn from it. i don't think that releasing a hack like this would hurt source release in any major way. what i think is that after getting the official source, all sourceports would add a native support for decorate-like scripting, and at this day we'd have a solid common ground for modding and scripting.

 

maybe i'm too optimisitc there, but this is what i want to believe. ;-)

[zokum]

I think it is good that Doom never had an option to run bundled machine code. That is a security nightmare. Carmack did the right thing with Quake C and the compiled progs.dat bytecode. Quake 2 compiled to native code dlls for increased speed. That left us with tons of mods that no longer work and only available on win32 x86. It was also a security nightmare.

With Quake 3 a hybrid approach was used, which seems to be a good tradeoff of speed, portability, ease of use and security. There's more about it in the wikipedia article. https://en.wikipedia.org/wiki/QuakeC

Edited November 19, 2020 by zokum

[ketmar]

7 hours ago, zokum said:

I think it is good that Doom never had an option to run bundled machine code

but... it always had this ability! just look at the topic we're commenting...

[xttl]

12 hours ago, ketmar said:

but... it always had this ability! just look at the topic we're commenting...

 

Ya, but not really by design, unlike if HIGHBLIT was kept in. I'm tempted to say it would have been chaos in the 1990s if it was kept in (have fun scanning wads and hoping someone just didn't write their own virus or trojan which the scanners don't yet know about), then on the other hand, I don't recall many problems with Quake 2's approach which -- as mentioned above -- allowed or rather required mods to run native code.

 

I do faintly remember something about people finding out about a backdoor id left for themselves in all Q2 servers and going wild with it though. Basically the game had a hardcoded check for packets coming from certain IP address ranges which belonged to id, and if your packets appeared to come from those addresses then you could administer a server (eg. kick people) without authing via rcon first. The source address could be spoofed because you didn't even need to have an established connection to the server, and ISPs back then didn't do as aggressive egress filtering as nowadays.

 

It was discovered even before the Q2 source got released in 2001, but IIRC QW has the same backdoor so perhaps they forgot to remove it from the publicly released sources? It might have been discovered even before the Q1/QW source release actually... (though Q2 was definitely already out)

Speaking of Q1, I'm willing to bet the progs.dat interpreter can also be somehow exploited to execute native code, and that the bug is something that could have been plausibly discovered even before the source was released since there were huge and successful reverse engineering efforts going on already even in the Q1TEST days. :-) The first published working QuakeC compiler* wasn't even Carmack's official one.

 

(plus, Q1 v1.01 source code leaked years before the official v1.09 source release anyway...)

 

^{* I wish I'd remember the name, but it was written by a French-speaking person (all the error messages were in French). He also wrote a decompiler (obviously had to since the game's QC sources weren't yet released), and his compiler didn't compile proper Quake C but only the weird language his decompiler produced which had some influences from Pascal but also C-style curly braces. It is deacc/reacc by Armin Rigo (who's Swiss judging from the email address in readme). And looks like I was wrong about it being released before Carmack's QCC: files in the zip have timestamps in August 1996, Carmacks's files are dated late July 1996)}

Edited November 20, 2020 by xttl

[Redneckerz]

Update:

@Gez started a DoomWiki page for ACE Engine, which you can find right here.

 

I just finished touching up the page with some features and the video of @Baratus II which is such a good highlight of some of the project's feature, it would be criminal not to include.

 

Thanks fly out to Gez for this and congratulations to @kgsws for making it to the Doom Wiki. Your work has earned a place in Doom history and i hope you will continue to work on it onwards!

 

[kgsws]

Nice and interesting.

 

I have already made quite a few updates already but nothing too special to release yet. You can see commit messages in git repository.

(custom player class is already possible without custom weapons)

 

I had to take a break and do other stuff. I would really like to release updated version of my demo map this year.

It might take two weeks, but i will post an update and official release here.

 

Or should ACE Engine have it's own thread in "source ports" ?

[Redneckerz]

2 hours ago, kgsws said:

Nice and interesting.

Right back at you for gifting the community with this awesome project, Kgsws :)

2 hours ago, kgsws said:

I had to take a break and do other stuff. I would really like to release updated version of my demo map this year.

It might take two weeks, but i will post an update and official release here.

 

Or should ACE Engine have it's own thread in "source ports" ?

Take the time you need. And yeah, perhaps ACE Engine should have its own thread there - Or maybe the most important posts/screencaps can be moved over instead.

[ketmar]

2 hours ago, kgsws said:

Or should ACE Engine have it's own thread in "source ports" ?

yeah, it's about the time, i believe. while techically it's not a "source port", i think it will better fit there.

Edited December 9, 2020 by ketmar

[DoomKrakken]

Just discovered this thread through the 2020 Cacowards, congrats on the uber-notable mention! :D

Quick question... if being able to execute (just about) any sort of code through these means can happen with DOS Doom... would it be possible to do this sort of thing for certain sourceport engines (like, say, Zandronum)?  Would be cool to add ZScript/DECORATE parsers to other engines that have better networking capabilities...

[Blzut3]

2 hours ago, DoomKrakken said:

would it be possible to do this sort of thing for certain sourceport engines (like, say, Zandronum)?

In short: no.

 

Long answer: ACE exploits are considered security flaws so if one was found the source port developer would basically be obligated to fix it.  Especially in the multiplayer space where the odds of someone trying to be malicious is higher.  Additionally you'd have to defeat security in modern operating systems such as address space layout randomization.  And this doesn't even get into the issues with there being multiple operating systems you'd have to support (finding unique security holes), across multiple architectures, and potentially across multiple compilers.

 

All of this requires a much higher skill set than just downloading the Zandronum source code and getting on the development team.  Also keep in mind that doing ACE wouldn't mean you get multiplayer sync for free.

[Gez]

5 hours ago, DoomKrakken said:

Quick question... if being able to execute (just about) any sort of code through these means can happen with DOS Doom... would it be possible to do this sort of thing for certain sourceport engines (like, say, Zandronum)?

Theoretically yes, but with some caveats. The first is that such exploits are extremely specific: here we have something that works with Doom II v1.9 only. It does not work with Doom II v1.8, it does not work with Ultimate Doom v1.9, it does not work with any other version. And it's also OS-specific: it works with the DOS version of Doom II v1.9, not with the Linux or Mac versions.

 

Now suppose you make such a hack for a port. You'll have to target only one version of it, on one operating system. And that's where hurdles start piling up: first, modern OSes have anti malware active that try to prevent exactly this kind of exploits to happen on their watch. Because the assumption is that code insertion is not done to make cool Doom mod features, but to infest a computer with viruses or botnet nodes or whatever other shit the user generally does not want, so you'd have to outsmart these. Secondly, ports that would be worth targeting are ports with a large user base, so that means active ports with regular new releases, so you'd have to redo your hack all the time. Thirdly, the user base is fragmented across OSes and architectures -- something that works on a x86 won't work on  x64, or on a RasPi. And fourthly, once your exploit is known, the port maintainers will try to plug it.

 

So it's all a lot of tedious, insanely hard work and if all you want is to put a Snake minigame on Zandronum's menu, it's much simpler to do that with a good old pull request, you know? At least you won't have to fight the entire universe to get it to work.

[Maes]

The only way this could work is if all source ports agreed to have a common minimum supported subset of scripting actions/macros, said actions/macros were also implemented in vanilla through ACE, and all script authors agreed to use those and only those for any and all scripting. This would be fairly machine independent :-)

[Wavy]

I joined a few weeks ago but haven't posted/replied to anything. But I just woke up from my slumber to say holy frick.

 

Thing is, does this have any real practical use? Not shitting on this though, this is awesome!

Edited December 12, 2020 by Wavy

[Blzut3]

32 minutes ago, Wavy said:

Thing is, does this have any real practical use? Not shitting on this though, this is awesome!

The answer to this is probably going to vary a bit person to person, but from what I can tell there's basically only two truly practical uses for this: 1) It allows distribution of modified vanilla code using DMX in a way that's more legal than redistributing the complete binary.  I'm sure it's possible to armchair lawyer for days on if code injection actually is a loophole, but we can definitely agree that redistributing DMX without a license would be illegal. By live patching you avoid this redistribution. 2) It allows embedding some of the same conveniences that ports like Chocolate Doom provide potentially allowing running vanilla mods that would traditionally need deusf and dehacked simpler.

 

I really can't think of anything else that couldn't more easily be solved with "just compile the released source code."  From my point of view ACE Engine is just another source port on the block, it just requires doom2.exe to launch it.  The technical difference between traditional source ports and this are academic.  Which of course doesn't mean this isn't cool to mess with.

[Wavy]

50 minutes ago, Blzut3 said:

The answer to this is probably going to vary a bit person to person, but from what I can tell there's basically only two truly practical uses for this: 1) It allows distribution of modified vanilla code using DMX in a way that's more legal than redistributing the complete binary.  I'm sure it's possible to armchair lawyer for days on if code injection actually is a loophole, but we can definitely agree that redistributing DMX without a license would be illegal. By live patching you avoid this redistribution. 2) It allows embedding some of the same conveniences that ports like Chocolate Doom provide potentially allowing running vanilla mods that would traditionally need deusf and dehacked simpler. 

 

I really can't think of anything else that couldn't more easily be solved with "just compile the released source code."  From my point of view ACE Engine is just another source port on the block, it just requires doom2.exe to launch it.  The technical difference between traditional source ports and this are academic.  Which of course doesn't mean this isn't cool to mess with.

Interesting. Thanks!

[M_W]

Practicality seems like a odd metric to measure modifications to a 26 year-old DOS game in a forum dedicated to said game.

[kgsws]

Well, it's not quite ready so i will at least post an update.

Custom weapon support is almost finished (no bullet attack yet). Custom player class should also work.

There is no inventory (including ammo!) yet.

 

New test wad:

https://github.com/kgsws/doom_ace/tree/doom2_ace/wad

 

And as always, there are probably a lot of bugs included. So try it and report what is broken here.

[termrork]

@kgsws works nicely for me! I would really like to use this for a wad. But is there currently a way to record demos? I fear like having to load a map which does not load a real map will not allow you to record a demo and play it back. am I missing something?

[kgsws]

First, an update:

I added few more changes (available on git, but no binary) and for now this project is paused. Feel free to test what is available and report bugs.

(ammo is implemented)

 

 

5 hours ago, termrork said:

@kgsws works nicely for me! I would really like to use this for a wad. But is there currently a way to record demos? I fear like having to load a map which does not load a real map will not allow you to record a demo and play it back. am I missing something?

Yeah, it is not possible at this moment. Even native demos won't play 100% correctly right now (some work though).

At this point it is possible to make custom game with decorate actors in doom map format. However, few things are unstable / unfinished. For example, status bar is crashing ... sometimes.

I could hunt some of those bugs out if you would like to make a WAD.

[wallabra]

Do you think a limit-removing/Boom-based source port could implement compatibility for this by using x86 virtualization? Possibly by declaring an Universal API by which those abstract code modifications could abide, to prevent platform idiosyncrasies (DOS syscalls, etc) from harming the portability of modifications?

[ketmar]

56 minutes ago, Gustavo6046 said:

Possibly by declaring an Universal API

like... ahem... something in a text file that describes a state machine and state calls... i'm pretty sure i've seen that somewhere, i almost remembered...

[wallabra]

@ketmar oh god no. You don't mean the big D, do you?

 

I meant like, a basic instruction set, with ADD SUB MUL DIV, and relatively low-level API endpoints instead of whatever that decostrosity is.

 

What this would allow is basically endless flexibility, as we could define hooks for any action (menu operations, renderer callbacks, etc), instead of having to define a separate language for every single one of them, like MENUDEFS, GLDEFS, POOPDEFS, DECORATEDEFS, whatever-defs... you get the idea.

 

Screw declarative definitions, my homies be imperative to the bone, baby!

Edited February 17, 2021 by Gustavo6046

[ketmar]

1 hour ago, Gustavo6046 said:

I meant like, a basic instruction set, with ADD SUB MUL DIV

i vaguely remembering that Z-something is compiled into something that resembles what you described here...

 

and it failed to be cross-port exactly the way new VM will fail: because it is closely tied to the internal engine structures and implementation details. and if cross-port part won't work anyway, then DOSBox is enough -- it already has a good x86 emulator.

[ArgentFrequencies]

Looking to try out demo.wad, but it seems to be missing from the doom_ace github repository. It mentions how the 'directory "wads" contains (demo.wad and acetest.wad)', but demo is gone. Was it removed? If so, why?

[Gez]

12 minutes ago, ArgentFrequencies said:

Looking to try out demo.wad, but it seems to be missing from the doom_ace github repository. It mentions how the 'directory "wads" contains (demo.wad and acetest.wad)', but demo is gone. Was it removed? If so, why?

IIRC it's in a branch, not in master.

[ArgentFrequencies]

Just now, Gez said:

IIRC it's in a branch, not in master.

I can only suppose the location of demo.wad is a mystery then?