A "in Executable Save" source port

[Frost-Core]

Hello! so i play on LZDoom (i might look like a potato and is a newbie, but not really) and my hard drive has a low amount of free space (probably because of wads) but i hate this one thing, LZDoom saves to a file called "zds" this is in *ZDoom, but is there another one source port that saves its stuff inside of the exe?

[Paf]

No. If they did it would eventually/potentially corrupt the exe file and corrupt other data.

[Gibbon]

Saving inside the exe..  not possible.  In the old days (not Doom) but the 80's, programs used to save data by dumping the raw memory onto the filesystem as a kind of 'save file'.  But no, no port can save their data into the compiled executable.

[Frost-Core]

Just now, Gibbon said:

Saving inside the exe..  not possible.  In the old days (not Doom) but the 80's, programs used to save data by dumping the raw memory onto the filesystem as a kind of 'save file'.  But no, no port can save their data into the compiled executable.

Well ok, what about some thing else than that?

[Jayextee]

Your "save inside the exe" concept wouldn't actually save the space you want, just saying. It would still dump a similar amount of data on your drive, just in the executable (if that were indeed possible) instead of a separate file.

[Gibbon]

2 minutes ago, Frost-Core said:

Well ok, what about some thing else than that?

 

There are save files.  Basically, ones that are the same as ZDoom's versions but just with different names.  Even so, saving data anywhere will increase space regardless.

[magicsofa]

Just hit pause and don't turn off your computer :P

[Gregor]

while we're on this topic-- wouldn't it be preferable to have a way to save in ram instead of to a hard drive all the time? Since most of us use an ssd as their hard drive nowadays, each time you overwrite a save file (which happens quite frequently after all) you use up the ssd a little more, shortening its life cycle overall. i know using a hdd also wears out its mechanical parts eventually but with an ssd that kind of save-and-overwrite usage is a far bigger factor for wear&tear. and most playing sessions happen in one go anyways, so saving to a hard drive isn't really necessary most of the time. That way you could reduce the number of proper saves you have to clot your ssd with. Right? i know it is not a feature with any of the existing sourceports, but i feel like it could be a nice addition...

Edited April 12, 2022 by Gregor

[dasho]

Just using your OS is going to wear your hard drive down far more than saving Doom games ever will.

Edited April 12, 2022 by dasho

[Astronomical]

26 minutes ago, Gregor said:

wouldn't it be preferable to have a way to save in ram instead of to a hard drive all the time? Since most of us use an ssd as their hard drive nowadays, each time you overwrite a save file (which happens quite frequently after all) you use up the ssd a little more, shortening its life cycle overall.

No, if the exe crashes it would actually be worse because you would loose saves.

[Gibbon]

32 minutes ago, Gregor said:

while we're on this topic-- wouldn't it be preferable to have a way to save in ram instead of to a hard drive all the time? Since most of us use an ssd as their hard drive nowadays, each time you overwrite a save file (which happens quite frequently after all) you use up the ssd a little more, shortening its life cycle overall. i know using a hdd also wears out its mechanical parts eventually but with an ssd that kind of save-and-overwrite usage is a far bigger factor for wear&tear. and most playing sessions happen in one go anyways, so saving to a hard drive isn't really necessary most of the time. That way you could reduce the number of proper saves you have to clot your ssd with. Right? i know it is not a feature with any of the existing sourceports, but i feel like it could be a nice addition...

Not cross platform and once you shut down your computer, your saves would be gone.

[Gregor]

12 minutes ago, Astronomical said:

No, if the exe crashes it would actually be worse because you would loose saves.

 

7 minutes ago, Gibbon said:

Not cross platform and once you shut down your computer, your saves would be gone.

yes of course. but i didn't mean as the default way of saving. just an extra quicksave feature you can use whenever you intend to go on a save-scumming safari. pc don't tend to crash that often while running Doom and when you actually want to turn off your pc you could still save regularly to your ssd.

Edited April 12, 2022 by Gregor

[Redneckerz]

Making this topic (And other topics) puts wear and tear on your system OP. Better abstain from doing this until you have your system upgraded.

 

2 hours ago, Frost-Core said:

my hard drive has a low amount of free space (probably because of wads)

The entire .idgames archive is about 40 gigabyte give or take i believe.

 

So Wads aren't the problem. Using an old PC and a hard drive that likely has never witnessed a defrag is.

2 hours ago, Frost-Core said:

 

but i hate this one thing, LZDoom saves to a file called "zds" this is in *ZDoom, but is there another one source port that saves its stuff inside of the exe?

No port does this. ZDS files are ZDoomSave files.

 

It would also be silly to do since every exectuable will then be unique and tied to the user. No more troubleshooting possible\!

34 minutes ago, Gregor said:

while we're on this topic-- wouldn't it be preferable to have a way to save in ram instead of to a hard drive all the time? Since most of us use an ssd as their hard drive nowadays, each time you overwrite a save file (which happens quite frequently after all) you use up the ssd a little more, shortening its life cycle overall. i know using a hdd also wears out its mechanical parts eventually but with an ssd that kind of save-and-overwrite usage is a far bigger factor for wear&tear. and most playing sessions happen in one go anyways, so saving to a hard drive isn't really necessary most of the time. That way you could reduce the number of proper saves you have to clot your ssd with. Right? i know it is not a feature with any of the existing sourceports, but i feel like it could be a nice addition...

RAM drive is also write/read cycles. It just uses your memory instead. So eventually that dies aswell.

 

If you are really that concerned use a big HDD and dump all your wads on it. Problem solved.

 

Just now, Gregor said:

 

yes of course. but i didn't mean as the default way of saving. just an extra quicksave feature you can use whenever you intend to go on a save-scumming safari.

Would add nothing. SSD's and even HDD speeds are sufficient for this task.

 

Can we also save to tapedrive then? :)

[chungy]

ZDS files are incredibly small, you need to buy a new hard disk if you're that low on space.

 

2 hours ago, Gibbon said:

Saving inside the exe..  not possible.

 

It's hypothetically possible, you just need to add a new resource inside the EXE file (Windows uses PE which allow non-code resource segments just fine, this is how icon files, for example, live inside EXEs; other OSes and formats like ELF support similar features). Not that this will make it a good idea or solve the real problem of the thread, but hey, hypothetically possible :)

[Gregor]

9 minutes ago, Redneckerz said:

It would also be silly to do since every exectuable will then be unique and tied to the user. No more troubleshooting possible\!

RAM drive is also write/read cycles. It just uses your memory instead. So eventually that dies aswell.

 

If you are really that concerned use a big HDD and dump all your wads on it. Problem solved.

 

Would add nothing. SSD's and even HDD speeds are sufficient for this task.

 

Can we also save to tapedrive then? :)

Well, ok. i stand corrected.

[PKr]

A self recompiling prgoram in real time. 🤔🤔🤔

[Gregor]

14 minutes ago, PKr said:

A self recompiling prgoram in real time. 🤔🤔🤔

The future.

[PKr]

6 minutes ago, Gregor said:

The future.

https://youtu.be/qChxzPDLhl0?t=213

[Gregor]

7 minutes ago, PKr said:

https://youtu.be/qChxzPDLhl0?t=213

"Whoa..."

[Gibbon]

1 hour ago, chungy said:

ZDS files are incredibly small, you need to buy a new hard disk if you're that low on space.

 

 

It's hypothetically possible, you just need to add a new resource inside the EXE file (Windows uses PE which allow non-code resource segments just fine, this is how icon files, for example, live inside EXEs; other OSes and formats like ELF support similar features). Not that this will make it a good idea or solve the real problem of the thread, but hey, hypothetically possible :)

Indeed, however icon files are very distinct in this regard as they only contain the resource to change the windows exe icon. Plus, if the executable is being run, you won’t be able to save to it anyway as it is in use.

[Blzut3]

4 hours ago, Gregor said:

Since most of us use an ssd as their hard drive nowadays, each time you overwrite a save file (which happens quite frequently after all) you use up the ssd a little more, shortening its life cycle overall.

There was once a time when SSDs were tens of gigabytes and firmware was primitive, but for the most part people's fears about SSD endurance are grossly overblown.  I just had to change my main drive since with container images I was running out of space on my 500GB drive and based on the data in SMART my drive would have lasted 34 years or so before endurance was a problem.  A 1TB drive would probably out last me at my usage.  If you can find a published endurance rating for your drive you can do the math yourself by finding the total number of bytes written in SMART (typically the SSD manufactures provide a tool which can give you this number, but any SMART tool should work too but you might get the number in a weird format like number of 512 byte sectors).  If you can't find an endurance number a good way to estimate at the moment is assume 0.3 drive writes per day (typical for TLC drive) for 3 years (typical warranty period), so for a 500GB drive that's 500GB*0.3*365*3 or about 164TB (my drive was rated for 200TB and I wrote 29.4TB over about 5 years).

 

Everyone's computer that I've inspected won't ever come close to seeing NAND endurance exhaustion, so I feel safe in saying that attempting to micro optimize writes like you suggest is simply pointless.  Perhaps with a QLC drive (especially if dram-less) there's a chance of it happening, but even then it's fairly likely one will want a new drive before the QLC drive wears out.  This is of course ignoring that the amount of writing Doom does is going to be a drop in the bucket anyway even if you're a compulsive quick saver.

 

Since there will inevitably be someone that comments that their workload can wear out an SSD, yes I recognize those workloads exist, but that's not what the average person worrying about protecting their SSD is doing.

[Edward850]

5 hours ago, Gregor said:

while we're on this topic-- wouldn't it be preferable to have a way to save in ram instead of to a hard drive all the time?

Fun fact, Strife does this on the Nintendo Switch. Well kinda; Strife saves to a temporary slot when you move between levels, used to keep offline level data out of RAM for later saves or level traversal. However this turned out to be writing a lot of files in the late game (there's one file for every level visited) and the Switch has a write limit requirement because of its flash, and Strife was easily going well over because of this. To fix this, we create a RAM disk (which the Switch amazingly has native support for) and just redirected the temporary save slot to that. As this slot is not kept when you close the game, storing it to the RAM disk actually made a lot of sense.

 

However, this is only used for the temp slot. Writing hard saves from the user onto a RAM disk is just asking for trouble if the game is ever closed improperly, as anything that's in RAM will no longer have a chance to be stored properly if the application crashes.

This is actually what caused people to lose their save data when Zelda:OoT crashed on the Switch (from users performing glitches deliberately, I should note). As the N64 emulation was caching all save data to RAM first and only ever finally writing the data when the app was closed normally, if the emulation crashed before that happened all the save data made during that session would be taken with it.

Edited April 12, 2022 by Edward850

[chungy]

3 hours ago, Gibbon said:

Indeed, however icon files are very distinct in this regard as they only contain the resource to change the windows exe icon.

It's only an example; icons work because they're a well-defined resource, but there really isn't anything stopping you from having arbitrary resources inside them anyway. It's like how WAD files can contain whatever lumps you want regardless of type.

[dasho]

I think the point is that, unlike an icon, a save game isn't a compile-time resource.

[Shepardus]

6 hours ago, Gregor said:

while we're on this topic-- wouldn't it be preferable to have a way to save in ram instead of to a hard drive all the time?

dsda-doom's keyframes (either the rewind feature or the save/load keyframe buttons) essentially serve this purpose when you're not recording a demo. Of course this only lasts until you close the program, but that's good enough if you're playing a single map and not keeping a continuous playthrough.

 

On Linux and other Unix-likes you can mount a "tmpfs" file system so that files written to that location are actually in volatile memory rather than on a disk. One might choose to make their /tmp directory a tmpfs, for example. Any program can then write files there as if it were any other location, but in fact the files are in volatile memory and will be lost when the system is powered off or the tmpfs is unmounted.

[gwain]

10 hours ago, Gregor said:

wouldn't it be preferable to have a way to save in ram instead of to a hard drive

correct me if I'm wrong but ram gets overwritten by other peices of ram during things like rendering and other basic stuff thats what makes tails os secure since all it takes is a boot up from another os and it gets wiped unless you partition ram but that would make your pc slower

[chungy]

8 hours ago, dasho said:

I think the point is that, unlike an icon, a save game isn't a compile-time resource.

I never claimed it needed to be a compile-time resource. Even icons don't need to be that, there isn't any issue in modifying an EXE file.

[dasho]

6 hours ago, chungy said:

I never claimed it needed to be a compile-time resource. Even icons don't need to be that, there isn't any issue in modifying an EXE file.

 

"You just need to add a new resource inside the EXE file" 

 

What possible other interpretation could there be for this? If you mean loading an external image into memory before displaying it as a program icon (which you can certainly do, and in fact I do it with FLTK in one of my own programs), that's fine, but that isn't modifying the executable itself.

Edited April 13, 2022 by dasho

[chungy]

45 minutes ago, dasho said:

What possible other interpretation could there be for this?

 

The PE format is well-known and there's plenty of utilities for modifying them. Heck that's even basically what the resource compiler does when you include an icon, though it's easy to not notice how that happens after the linker already runs.

 

There really shouldn't be any kind of implied circumstance where modifying an EXE at run time requires recompilation.

[dasho]

5 minutes ago, chungy said:

There really shouldn't be any kind of implied circumstance where modifying an EXE at run time requires recompilation.

 

Cool, never said that was the case. I'm referring to including resources. I'll quote Microsoft here:

 

Quote

To include resources in your Windows-based application with RC, do the following:

Create individual files for your cursors, icons, bitmaps, dialog boxes, and fonts.

Create a resource-definition script (.rc file) that describes the resources used by your application.

Compile the script with RC. For more information, see Using RC (The RC Command Line).

Link the compiled resource (.res) file into the application's executable file with your linker.

 

Sure, if you want to go through the effort of replacing files and file pointers within the PE format during runtime, assuming that this will pass muster on any kind of anti-malware, instead of just having functions to load/save from other locations as needed, go ahead.