Testing an unpacked IWAD for speed

[Linguica]

As you might know, Doom / Doom 2 store their graphics as "patches." Wall textures are then built up out of these patches, sometimes as just one big patch, other times by taking a bunch of small patches and laying them next to / on top of one another to form the final texture.

Turns out that much of the infamous "refresh daemon" wait when loading up Doom was due to the engine building up the textures out of patches and storing them in memory.

It made me wonder what would happen if you made a new IWAD that replaced all textures with single-patch versions of themselves, so there was none of that initial building process. Would it make the game load up quicker?

I got Doom 2 running on DOS on an old laptop for other reasons and it made me think that I should try it out:

Turns out, yup, the game loads noticeably faster with an IWAD where all the textures have been "unpacked" into their own individual patch.

The IWAD is about a megabyte larger (15.3ish megs instead of 14.3ish megs) but in this test on an old laptop it loaded 3-4 seconds faster. On slower computers I imagine the time disparity would have been even greater.

[Quasar]

And for all the trouble gone to in the engine to build and maintain in cache these multipatch textures, which can't be used in all situations either. You could probably easily call it the largest technological mistake in the renderer, in hindsight.

[Aliotroph?]

I didn't realize Doom built all the textures before the game runs. I wonder how much smaller its RAM footprint could be if it loaded only the texures it needed on each map transition.

Did you use a tool to unpack all the textures or did you do that by hand? It seems like a lot of effort just to test this.

[Linguica]

Aliotroph? said:

Did you use a tool to unpack all the textures or did you do that by hand? It seems like a lot of effort just to test this.

With SLADE it's practically trivial to export all the textures and then re-import them as single-patch textures.

[kb1]

Quasar said:

And for all the trouble gone to in the engine to build and maintain in cache these multipatch textures, which can't be used in all situations either. You could probably easily call it the largest technological mistake in the renderer, in hindsight.

Technologically: True.
Financially: Hmmm.

I would vote Medusa, then the linear wad lookups - yuck.


Id was able to save a floppy for each game sold - that's more than pizza money. Actually I'm a bit disappointed that they did not get more use out of patches. They could have either made a lot more textures, or saved a lot more memory. As it stands, they just didn't get used as much as they could. Of course the textures would repeat even worse than they do now, but, in support Quasar's post, for all the trouble, they were actually pretty under-utilized. Could have been stretched to get used for sprites as well (which would require a Medusa fix).

Aliotroph? said:

I didn't realize Doom built all the textures before the game runs. I wonder how much smaller its RAM footprint could be if it loaded only the texures it needed on each map transition.

They eventually are needed, so it's hard to tell how useful it would be in vanilla. In my research, the sounds tend to fill up the cache, and, due to vanilla's strange custom memory allocator, full cache thrashing could occur, where sounds push out textures, which push out sounds, etc. Each frame. Map 30 could do this after a few minutes.

@Linguica: Remember all that talk about the Doom engine book? You're doing the proper research to be able to write a unique tome, with an unparalleled level of technical expertise. I'd jump at the opportunity to assist in such a masterpiece :) I suppose the Wiki serves some of that purpose.

[Quasar]

kb1 said:

Technologically: True.
Financially: Hmmm.

...

Id was able to save a floppy for each game sold - that's more than pizza money. Actually I'm a bit disappointed that they did not get more use out of patches. They could have either made a lot more textures, or saved a lot more memory. As it stands, they just didn't get used as much as they could. Of course the textures would repeat even worse than they do now, but, in support Quasar's post, for all the trouble, they were actually pretty under-utilized.

I'd be interested to know how much of that space could be recovered by removing duplicated WAD lumps, unused beta sprites, etc. You'd have to consider how ineffectually they used the space available when doing that cost analysis.

[Jon]

And they didn't do any lump compression, and tools like wadptr get pretty good results and remain vanilla compatible.

[Da Werecat]

The patch system is likely to be an artifact of the time when the engine couldn't tile anything, and they were going for realistic interiors. It would be ridiculous to draw and store separate big textures for unique walls.

Changing goals and engine specs made a mess of the textures from the artistic standpoint as well. A lot of walls were drawn at inappropriate sizes - height of 72 was very common, for example. Then they had to be shoved in power-of-two x 128 dimensions by repeating and cropping things.

[Jon]

Da Werecat said:

The patch system is likely to be an artifact of the time when the engine couldn't tile anything, and they were going for realistic interiors. It would be ridiculous to draw and store separate big textures for unique walls.

I always thought it was so if the artists tweaked the appearance of a switch, say, they didn't have to manually copy and paste their adjustment over all the textures that used it.

[andrewj]

Yeah imo they just thought that the texture composition system would be a very useful feature.

[Maes]

I think that was a classical example of a speed-space tradeoff. Back in the day average PCs didn't have much of either, but space (especially RAM, and to a secondary extent disk space) was more likely to be at a premium. It was usually more acceptable to have the user wait longer than to fill his HDD to the brim.

Compounded with the editing convenience (defining a new texture just by using a reference structure), and the fact that texture compositing was a one-time (?) overhead (at least for single-sided textures), it was deemed as a totally acceptable solution.

[Da Werecat]

Yeah, it's pretty useful with switches. Instead of coming to an artist and whining about not having a switch X on a surface Y, you can combine them yourself. Also wall decals, although they aren't used nearly as much.

On the other hand, when it came to regular walls, Doom 2 didn't rely on texture combining as much as the first game.