Vrack2-SuperHuge - So large (10x Vrack2), it requires a custom linedef format

[Redneckerz]

EDIT 2: A map overview of SuperHuge. Credits to @RjY:

 

 

EDIT: The puzzle is solved:

 

17 hours ago, Gez said:

Some sleuthing:

 

For Vrack 2 original flavor:

 

Vrack 2 Biggest Ever (aka Vrack2 x5)

12575x5=62875, so that's an extra 65 linedefs here. But when you look at the map, it makes sense. Four of the Vrack2s have been connected:

 

Vrack2 Super Huge:

Now that's a DMMPST count. And if it's a weird format, it's no longer valid.

 

But ten times Vrack2 would be 125 750 lines. (Looking at the mess of points, I don't see the trace of "connective tissue" between the Vrack2s here.) That's an extra 69 762 linedefs that are counted. Far too many, based on the BiggestEver precedent.

 

The lump size is 2 737 174. A Doom format linedef is 14 bytes. So if you had 195 512 lines, as counted by DMMPST, that'd give us a lump size of 2 737 168. But we have six extra bytes.

 

Let's try to find a value larger than 14 that 2737174 would divide cleanly by. Turns out that 2 737 174 = 22 × 124 417. That's a bit less than 125 750. We lose 1333 lines. But despite this discrepancy, I think this is correct.

 

 

Edit: 12 575 lines is for Vrack2b. The original Vrack2 "only" has 12 441 lines. That gives us 735 extra lines for BiggestEver (makes sense when you look at the long stair sectors, 65 lines wouldn't be enough!) and only 7 extra lines for SuperHuge according to my guess that it uses 22-byte linedefs.

 

Re-Edit: turns out that if you take the vanilla linedef format (14 bytes, remember) and decide to put 32-bit values instead of 16-bit values for its start vertex index (+2 bytes = 16), end vertex index (+2 bytes = 18), front sidedef (+2 bytes = 20) and back sidedef (+2 bytes = 22), you arrive at exactly the size that makes sense according to everything I posted previously. Mystery solved, I say.

 

Original Post:

Introduction:

(Thanks to @Xymph for stats generation)

 

(Boom-compatible port, possibly a REJECT-Table, a node builder and sanity required)

(Perhaps Risen3D runs this?)

 

Okay, so this obviously isn't mine. But as i was researching some test wads for Doom128, i stumbled upon this by chance. Story time.

 

PS: I don't dare putting my own PC to the test for this - What with a Core i5-4590T, 16 GB DDR3 and HD4600. This thing seems to defy what makes common sense even for the extreme heights of Doom maps. You have been warned.

 

Back in 2014, @Gez mentioned Vrack2-BiggestEver - 5x the map VRack2 merged into one WAD.  It was a testing map from 2005 meant to test the capabilities of Deep and Risen3D. It clocks in at 3.9 MB.

 

For the record, lets see how stock Vrack2 stacks up - by itself a hefty map.

 

Stats of Vrack2:

 

Map data
Things 	        1337
Vertices 	13593*
Linedefs 	12575
Sidedefs 	21705
Sectors 	2516
* The vertex count without the effect of node building is 10549.

 

Lets stack them against BiggestEver:

 

Stats of Vrack2-BiggestEver:

Map data
Things 	        6567
Vertices 	70414*
Linedefs 	62940
Sidedefs 	33839
Sectors 	12647
* The vertex count without the effect of node building is 52602.

So... what's this then?

As it now turns out there was a 2018 successor to this. It is called Vrack2-SuperHuge, and it clocks in at 9.4 MB.

 

Stats of Vrack2-SuperHuge:

Map data
Things 	        13180
Vertices 	140868*
Linedefs 	195512
Sidedefs 	214344
Sectors 	24820
* The vertex count without the effect of node building is 58468.

 

Like BiggestEver, i assume this is a testing map for ports and level builders to see how far you can go.

 

Trivia:

• SuperHuge not only beats out Deus Vult MAP05, Okuplok, and even Planisphere 2 when it comes to Things, Vertices, Linedefs, Sidedefs and Sectors, it completely oblitarates them. However, if BiggestEver is 5x Vrack2 connected together, how much will this be?
• DMMPST handled the wad for stats generation, so Xymph's tooling survived another day!

 

Link:

• Vrack2-SuperHuge

Edited February 19, 2022 by Redneckerz
Edited to include Rjy's map overview, + general editing.

[Gibbon]

Add to that both PrBOOM+UM and the latest GZDoom, both locked up my Mac.  Nothing can run this wad!

Edited February 18, 2022 by Gibbon

[Walter confetti]

This sounds insane!

[thiccyosh]

Oh my, this wad eats up quite the ram.

 

Brb gonna download more ram and a port that doesn't crash when loading any of these maps. Not even Ultimate Doom Builder (or Slade) is mighty enough to handle this thicc wad.

 

Oh my god it crashes everything, and Doom Builder is not having a great time. What is this black magic?

 

Edit: well, at least the music is nice I guess

Edit 2: kills dsda-doom because line 2016 is missing a tag?

Edit 3: kills Zdoom too. I know it was obvious but it was worth a try.

Edit 4: I can kinda look into the map by using Slades map editor. Hmm, just lines that summon godrays and inside visual mode it's just textures all over the place. Strangely enough there are no flats to be seen.

Edited February 18, 2022 by thiccyosh
hungagalunga

[Andromeda]

Finally a reason to upgrade my PC.

[Gez]

SLADE doesn't recognize the LINEDEFS lump and declares the map invalid. If I force it to accept the linedefs however, I can see a completely glitched mess of lines but the vertices hint that it's a full ten copies of Vrack2 crammed in. And then it gets an unhandled exception error.

 

 

 

Based on that, I figure that it uses a custom LINEDEFS format, perhaps to allow to overcome the sidedefs limit. And then it makes sense nothing supports it.

Edited February 18, 2022 by Gez

[Redneckerz]

7 minutes ago, Gez said:

SLADE doesn't recognize the LINEDEFS lump and declares the map invalid. If I force it to accept the linedefs however, I can see a completely glitched mess of lines but the vertices hint that it's a full ten copies of Vrack2 crammed in. And then it gets an unhandled exception error.

 

 

 

Based on that, I figure that it uses a custom LINEDEFS format, perhaps to allow to overcome the sidedefs limit. And then it makes sense nothing supports it.

I honestly can't say. The only thing i can think of is attempting to open it in DeepSea or Risen3D. There wasn't a readme or anything. It just resided on the files directory, with nothing else linking to it.

 

If it uses a custom LINEDEFS format, that seems such overkill. I would think SuperHuge is like BiggestEver, just exponentially larger (As you say, 10x Vrack2). I also wouldn't want to count this off as a jokewad either, because BiggestEver wasn't. That was a demonstration map, this ought to be the same, but adjusted for 2018 standards.

 

PS: I am amazed SLADE can even open it first hand.

 

No matter what this turns out, it atleast is a fascinating story.

[roadworx]

9 minutes ago, Gez said:

Based on that, I figure that it uses a custom LINEDEFS format, perhaps to allow to overcome the sidedefs limit. And then it makes sense nothing supports it.

so it makes everything crash not because it's big, but cuz it's just incompatible with everything? well, that's disappointing

[Shepardus]

The Doom/Boom map format has a limit of 65536 linedefs since it uses 16-bit values, so SuperHuge is either UDMF or something else entirely; no surprise most of these source ports can't run the map.

 

I thought I'd mention that Ar Luminae, a map you can actually play, has:

9490 things

121191 vertices

143840 linedefs

265941 sidedefs

25952 sectors

With total dimensions of 31175x26683.

[Gez]

Some sleuthing:

 

For Vrack 2 original flavor:

1 hour ago, Redneckerz said:

Linedefs 12575

 

Vrack 2 Biggest Ever (aka Vrack2 x5)

1 hour ago, Redneckerz said:

Linedefs 62940

12575x5=62875, so that's an extra 65 linedefs here. But when you look at the map, it makes sense. Four of the Vrack2s have been connected:

 

Vrack2 Super Huge:

1 hour ago, Redneckerz said:

Linedefs 195512

Now that's a DMMPST count. And if it's a weird format, it's no longer valid.

 

But ten times Vrack2 would be 125 750 lines. (Looking at the mess of points, I don't see the trace of "connective tissue" between the Vrack2s here.) That's an extra 69 762 linedefs that are counted. Far too many, based on the BiggestEver precedent.

 

The lump size is 2 737 174. A Doom format linedef is 14 bytes. So if you had 195 512 lines, as counted by DMMPST, that'd give us a lump size of 2 737 168. But we have six extra bytes.

 

Let's try to find a value larger than 14 that 2737174 would divide cleanly by. Turns out that 2 737 174 = 22 × 124 417. That's a bit less than 125 750. We lose 1333 lines. But despite this discrepancy, I think this is correct.

 

 

Edit: 12 575 lines is for Vrack2b. The original Vrack2 "only" has 12 441 lines. That gives us 735 extra lines for BiggestEver (makes sense when you look at the long stair sectors, 65 lines wouldn't be enough!) and only 7 extra lines for SuperHuge according to my guess that it uses 22-byte linedefs.

 

Re-Edit: turns out that if you take the vanilla linedef format (14 bytes, remember) and decide to put 32-bit values instead of 16-bit values for its start vertex index (+2 bytes = 16), end vertex index (+2 bytes = 18), front sidedef (+2 bytes = 20) and back sidedef (+2 bytes = 22), you arrive at exactly the size that makes sense according to everything I posted previously. Mystery solved, I say.

Edited February 18, 2022 by Gez

[Redneckerz]

40 minutes ago, Gez said:

Some sleuthing:

 

For Vrack 2 original flavor:

 

Vrack 2 Biggest Ever (aka Vrack2 x5)

12575x5=62875, so that's an extra 65 linedefs here. But when you look at the map, it makes sense. Four of the Vrack2s have been connected:

Humor me: Why is the last VRack not interconnected?

40 minutes ago, Gez said:

Mystery solved, I say.

There is usually no reason to the question who is the actual Wiki editor because that usually is you. And you are showing that here once more. Thank you for carrying out calculating the hard values.

 

A few personal questions remain:

• What would be needed to have this run, outside of what i suggested (Deepsea/Risen3D)? Or could this run if it were attempted through a different node builder
• Because i am curious of how it looks, is it possible to generate a map overview for SuperHuge? As you showed, it even crashes Slade with an exception, so i assume this needs some python script to get going?

 

[Xymph]

55 minutes ago, Gez said:

Vrack2 Super Huge:

2 hours ago, Redneckerz said:

Linedefs 195512 

Now that's a DMMPST count. And if it's a weird format, it's no longer valid.

I wasn't aware of the custom linedef format, and only gave the map a cursory glance in Slade3 (an older version at that) which errors out on previewing the map. DMMPST didn't crash, but indeed miscalculated the line count. I don't think it'd make sense to support the format as deducted by Gez though.

 

Omgifol doesn't support the custom format either, so no easy way to generate a map view there either. Maybe with a modified version?

Edited February 18, 2022 by Xymph

[Redneckerz]

Just now, Xymph said:

I wasn't aware of the custom linedef format, and only gave the map a cursory glance in Slade3 (an older version at that) which errors out on previewing the map. DMMPST didn't crash, but indeed miscalculated the line count. I don't think it'd make sense to support the format as deducted by Gez though.

I understand. At the very least, it highlighted that DMMPST miscalculated the line count. I am not sure if it is useful for improving your scripts (Because that would be the icing on the cake) but there you go.

Just now, Xymph said:

 

Omgifol doesn't support the custom format either, so no easy way to generate a map view there either.

I was about to message you over the Wiki for this. I agree that supporting the format would be useless since only SuperHuge needs it. But would it be possible to generate map overview for this? I kind of want to know if SuperHuge is merely 10x VRack 2 in one wad, or that they are interconnected like BiggestEver.

 

If its the latter, then it would be worth investigating if Risen3D runs it because at this point i am wondering what its purpose is for a test map. It should be runnable by something.

[Gez]

2 minutes ago, Redneckerz said:

Humor me: Why is the last VRack not interconnected?

🤷 Ask Jack Vermeulen.

 

5 minutes ago, Redneckerz said:

What would be needed to have this run, outside of what i suggested (Deepsea/Risen3D)? Or could this run if it were attempted through a different node builder

No, the nodebuilder is irrelevant here.

 

What's needed is to have the port be aware of the different linedef_t format used here. Some sort of "if you fail loading the lines as standard Doom format, try loading them as this format". The heuristics could also look at the size of the SIDEDEFS lump. If it contains more than 65534 sides and the map is in Doom format, try loading lines as extended format.

 

10 minutes ago, Redneckerz said:

Because i am curious of how it looks, is it possible to generate a map overview for SuperHuge? As you showed, it even crashes Slade with an exception, so i assume this needs some python script to get going?

Take a script that draws a map, find where it defines the linedef_t structure, and edit that part to change the vertex and sidedef indices to 32-bit ints instead of 16-bit ones. It should now read his map correctly (and be broken for every regular map, but that's okay, you'll just have to revert your change once your curiosity is sated).

[Xymph]

48 minutes ago, Gez said:

Take a script that draws a map, find where it defines the linedef_t structure, and edit that part to change the vertex and sidedef indices to 32-bit ints instead of 16-bit ones. It should now read his map correctly (and be broken for every regular map, but that's okay, you'll just have to revert your change once your curiosity is sated). 

Tried to modify Omgifol, which defines in mapedit.py:

Linedef = make_struct(
  "Linedef", """Represents a map linedef""",
  [["vx_a",   'H',  0],
   ["vx_b",   'H',  0],
   ["flags",  'H',  0],
   ["action", 'H',  0],
   ["tag",    'H',  0],
   ["front",  'H', -1],
   ["back",   'H', -1]],
  ["impassable", "block_monsters", "two_sided",
   "upper_unpeg", "lower_unpeg", "secret",
   "block_sound", "invisible", "automap"]
)

Changed the H's into I's for the first two and last two fields, but drawmaps.py crashes as before ("struct.error: unpack requires a string argument of length 24"). Not sure why, maybe @Revenant can assist?

[RjY]

3 hours ago, Redneckerz said:

I kind of want to know if SuperHuge is merely 10x VRack 2 in one wad, or that they are interconnected like BiggestEver.

 

 

Sadly it's only ten copies of vrack2.wad. No apparent interconnection.

(I don't know what the seven extra lines are for, or even where they are.)

BTW, dare I ask why you write the title with a capital R in the middle..?

[Redneckerz]

11 hours ago, RjY said:

 

 

Sadly it's only ten copies of vrack2.wad. No apparent interconnection.

(I don't know what the seven extra lines are for, or even where they are.)

Damn, that looks impressive. Thank you!
 

Almost looks like a schematic for a chiplet design :P  It does look like it would be easy to interconnect.

 

How were you able to make that map overview if i may ask, considering the custom linedef format?

11 hours ago, RjY said:

BTW, dare I ask why you write the title with a capital R in the middle..?

Unintentional habit. Ill correct it straight away.

[RjY]

1 hour ago, Redneckerz said:

How were you able to make that map overview if i may ask, considering the custom linedef format?

I already had the code written from years ago: a simple map dumper, and some glue to feed it into ImageMagick. I modified it in exactly the way Gez suggested, and everything just worked :).

1 hour ago, Redneckerz said:

Unintentional habit. Ill correct it straight away.

Oh right. It was just that you were so consistent that I thought you must have found a source that "VRack" was the proper way to write it, perhaps as part of your research for the wiki. Fair enough if it's just a personal preference though. I mean, look at all the different ways people capitalize Doom.

[Revenant]

22 hours ago, Xymph said:

Changed the H's into I's for the first two and last two fields, but drawmaps.py crashes as before ("struct.error: unpack requires a string argument of length 24"). Not sure why, maybe @Revenant can assist?

 

The resulting struct should only be 22 bytes, not 24. There should still be three 16-bit fields in the middle.

 

(On a sort of related note, a few weeks ago I totally reworked how structs are specified in the master branch of omgifol on GitHub. You might find the new style a bit easier to work with - the only real difference for this situation is that you'd be changing "ctypes.c_uint16" to "ctypes.c_uint32")

[Redneckerz]

5 hours ago, RjY said:

I already had the code written from years ago: a simple map dumper, and some glue to feed it into ImageMagick. I modified it in exactly the way Gez suggested, and everything just worked :).

My heartfelt thanks for doing the kind of thing i cannot :)

5 hours ago, RjY said:

Oh right. It was just that you were so consistent that I thought you must have found a source that "VRack" was the proper way to write it, perhaps as part of your research for the wiki. Fair enough if it's just a personal preference though. I mean, look at all the different ways people capitalize Doom.

Some ways of capitalization tend to do this, but, me being stubborn after reading Vrack without the capital R, subconsciously continued on as it were.

 

Thanks to you and Gez for digging up an oddball wad and cracking its little puzzle :)

[boris]

On 2/18/2022 at 10:15 PM, Gez said:

Take a script that draws a map, find where it defines the linedef_t structure, and edit that part to change the vertex and sidedef indices to 32-bit ints instead of 16-bit ones. It should now read his map correctly (and be broken for every regular map, but that's okay, you'll just have to revert your change once your curiosity is sated). 

Can also confirm the changed data structure:

 

 

[boris]

And I raise you Vrack 2 Superhuge 2, that includes 40x Vrack 2!

 

 

And there's even space left that could hold some more!

 

Writing and loading that thing works, but performance is, uh, less than stellar.

[Andromeda]

1 hour ago, boris said:

Writing and loading that thing works, but performance is, uh, less than stellar.

 

That thing probably needs a NASA supercomputer to get a playable framerate!

[Gez]

I doubt the map would be playable anyway; unless each copy has its own tag range things will start to get screwy in the second map as you re-trigger events that are supposed to happen only once. And monster teleport ambushes from all of the map will pour through a single map, which may or may not be the one you are currently on.

[Redneckerz]

2 hours ago, boris said:

And I raise you Vrack 2 Superhuge 2, that includes 40x Vrack 2!

 

 

And there's even space left that could hold some more!

 

Writing and loading that thing works, but performance is, uh, less than stellar.

I would already take a stab at the original if it:

• was interconnected
• Actually runs in a port versus using custom tricks to get there
  • Easy way out is to take the Ar Luminae route

As for this 40x monstrosity:

It actually looks nice from a overview map. Now, if that was delivered in a playable state... you'd have the map that kills all other maps.

 

Can your PC run Crysis? Sure. Can it run SuperHuge 2? Oh hell no.

 

Should be the title: Vrack2: Oh Hell No.

[SiFi270]

what if there was a nuts between every vrack

[Xymph]

On 2/19/2022 at 9:47 PM, Revenant said:

The resulting struct should only be 22 bytes, not 24. There should still be three 16-bit fields in the middle.

Yes, I kept H for those fields, but the old code kept erroring out.

 

Quote

On a sort of related note, a few weeks ago I totally reworked how structs are specified in the master branch of omgifol on GitHub. You might find the new style a bit easier to work with - the only real difference for this situation is that you'd be changing "ctypes.c_uint16" to "ctypes.c_uint32"

Thanks for the heads-up, I finally got around to installing this. And now the drawmaps script can generate the map view correctly (at standard scale 4.0):

 

Edited February 24, 2022 by Xymph

[Redneckerz]

A great ending to a story. Im glad this thread brought improvements to things i didn't feel needed improving, but here we are. Great stuff @Xymph :)