ZokumBSP 1.0.10 finally released!

[zokum]

ZokumBSP 1.0.10 is out after months of beta. I figured I'd post it here rather than in the less read mapping forum since it has a few really nifty tricks like the new multi-tree mode and code to fix bad seg angles, as seen in many pwads and iwads from id software. It's tagged as release candidate 1, if no one reports any bug, this will be the official release. Previous release candidates have been very stable.

Highlights in this release:
* Multi-tree mode. Builds better trees, but takes a lot lot longer, so not standard.
* Angle fix. Makes some diagonal walls render correctly unlike previous versions, ZenNode, DoomBSP and BSP.
* Texture alignment fix for diagonal walls with more than one split.
* Speed and memory improvements.
* Can now list the amount of seg splits when done building a map.
* Massive improvements to the output of the build process.
* Neat progress bar that is fairly accurate, even for multi-tree mode.
* Command line switch -c for color coded output support, requires a VT220 / ansi terminal.
* Command line switch -cc for 24bit color support for really nice progress bars and other tweaks.
* New time format, showing time in days, hours, minutes, seconds, milliseconds.
* 64bit windows build and greatly updated tool chain should give better performance.
* Updated manual.
* Source code for partition tuner for expert guru only users.
If anyone has some spare cpu cycles to burn, I'd love to get some more numbers for my on-going case study of multi-tree builds. The ~15 year old Pentium 4 machine has been working on e1m6 for about two weeks now.

Download link for win64 binary: http://www.doom2.net/zokum/zokumbsp/releases/zokumbsp-1.0.10-rc1-win64-mingw2.zip

Manual: https://raw.githubusercontent.com/zokum-no/zokumbsp/master/doc/zokumbsp.txt
Changelog: https://raw.githubusercontent.com/zokum-no/zokumbsp/master/doc/changelog.txt

New case study with loads of numbers from multi-tree builds now available: http://doom2.net/zokum/zokumbsp/case-multitree

Brad Spencer's Beginner's Guide: http://doom2.net/zokum/zokumbsp/guide-idiot
 

For more infomation, check out the web site http://doom2.net/zokum/zokumbsp/ or contact me on irc, the usual channels.

Edited October 19, 2018 by zokum

[zokum]

This release also includes a small demonstration wad showcasing linedef types 1078 and 1079. These are used by the node builder to move multiple linedefs to span from new start and end vertices. This allows you to easily set up trigger lines that span multiple sectors, exactly on top of eachother to trigger multiple effects at the same time.

The demo map has 4 lines at once. Two of those are one-time, and two are repeatable. Since they are exactly on top of eachother, it's impossible to bug a trap by only triggering one line. Beware that Doom has a limit of 8 linedef specials per game tic, triggering more will lead to array overflow and overwriting other data strucures, most likely crashing or segfaulting.

Just give the map a try and it will be more obvious what this can be used for.

[elend]

First; congrats and thanks for your effort!  

 

Second; I don‘t want to sound ungrateful or stupid, but does an optimized BSP tree and or optimized nodes improve the fps one might get in my map? Can I just add ZokumBSP to my GzDoom Builder config? :)

[zokum]

Better nodes improve the speed and allow you to make larger and more detailed maps. You can just add it to GZDoombuilder. Either as a separate node builder or just replace zennode.exe by renaming the executable. ZokumBSP supports the same stuff as ZenNode, and fixes several bugs and adds optimizations.

If you use anotak's Doombuilder X fork, it comes with support out of the box and a slightly older version than the one i released tonight. It also has support for some of the new linedef stuff.

[Spie812]

Sweet! Thanks for all your hard work on improving node building; ZokumBSP is definitely my go-to when I make stuff.

[zokum]

If anyone has a wad / project where ZokumBSP is used, I'd love to put it on a list on the web site. I'd like to have: Project name, map slots, author(s), url. The Doom community tends to slightly sceptical of programs without a proven track record.

Edited October 19, 2018 by zokum

[kb1]

Just wanted to say congratulations - great job!

[ChrisVR]

Sorry, what is it and what does it do?

Can we covert bsp to doom map files?

[zokum]

10 minutes ago, ChrisVR said:

Sorry, what is it and what does it do?

Can we covert bsp to doom map files?

It's a nodebuilder / blockmap builder / reject generator. It makes the stuff you make in an editor playable in Doom. Bsp is just the type of algorithm Doom uses.

 

[Reisal]

This just builds standard nodes/blockmap/reject, correct? No GL stuff?

[boris]

On 10/19/2018 at 5:05 AM, zokum said:

* Multi-tree mode. Builds better trees, but takes a lot lot longer, so not standard.

Is it using multithreading?

[zokum]

4 hours ago, Glaice said:

This just builds standard nodes/blockmap/reject, correct? No GL stuff?

Yes, for now the focus has been Doom, not Boom or GZDoom etc. The limits aren't that important for those executables.

[zokum]

3 hours ago, boris said:

Is it using multithreading?

No, but it is on the todo-list. ZokumBSP has a lot of global state, that needs to be fixed before multithreading is possible. None of the original code was written with multithreading in mind and global variables are used to speed up the algorithms.

When / if I add multithreading, i will probably do it for the blockmap code first, as it's easier to implement here. Blockmap is still the biggest hurdle for people to get below the limits, so making this process quicker is important.

Edited October 20, 2018 by zokum

[Benjogami]

I tried out zokumbsp on one of my zekhmet maps: benjozek_v1zokumbsp.zip

 

I was hoping it'd help with the leaky sectors on the big set of stairs, but it's causing even more leaky stuff all over the map. For example:

 

 

To properly view the map you'll also need the resource (and music!): http://aanaaanaaanaaana.net/drop/zek_proj.zip

 

I'm using GZDB, replaced ZenNode.exe with the win64 zokumbsp.exe found here, and kept the same old ZenNone.cfg that I had. Sure would be nice if a nodebuilder could magically save me from having to wiggle all those vertices, which is the only idea that I have left. :D

[zokum]

1 hour ago, Benjogami said:

I tried out zokumbsp on one of my zekhmet maps: benjozek_v1zokumbsp.zip

 

To properly view the map you'll also need the resource (and music!): http://aanaaanaaanaaana.net/drop/zek_proj.zip

 

I'm using GZDB, replaced ZenNode.exe with the win64 zokumbsp.exe found here, and kept the same old ZenNone.cfg that I had. Sure would be nice if a nodebuilder could magically save me from having to wiggle all those vertices, which is the only idea that I have left. :D

This is exactly the kind of feedback I need. I now got a nice testing map to try to work out what the problems are. I'll have a look at it at once! :)

Edit: Ok, I had a look at it. It crashed Chocolate Doom. If this requires a limit-removing port of some sort, what did you use? The blockmap for this map would be 150kb. I doubt you could ever get this to run inside Doom. I am afraid the area this map spans is just too big. Both ZenNode and ZokumBSP would fail at building a blockmap for this map.

Edited October 22, 2018 by zokum

[Benjogami]

Sorry, I should have mentioned that it's a Boom map. I'm using prboom-plus-2.5.1.5.r4526-win32. The stock ZenNode that came with GZDB seems to build it mostly fine, except for the leaky sectors I mentioned. Here's that version of the map for reference: benjozek_v1.zip

[zokum]

ZenNode and ZokumBSP aren't built to handle Boom maps. If they build properly, it's because you're not pushing limits very far.

[Benjogami]

I'm surprised to hear that ZenNode wasn't built to handle Boom maps. I've successfully built some very large maps with ZenNode:

 

 

This is right around the limit for ZenNode; I opted to remove some stuff so that the blockmap would behave, rather than switch to ZDBSP. Maybe it wasn't designed for it, but ZenNode is the go-to node builder for all but the very largest Boom compatible maps, as far as I know.

[Ribbiks]

6 minutes ago, Benjogami said:

ZenNode is the go-to node builder for all but the very largest Boom compatible maps, as far as I know.

 

eh. ZenNode works well for most things, but gets angry at certain quirks, like overlapping lines and whatnot. If you're goal is to minimize slime trails I recommend zdbsp -X

[Benjogami]

Hmm, yeah. I resist ZDBSP because a proper REJECT table is good for performance sometimes, but that's certainly not the case for the zekhmet map. And then there's ZDaemon support... but that probably doesn't matter. :P

 

Thanks for the help y'all.

[VGA]

Can't you use another tool afterwards to create a REJECT table?

Edited October 22, 2018 by VGA

[Edward850]

21 minutes ago, VGA said:

Can't you use another tool afterwards to create a REJECT table?

No other tool exists, especially one that could even hope to handle Boom limits. 

[VGA]

The readme says that e4m7 and all Doom 2 maps were not processed with DoomBSP. Then which node builder was used?

 

@zokum

Which cmdline parameters are the best for the performance of the map on slow hardware?

[zokum]

ZokumBSP can build reject maps for really big maps. There aren't really any stringent vanilla limits there. ZenNode will crash on bigger maps due to using a 32bit variable with values that can't fit into so few bits. ZokumBSP uses 64bit for that variable.

"./zokumbsp -n- -b- map.wad" will only build the reject table, no nodes or blockmap. You can do that after having run zdbsp i guess.

ZokumBSP should be able to handle everything ZenNode does and a bit more, due to fixing bugs. There's no point in using ZenNode anymore. There are several known bugs. Maps built with ZenNode will be built badly in most cases where you have diagonal walls etc.

[zokum]

14 hours ago, VGA said:

The readme says that e4m7 and all Doom 2 maps were not processed with DoomBSP. Then which node builder was used?

 

@zokum

Which cmdline parameters are the best for the performance of the map on slow hardware?

Only e4m7 and Final Doom weren't built with DoomBSP due to these not being made inhouse by id software. E4m7 was most likely built with bsp and rmb, i haven't checked. It would be an interesting thing to find out about. Each nodebuilder has its own little quirks and signaure in the way it builds the bsp tree.

I will update the docs to make it more obvious I am talking about Final Doom, not Doom 2.

ZokumBSP has a blockmap algorithm that sorts the blockmaps lists by size, so the longest list of linedefs always come first. That makes it fairly easy to identify, unless another tool does the same of course. It also uses (default setting) a few different offsets to see which gives the smaller blockmap, so not having an offset of 8 (DoomBSP) or 0 (ZenNode, bsp, probably others), marks it as highly likely to be built with ZokumBSP. ZenNode's blockmap compression has a few other quirks that make it fairly easy to spot. Bsp doesn't have it turned on by default, unlike ZenNode. Uncompressed maps are therefor usually built with bsp.

Early 94-97 stuff might vary though, with other tools being used, like DEU etc. Anything after that tends to be built with only a few sets of tools, especially the bigger more well tested projects that are more demanding / cutting edge.

[zokum]

14 hours ago, VGA said:

The readme says that e4m7 and all Doom 2 maps were not processed with DoomBSP. Then which node builder was used?

 

@zokum

Which cmdline parameters are the best for the performance of the map on slow hardware?

The default settings should be fine unless you use really ancient hardware. I can build all of Doom 2 on 5+ year old hardware in about 3 seconds.

There are however a few speedups for slightly quicker builds:

-bo=0 // This builds only one blockmap, with an offset of 0,0 instead of 36 different and picking the best.
-na=f // Use the fast algorithm for bsp, but this will often produce slower maps, possibly more visplanes.
-rz // Construct a valid reject resource, but fill it with zeroes.

 

[VGA]

Thanks for the replies. As for the your last reply though, you misunderstood me. I am talking about the best settings for the map's performance on ancient hardware, not the node builder's performance.

[zokum]

7 minutes ago, VGA said:

Thanks for the replies. As for the your last reply though, you misunderstood me. I am talking about the best settings for the map's performance on ancient hardware, not the node builder's performance.

For "final" builds of maps, try this:
-bo=3 // This usually builds smaller blockmaps, -o=4 has never made smaller blockmaps when I tested it. Other than that, just use default settings. This lowers the memory needed slightly, but it might be slower run-time, not noticable though. A bit more memory is probably better on 4-8 meg machines.

You really should go in and mark all the lines you don't need checked with the appropriate linedef type for blockmap exclusion. Smaller size and faster runtime performance. Same with the various seg reduction stuff. Pick the stuff that suits your intended port audience. Not all ports have good Doom compatibility.

If you have a lot of time and an ok amount of ram, probably 128 or more, and the map isn't very complex you can use the multi-tree algorithm. Building low-split trees is generally faster. You could just borrow a Linux based super computer with good single thread speed and let it churn through maps for you.
-na=mw=2m=s // low split
-na=mw=2m=b // balance of subsector and splits

This tends to reduce the final maps by about 5% in subsectors and splits, at least for the doom iwads. There's more information about this in my case study: http://doom2.net/zokum/zokumbsp/case-multitree

I am working on e1m6 on 15 year old hardware, and i've been at it for about 19 days and I am 53.39% done. This is on an older and slower build though. Absurdly long nodebuilder times might be a fun thing to put in your project txt file :)

[zokum]

I have found a nice optimization for the multi-tree algorithm. It shoudl reduce build times by a factor of 4 to 15, depending on the geometry. I had one map that used to take a bit over 13 hours, got built in 55 minutes.

I'll look into another minor console output issue and release 1.0.10b. You can just grab it from GitHub if you're really anxious about building maps with the multi-tree algorithm. I'll update more about it in the dedicated dev thread.

[Benjogami]

I really wanted to leave this topic be, but...

 

Quote

ZokumBSP should be able to handle everything ZenNode does and a bit more, due to fixing bugs. There's no point in using ZenNode anymore. There are several known bugs. Maps built with ZenNode will be built badly in most cases where you have diagonal walls etc.

 

Are you willfully ignoring the example I brought here? ZokumBSP chokes on maps that ZenNode builds fine, yet you continue to assert that "there's no point in using ZenNode anymore." Then why does ZenNode build my zekhmet map mostly fine, and ZokumBSP doesn't?

 

If ZokumBSP is only designed for vanilla maps, that's fine, but it's incorrect to say that it's just like ZenNode but fixed and better.