k8vavoom: no good thing ever dies!

[Kafelnikov]

8 hours ago, ketmar said:

p.s.: this "bug" is called "episode finale". ;-) you finished episode 1, and got finale screen. this is exactly how it should work.

 

Ahah now I feel like an idiot, the map was a homage to Phobos Anomaly and it didn't cross my mind at all.

I'm gonna keep playing them wads using K8vavoom then, thank you and really great work with the port. cheers

Edited May 17, 2020 by Kafelnikov

[ketmar]

ah, don't mind it. i didn't get it too until i started the map and made may way to its end. ;-) and thanks for reminding me about this wad, i'll prolly replay it now. i mean, i'll test k8vavoom with it, of course, i don't have time to play games, only for fully testing pwads!

 

;-)

Edited May 17, 2020 by ketmar

[ketmar]

@Kafelnikov but there should be finale text, and it is missing. this is something i've been reported several times, and each time i'm forgetting to fix it. that's why you're so confused, i guess.

[ketmar]

@Kafelnikov i fixed finale text, so it should work as intended in the next public build. thank you for the report!

[ketmar]

@Gunrock btw, i reworked ACS HUD code:

Spoiler

 

[Gunrock]

Outstanding work!!!! Looks great;)

[ketmar]

eh. spent almost three days rewriting big part of VC compiler codegen. as Janis left us with no compiler tests, and my tests are far from having good coverage (i also found a bug in one of my tests, lol), let's hope i didn't fucked up too much.

 

technobabble for those who are interested in changes, for some strange reason.

Spoiler

VavoomC is not as easy as one may think. it has two data types which need to be managed: strings and dynamic arrays. both those things are dynamically allocated, and the compiler should take care of deallocating them automatically.

 

in the original VavoomC Janis was simply collecting all local variables in one big list, and emiting cleanup code right before `return`. that was elegant, and it worked. there was another little problem, tho: foreach iterators.

 

internally, foreach iterator is dynamically allocated too, and it should be freed after using. Janis did this by tracking all allocated iterators while the function was running, and killing them all together at exit. not the best way, but hey, it worked too, and it was simple. (actually, it was slightly more complex, because "proper" loop exit will destroy iterator by itself, and remove it from the list; but the maing thing is that those iterators aren't kept in any local variables, so they had to be tracked separately).

 

later, i added several more foreach types (iota, scriped, on array), and iterator management started to being unmaintainable. i wrote a very-very dirty solution with manual scope tracking in codegen, and it worked (to some extent), but it was REAL UGLY.

 

also, with the scheme Janis used, i couldn't implement stack space resusing. i.e. there was no way to reuse stack slot for different local variables in different scopes (due to destruction code emited before `return`).

 

i finally decided to remove my hack, and rewrite scope management code. internally, scope information is available, because of recursive nature of the compiler. i.e. block statement (`{ ... }`) has `Emit()` method, that will call `Emit()` methods for each statements inside the block. i.e. in each `Emit()` we have a perfect list of all "upper" scopes.

 

when i realised that fact (yeah, call me slowpoke and idiot here, you'll be right!), i removed my old hack, and changed local variable allocation scheme: now, if we have any locals, they are allocated at the beginning of `Emit()`,  and "released" at the end (with proper destruction, if necessary). this way i can reuse stack slots, have perfect knowledge of all active locals at any time (so i can generate proper destructor calls on `break` or `return`), and more -- i can track all foreach loops, and generate proper opcodes to deallocate hidden iteration objects.

 

the most frightening part here is "stack slot reusing". of course, i can turn it off, and simply go with the old scheme (never "releasing" stack slots i don't need), but it is not fun.

 

ah, if you don't know (and i bet you don't! ;-), there is no "uninitialised variable" problem in VavoomC, because all variables are guaranteed to be initialised to some default value (zero, `none`, `nullptr`, empty array, etc.).

 

so, with the old scheme i could simply `memset()` all the slots the function is using, and have my default values in place. but with stack slot reusing, some local could take a slot that is "dirty", so i have to detect that fact, and zero such slot.

 

sure, the easiest way is to simply zero each slot before using. but this emits alot of unnecessary VM opcodes, and each additional opcode adds to the execution time. so i had to write some logic to avoid unnecessary clears. this logic is quite complex (because there is no need to clear freshly allocated slot, for example, but only if it is not inside some loop -- because on each loop iteration the code expects all loop scope variables to be cleared), and it was hard to debug it (especially without the proper test suite).

 

and you know what? i found some really old bugs in the compiler while writing all this. "in-loop" variable clearing may sometime fail (yet i knew that, and wrote most of new k8vavoom VC code with that in mind, heh ;-), for example. and one of my tests that checks such thing had a bug in it, so it was actually checking if the bug is still in place instead of if it is gone. ;-)

 

anyway, the rewrite is almost finished now, i added some more tests for the bugs i found, and i am quite happy with the current compiler design. and we are one step closer to register VM. and when we'll have register VM in place, it will be MUCH easier to integrate libjit, to run VavoomC with native speed instead of in VM.

 

i don't know why you're still reading this, but i hope that my wall of text had at least something interesting to justify all your time wasted on it.

 

Edited May 21, 2020 by ketmar

[Martin Howe]

5 hours ago, ketmar said:

I don't know why you're still reading this

 

I'm a guy geek who asked GZ 'when are we getting property accessors in ZScript' - that answer your question? :p

Edited May 22, 2020 by Martin Howe

[ketmar]

5 hours ago, Martin Howe said:

I'm a guy geek who asked GZ 'when are we getting property accessors in ZScript' - that answer your question? :p

(whispering, but loudly) VavoomC is more powerful than ZScript! dynamic arrays of arbitrary types (including any user data types, and other dynamic arrays)! dictionaries with arbitrary key/value pairs (almost)! recently added methods for structs (not really needed with UFCS, but why not)! UFCS! paren-less function calls! references and pointers! four types of `foreach` iterators! string slicing! come close, it doesn't bite! ;-)

Edited May 22, 2020 by ketmar

[ketmar]

trying to figure out portal code. if you won't move the camera, it looks great. of course, it breaks horribly if you'll try to walk, but hey, at least i can make pretty screenshots!

 

Spoiler

 

Edited May 24, 2020 by ketmar

[ketmar]

the project is ok, and there will be another public build soon (i hope). right now i am working on SVG renderer based on heavily modified libart. i hope to use it to render UI for in-engine tools later. but don't worry, k8vavoom is not on hold, i already have a big changelog. i just want to implement some more things, and do some more testing.

 

tl;dr: don't worry. there is no strict schedule for public builds, but you'll get a shiny new one this month. do not adjust your set!

Edited June 11, 2020 by ketmar
synonyms

[Mr.Rocket]

Hey ketmar, any luck with the 3d slope issue where one side isn't being drawn yet?

 

~ I maybe off work for a couple of days and may have time to finish my map up finally, heh.

 

Thanks 

Edited June 16, 2020 by Mr.Rocket

[ketmar]

33 minutes ago, Mr.Rocket said:

Hey ketmar, any luck with the 3d slope issue where one side isn't being drawn yet?

nope, not yet, sorry. i found some other maps with this bug, tho. but i still can't understand wtf is going on. i'm pretty sure that it is something stupid, of course. it was much better when i was sure that it is simply some mapping bug, and not my fault. ;-)

[Mr.Rocket]

no worries, I'm sure something will stir up.. 

Only thing I can think of is if there's something having to do with its direction. *shrug.. but I have no idea where to go from there either.

Does k8vavoom rebuild the nodes on the fly differently and/or separately than other engines?

Let me reword that, does k8vavoom load a different node builder at run time than other source ports?

If so, which node builder?

 

Thanks

 

Edited June 17, 2020 by Mr.Rocket

[ketmar]

On 6/17/2020 at 8:00 AM, Mr.Rocket said:

Let me reword that, does k8vavoom load a different node builder at run time than other source ports?

If so, which node builder?

yep, k8vavoom always rebuilds nodes (there is still code to read nodes lumps, but it isn't even called). internal node builder in Andrew Apted's AJBSP. there is ZDBSP too, but i won't recommend it, it is either wrongly ported (by me), or too extravagant, but it often producing strange trees leading to rendering bugs. as the same builder works in GZDoom, i think that it's my fault, but nevertheless...

 

tl;dr: nodes are always rebuilt with AJBSP, and you cannot turn it off.

[ketmar]

it seems that my HDD is havind a hard time (according to S.M.A.R.T.). as new hardware is as realistic to me as flying to Mars by waving my hands in the air (yep, "do backups", i know; i cannot replace even one working HDD, so having another one for backups is... let's say, absolutely unrealistic), i cannot make any promises about k8vavoom now. i am still as passionate as ever, but it is hard to work on the project with hardware failing. especially if it is a data storage. sorry for inconvenience, but there's little i can do here. in the (absolutely) unlikely case of getting new hardware, i'll resume full-speed k8vavoom developement.

Edited June 20, 2020 by ketmar

[Dark Pulse]

16 minutes ago, ketmar said:

it seems that my HDD is havind a hard time (according to S.M.A.R.T.). as new hardware is as realistic to me as flying to Mars by waving my hands in the air (yep, "do backups", i know; i cannot replace even one working HDD, so having another one for backups is... let's say, absolutely unrealistic), i cannot make any promises about k8vavoom now. i am still as passionate as ever, but it is hard to work on the project with hardware failing. especially if it is a data storage. sorry for inconvenience, but there's little i can do here. in the (absolutely) unlikely case of getting new hardware, i'll resume full-speed k8vavoom developement.

An HD doing that is absolutely on its last legs. It's time to back it up on something - USB sticks, an external USB HD, BD-Rs, something - or up to the cloud.

 

Then yes, get a new HD. Replacing them is actually not as hard as you think it is though.

[ketmar]

1 hour ago, Dark Pulse said:

Replacing them is actually not as hard as you think it is though.

sure. but it is hard to replace it with nothing. i barely have money to buy food, so buying new hardware is a pipe dream. backuping my whole working environment to some "cloud" requires payed plans too (due to size, complexity and... sensitivity of many things), and i can't afford even a cheap hosting for k8vavoom site. everything i could made public is already public. so i can only sit and watch. alas.

[Dark Pulse]

33 minutes ago, ketmar said:

sure. but it is hard to replace it with nothing. i barely have money to buy food, so buying new hardware is a pipe dream. backuping my whole working environment to some "cloud" requires payed plans too (due to size, complexity and... sensitivity of many things), and i can't afford even a cheap hosting for k8vavoom site. everything i could made public is already public. so i can only sit and watch. alas.

Google Drive can at least back up the critical stuff, especially if it's stuff like source code.

 

Or some sort of Git/Mercurial/whatever for those.

Edited June 20, 2020 by Dark Pulse

[taufan99]

23 minutes ago, Dark Pulse said:

Google Drive can at least back up the critical stuff, especially if it's stuff like source code.

 

Or some sort of Git/Mercurial/whatever for those.

Yeah, I suggest Git a lot, since it also allows others to easily contribute, whether just by posting issues, requesting pull issues, or even forking projects.

Edited June 20, 2020 by InDOOMnesia

[ketmar]

56 minutes ago, Dark Pulse said:

Google Drive can at least back up the critical stuff, especially if it's stuff like source code.

source code is already public -- most of my projects are FOSS, so there is no reason to not upload them to public git hostings. ;-) the code is safe. but most of my other data is not, alas (20+ years of my life, basically). but eh, i'm not here to cry, i just wanted to warn people that if i'll suddenly disappear or something, it doesn't mean that the project is abandoned on my own will. and in the case of various failures i'll try to do what i can to recover and go on. the source code is public, so sooner or later either me or somebody else will be able to continue. of course, i still hope that this is just a slight deviation, and sector remapper will be able to recover, it happens sometimes. but if not... eh. there's basically nothing i can do about it besides what was already done.

Edited June 20, 2020 by ketmar

[Mr.Rocket]

Yes! make a google drive account, it's free, I use it for just this sort of thing!

I hope you're still with us.

 

rar or 7z your kvavoom source and get it on google drive man!

before your hdd is totally dead.

 

 

 

 

Edited June 20, 2020 by Mr.Rocket

[ketmar]

3 hours ago, Mr.Rocket said:

rar or 7z your kvavoom source and get it on google drive man!

the source is in public repository on repo.or.cz (the link is in the first post), it won't go anywhere. ;-) it's terabytes of my other (private) data i might lose, something i can't even upload because i neither have such internet connection, nor i can get several TB storage for free. ;-)

[ketmar]

as i will hardly finish everything i wanted for the new build in time (yeah, i wanted yet another some cool new feature nobody will notice; but it have to wait a little), i guess we'll go with what i have. the plan is to release a new public build this week. nothing really exciting there this time, it is mostly under-the-hood VC compiler rewrites and small bugfixes.

 

p.s.: sorry, the part of my brain that does English is slowly rotting. expect everything become more and more unreadable. at least until i'll have a chance for a good rest. ;-)

Edited June 22, 2020 by ketmar

[ketmar]

elves and ketmars are always stand up to their promises. so, here's new build for ya!

 

nothing really exciting this time. paying technical debts, and improving the compiler.

 

Spoiler

* VavoomC: structs can have methods now
* VavoomC: rewritten some compiler internals (should be user-invisible), fixed some old bugs
* VavoomC: added support for `scope(return)`
* VavoomC: allowed `return` in `scope(XXX)` body
* VavoomC compiler can report unused locals now (and i removed all unused locals from VC code; there were surprisingly few of them)
* scale intermission screen so it will become FS (thanks, id0)
* slightly better "transparent door" hack handling in clipper and renderer (engine can recognize more such special cases)
* shadow volumes now knows about "transparent doors" (but lightmap tracer is still believes that they're solid)
* changed OpenAL initialisation code a little, it should be more robust on windows (thanks, Mr.Rocket)
* added support for "hidden" UDMF sector flag
* faster lightmap tracer (the engine now using custom raycasting code instead of old general one, and avoids alot of unnecessary work)
* fixed some potential bugs with sound format detection (thanks, id0)
* fixed serious bug with ACS HUD messages (if the code creating alot of id-less hud messages, they could endlessly accumulate) (thanks, id0)
* restored finale text for Doom 1 (thanks, Kafelnikov)
* partially implemented ACS `Warp()`
* implemented (almost) proper ACS HUD scaling
* added "pseudo-3d" sprite fake shadow mode, and two possible orientations for it (this mode is default now, i think it looks better than 2d fake shadows)
* if the map has "lightning" flag set, but didn't defined the second sky, make both skies the same
* reworked code for rendering stacked sectors and mirrors (still glitchy, but at least it is "beta than nothing"; allowed by default, but expect ALOT of visual glitches)
* added support for gzdoom md3 submodel skinning (thanks, id0)
* implemented `A_SetSpecies()` decorate action
* added support for `Player.UseRange` decorate property
* added shadow volume optimisation that can win 1-3 FPS on maps with huge lights (some floors and ceilings cannot cast shadow, and so they aren't rendered as shadow casters anymore)
* some small optimisations and bugfixes in vertex merging code (udfm, nodebuilders) (should not be user-noticeable)
* fixed bug with texture scroll line specials (flags were ignored, so it always scrolled all textures)

 

[ketmar]

@-TDRR- promised ping. ;-)

[Cambertian]

Sorry if this has been brought up already, but the weapon on the HUD warbles and glitches anytime I move the mouse.

Video example: https://imgur.com/a/IWx6teO

On an NVIDIA card (RTX 2080ti) if that somehow helps narrow things down (for instance, if you've only been testing with an AMD card.)

[Cacodemon345]

2 hours ago, Cambertian said:

Sorry if this has been brought up already, but the weapon on the HUD warbles and glitches anytime I move the mouse.

Video example: https://imgur.com/a/IWx6teO

On an NVIDIA card (RTX 2080ti) if that somehow helps narrow things down (for instance, if you've only been testing with an AMD card.)

Ketmar uses a Nvidia card. Maybe an issue with your drivers? Remember that he uses Linux too.

[Gunrock]

I had that problem too. Strange however, it was fixed when i updated my gpu drivers and downloaded the latest K8Vavoom build.

[ketmar]

22 hours ago, Cambertian said:

Sorry if this has been brought up already, but the weapon on the HUD warbles and glitches anytime I move the mouse.

this definitely should not happen. as Gunrock said, it looks like a problem with drivers. i have nVidia too (with the latest GNU/Linux 32-bit drivers they provide, i.e. quite outdated ;-), and it works for me. i also got reports of missing water (which is present on my system). it looks like nVidia did something wrong with their windows drivers again.

 

i'll look at k8vavoom code too, but tbh, there is nothing fancy going on there: just your standard quad rendering. and on your video it looks like something is wrong with vertices. if my code is doing them wrong, then it shold be broken for everyone (including me). so it is something with videodrivers.

 

i switched to VBO rendering not a long time ago, so it may be a VBO issue. i am heavily reusing VBO buffers -- which is fine by the specs, but the driver could have some different opinion. ;-)

 

still, the very same VBO is used for all sprites, including monsters (it is even the same function), so i haven't a slightest idea why it works with normal sprites, and doesn't work with psprites. the only difference between them is drawing coordinates.

 

tl;dr: sorry, i don't know. i will prolly try some workarounds, but as i have no such bug, i can't really debug it.

 

p.s.: i am testing new builds with Wine, and everything is working as expected there. this is not a real windows system, of course.

Edited July 3, 2020 by ketmar