New (cross-port?) scripting languages for Doom

[Gez]

11 hours ago, JPL said:

 

• Engine and editor are wholly separate. Changes to maps/mods must be saved in-editor and then launched in-game.

Yeah, it is a problem compared to some modern games where everything is integrated. However, separate game and toolset remain frequent enough (e.g. the TES/New Fallout games) that I don't think it's a showstopper. Also launching Doom is remarkably fast; for mods that aren't like Paranoid or Blade of Agony we're talking about something that takes up less than 5 seconds even on a cluttered HD.

 

Also it was a historical inevitability. The first third-party editors appeared back when there weren't source ports yet after all, and Doom was just the game. Then when source ports appeared, a lot of them did and they went into separate directions, so the priority for people working on editors was to support the different engines that existed.

 

Still it would be interesting to see what could happen if you tried to fully implement the game into an editor to see what happens. But because starting the game is normally very fast, I'm afraid it'd mostly be a lot of bloat.

[JPL]

2 hours ago, Gez said:

Yeah, it is a problem compared to some modern games where everything is integrated. However, separate game and toolset remain frequent enough (e.g. the TES/New Fallout games) that I don't think it's a showstopper. Also launching Doom is remarkably fast; for mods that aren't like Paranoid or Blade of Agony we're talking about something that takes up less than 5 seconds even on a cluttered HD.

 

Also it was a historical inevitability. The first third-party editors appeared back when there weren't source ports yet after all, and Doom was just the game. Then when source ports appeared, a lot of them did and they went into separate directions, so the priority for people working on editors was to support the different engines that existed.

 

Still it would be interesting to see what could happen if you tried to fully implement the game into an editor to see what happens. But because starting the game is normally very fast, I'm afraid it'd mostly be a lot of bloat.

 

I think hot-reload of textures, sprites, and sounds alone would be a pretty large win. These are the assets you do lots of little tweaks to in your art/audio software, and changes to them don't affect game state so you don't have any of the issues you'd have with hot-reloading maps and script. If there's a real perf hit to regularly (~1Hz, no point in anything finer) checking files for changes, put the checks behind a "developer mode" cvar that defaults off.

 

Of course this is probably only applicable to ZDoom family, I always work with loose files in directories and forget other ports don't support that.

Edited July 10, 2017 by JPL

[Graf Zahl]

29 minutes ago, JPL said:

 

I think hot-reload of textures, sprites, and sounds alone would be a pretty large win.

 

That'd still require a nearly complete reload to properly initialize everything. If *something* may have changed, *anything* may have changed and needs to be checked. So you'd be back to square one (i.e. ZDoom's 'restart' CCMD.)

 

[Gez]

2 hours ago, JPL said:

 

I think hot-reload of textures, sprites, and sounds alone would be a pretty large win.

Graphics have a cascading effect (a texture can be used as part of another texture's definition). I don't think the engine keeps track of which "base" asset is used where, it just turns textures to an internal bitmap format when needed and keep them cached that way, so if you detect that one patch has been changed you pretty much have to reload every composite textures just to be safe.

[Linguica]

While it's not remotely what you are suggesting, it's worth noting that vanilla Doom had a little known feature that let you reload a map and see the changes in near real time.

[printz]

1 hour ago, Linguica said:

While it's not remotely what you are suggesting, it's worth noting that vanilla Doom had a little known feature that let you reload a map and see the changes in near real time.

How about going one step further and reload the map while it's being played whenever a change is detected? Preferably while preserving things as much as possible 😀

[Gez]

1 hour ago, printz said:

Preferably while preserving things as much as possible 😀

You'll have to come up with some comparison algorithm that's smart enough to recognize what has changed and what hasn't between two different versions of a map, knowing that there are no valid identifiers (something changing its index position doesn't mean it has actually been changed). And only after this analysis is done can you proceed.

[VGA]

I want the port to upload the wad to idgames as I am testing it and show me the feedback, so I can make improvements while in the game. Also, I want to press a button which pauses the gameplay and puts me in a map editor mode.

 

Also, I want the obsolete text-based Zscript to be deprecated and replaced with a WYSIWYG drag and drop interface that supports touch controls, so that I can interactively map using my Android tablet. It is the current year, after all, get with the times people!

 

 

[kb1]

(Responding to some earlier posts, as well as later ones)

On 7/4/2017 at 6:39 PM, Arctangent said:

I don't think it was even ruled out of the Boom+ effort, just ruled out as part of the first stage of it.

 

Also, it's kinda moot to discuss stuff without adopting port adoption, because ... I mean, if there's no port that adopts it, there's literally no way you can make use of it in the first place.

No, not ruled out, but considered very distant. Luckily, scripting can be thought of separately, with no overlap. Hence the new thread (thanks, Jon!) And, yeah: The first step is to define common functions needed by ANY scripting language. And, that's a lot of work itself, before ever thinking about language, technology, etc. Anyone serious about the idea would have to do that first.

 

On 7/4/2017 at 7:44 PM, Quasar said:

If you want to aim for something experimental then it's better to just "do" than to discuss. If you want to see, for example, a Lua-powered Doom port in action, then just grab Choco, Crispy, or PrBoom-Plus and start hacking. It shouldn't even take a whole day to have the basic embedding functional.

Yes. Sometimes, you have to build something, just to know how to best build it: Chicken and egg problem.

 

On 7/5/2017 at 3:00 AM, Graf Zahl said:

 

Would it?

We've seen how that will end. People get carried away with grand ideas that never see the light of day.

 

The problem both here and in the 'Boom+' thread is that crazy stuff gets tossed around, but there's nobody who actually makes a concerted effort to design a spec from the collection of ideas. That's just not going to work out and at the end we'd still be where we are now: We got several different ports, each doing its own thing with Boom as the common denominator.

 

Nobody willing to design a spec? What am I, chopped liver? I said it was going to take me some time. You're pressuring me to speed through this thing again. I am not going to provide a half-baked spec, regardless of the pressure. I need more time. We are a lot further than we were, already: We are deep into discussions and ideas. It's not magic. This is the process: Gather and compile ideas is the first step. The spec is underway. When it arrives, all of the ideas will be tested, with examples and working code. The spec is the last step, before implementation. And the rough draft is immediately before the spec. And, before that, there's a lot of testing, coding, tweaking, etc. Please be patient, and stop saying that no one is working on it. You know what's involved, and what it requires. Please give me a break.

 

On 7/5/2017 at 3:37 AM, Jon said:

 

This thread is pretty young. It's too early to write it off as going nowhere. Yes similar such thread often do get a lot of ambitious suggestions from non programmers that are pipe dreams, but that hasn't happened here yet, most of the participants (yourself, myself, ladna) are programmers and have some idea as to what is possible.

 

I think there's still hope for this thread and the boom+ one which is slowly coalescing. I plan to add some concrete info (linedef numbers, links to example implementations) to the wiki spec page sometime this week (assuming kb1 doesn't take it on first).

I don't mind the help, but I worry that our plans will conflict. This is a large project that I definitely could use some help on. I think it would work out better if I could get a preliminary spec sent to you, and if I could ask for your help in setting up that page. You mention linedef numbers. I have some ranges defined, that, based on some technical reasons, should be set up a certain way, That's what worries me about pre-specs. Would you be willing to wait a bit, until I can provide you some concrete stuff, and, would you be able to help by formatting that stuff into the wiki spec?

I'd also like to see some of your plans, so that maybe I can incorporate them into what I've accomplished, allowing us to collaborate.

I have always had some difficulty with the coordination of multi-user projects, but this project is of a size that I cannot easily handle all of it myself. Please, if you get a chance, send me a few of your ideas via PM, if you wish.

On 7/7/2017 at 7:10 PM, Blastfrog said:

To be frank, I don't see much use in debating whether or not to use ACS, the conclusion is obvious. If the point is to both be solid and widely adopted without issue, there is no other option.

 

There is absolutely nothing wrong with ACS, its purpose and scope is clear, it's excessively easy to work with and it does not have glaring issues or limitations like Fragglescript does for example.

 

Scripting does not need to be advanced in any way, it just needs to manipulate the levels and events during gameplay per the map author's intentions. I don't care about complex string manipulation or any kind of external filesystem read/write access when all I'm trying to do is print a message and raise a set of platforms in a certain order.

 

Anything outside of such a simple scope is better off as a new supplementary feature than to replace an existing and well-established one.

Comparing ACS and Fragglescript does not make a lot of sense - each one does different things, each have their own issues, each can be expanded and enhanced. But, I do agree: Having a well-designed byte-code VM-type backend is the obvious path to compatible scripting. And, with that being true, we do, in fact, already have such a system with the distinction of having been born in an idTech1 engine: ACS. These are huge hurdles to cross when justifying an alternative.

 

Alternatives are do-able, but they really should compile to byte-code. Here's why: With byte-code, each function can be thought of as being "in it's own black box". Function #3 is, for example, a PrintHUDMessage function in each and every port, that can be defined to work 1 and only 1 exact way, across ports. With the byte-code approach, this puts the programmer into this "compatible function" mindset where each specific byte-code instruction has it's own definition, it's own specs, it's own defined inputs and outputs, and it's specific purpose.

 

Starting with a different already-built, non-byte-code language, you start with a language that can already do 1,000 things, each of which may work a little different on different OS'es/platforms, bit-sizes, etc. In this scenario, for compatibility, you are working "in the wrong direction" towards compatibility, whereas, in the byte-code path, your engine is compatible from the start, and can be labelled "compatible" after each instruction you add. It's the smooth, goal-defined approach.

 

On 7/9/2017 at 9:28 PM, Arctangent said:

I kinda feel like this is an entirely different conversation. One strictly aimed at those who make the devs of map and lump editors, rather than the dev of source ports. There is still the important part of "does ACS or any other potential language translate well into visual coding," sure, but most of the discussion around it would need to be done by an entirely different set of people.

 

Though,

 

both of these are pretty well minimized by GZDoom Builder and I'm pretty sure Doom Builder as a whole for a pretty long time. There's a text editor built right into the program for the sake of writing ACS scripts and those scripts are compiled by an included version of ACC ( which can be updated manually, but is usually also straight-up updated with each version ) upon doing standard actions that would require updating the compiled lump.

You know, there's no reason that the compiler cannot live inside the game, ,as well, and compile the scripts on-the-fly. The "graphical editors" can output ACS, which could then be compiled by the map editor, or, in the future, right within the game. That's one way to go.

 

[kb1]

9 hours ago, Gez said:

Graphics have a cascading effect (a texture can be used as part of another texture's definition). I don't think the engine keeps track of which "base" asset is used where, it just turns textures to an internal bitmap format when needed and keep them cached that way, so if you detect that one patch has been changed you pretty much have to reload every composite textures just to be safe.

 

8 hours ago, Linguica said:

While it's not remotely what you are suggesting, it's worth noting that vanilla Doom had a little known feature that let you reload a map and see the changes in near real time.

Back then, this was a real advantage when it took 30-45 seconds to stop/start the game, and load up the map!

7 hours ago, printz said:

How about going one step further and reload the map while it's being played whenever a change is detected? Preferably while preserving things as much as possible 😀

Jokes aside, in-game editing is possible, with a lot of work: When switching to developer mode, all sectors get converted into Eternity/ZDoom polyobjects to allow for sector moving, resizing, adding, and deleting. Patches could have a "modified" flag, which gets propagated to a "texture modified" flag. Then, you unload the texture, and cause it to get reloaded and cached on next use. Scripts could be recompiled, though they might have to be invalidated if changed too much. Again, it can be done, but it's a lot of work. The building of the GUI would be very hard. And, you'd almost need a complete WAD editing package as well to pull in patches, so to save the map and scripts.

 

It's hardly worth it for a full edit capability. But, script editing and simple texture/flat replacement could definitely be done reasonably. Neat idea.

[Arctangent]

I think SRB2 actually has a limited form of in-game map editing, with the ability to spawn actors on-demand and then save all current actors to the wad. From what I recall, it wasn't actually terribly useful outside of creating NIGHTs levels, although it could still be useful if you're using a non-SRB2 editor for ring placement, as their height is hacked into their map flag field.

[Gez]

I could see in-game editing working as visual mode on steroids: you could spawn additional actors, change textures, change heights and light levels, but you could not move, insert, or delete vertices, lines, or sectors.

 

[kb1]

6 hours ago, Gez said:

I could see in-game editing working as visual mode on steroids: you could spawn additional actors, change textures, change heights and light levels, but you could not move, insert, or delete vertices, lines, or sectors.

 

I think you might be able to even move sectors around, if the edit tool were to convert the sector into a polyobject, hacking it out of and back into the BSP, for each sector you want to edit. You'd still want to have a final, clean BSP rebuild before releasing the map for performance, cleanliness, and compatibility. But, again, it's probably not worth the effort.

[Gez]

Polyobjects aren't sectors, though. They don't have ceilings or floors.

[kb1]

7 minutes ago, Gez said:

Polyobjects aren't sectors, though. They don't have ceilings or floors.

Ah. Forgot about that. Maybe that would be ok for edit mode, though? It'd probably work better for a GL port, I guess. A GL port could "fake it" during an edit session. It's an interesting idea, and yet, an idea that I'd be afraid to add to my project list, for sure :)

[Shadow Hog]

17 hours ago, Arctangent said:

I think SRB2 actually has a limited form of in-game map editing, with the ability to spawn actors on-demand and then save all current actors to the wad. From what I recall, it wasn't actually terribly useful outside of creating NIGHTs levels, although it could still be useful if you're using a non-SRB2 editor for ring placement, as their height is hacked into their map flag field.

Yep, definitely recall that being a thing - though IIRC it was more of a shout-out to the original Sonic games having a similar mode when you entered in certain codes. I never found it that useful, myself, since if I wanted to test the functionality of an object, I'd probably already made a small test map specifically for it. (And as far as testing, say, DECORATE objects in ZDoom goes, it already has the "summon" and "summonfriend" CCMDs.)

 

The idea of having an in-engine Visual Mode where you could rapidly change Thing placements and sector properties (maybe sidedef properties too, I dunno) would be interesting I suppose, but also sort of outside the scope of this thread, innit? That's hardly scripting. Besides which, it'd be more of a feature for an engine maker to do at their leisure than anything especially useful.

Edited July 11, 2017 by Shadow Hog

[Jon]

On 7/11/2017 at 4:42 AM, kb1 said:

I don't mind the help, but I worry that our plans will conflict. This is a large project that I definitely could use some help on. I think it would work out better if I could get a preliminary spec sent to you, and if I could ask for your help in setting up that page. You mention linedef numbers. I have some ranges defined, that, based on some technical reasons, should be set up a certain way, That's what worries me about pre-specs. Would you be willing to wait a bit, until I can provide you some concrete stuff, and, would you be able to help by formatting that stuff into the wiki spec?

Sure ok

On 7/11/2017 at 4:42 AM, kb1 said:

I'd also like to see some of your plans, so that maybe I can incorporate them into what I've accomplished, allowing us to collaborate.

I have always had some difficulty with the coordination of multi-user projects, but this project is of a size that I cannot easily handle all of it myself. Please, if you get a chance, send me a few of your ideas via PM, if you wish.

I'd prefer public discussion (in the other/correct thread if that's ok). I'll get the ball re-rolling with a simple question about one feature for now.

 

[kb1]

@Jon I had assumed, since you read that entire thread, that you might prefer a more streamlined approach. However, I'm good with public discussion, in the correct thread, of course. Please check that thread for an idea you might be interested in.

[LexiMax]

Positron has fixed up his reference document on BCS and it's actually readable now.  It's...quite the list.