Wad compatibility option database - ideas

[RestlessRodent]

I implemented a very basic XML parser in ReMooD, it loaded up XML and it worked, was only about 100 or so lines of code. However, I removed the code since I decided to bring back RMOD and use that instead.

[kb1]

DaniJ said:

It would basically be up to individual implementers what local representation they decide to use. If their preference happens to be JSON then thats just fine.

This.

And, here's some more for the list (in no particular order):

- lookup/lookdown/autoaim
- all lines passthru
- final doom teleport height bug
- jump allowed
- crouch allowed
- nightmare respawn at 0,0 fix
- MBF monsters avoid dropoffs
- invincible_ghosts (some would argue)
- P_FindHighest.../P_FindLowest... fixes

Not a complete list, but it's a start.

[DaniJ]

kb1 said:

lookup/lookdown/autoaim

These are not map compatibility options, they are user preferences. How can any of those "break" a map?

Personally, I can't understand why anyone would want to export the central database, transform it into an even more complicated representation, which you'll then need a lexing parser to consume, before then transforming into something useful for runtime use. Seems like a big round trip with no net gain. Oh well, what do I know :)

@GhostlyDeath
I wouldn't even attempt to write an XML parser unless for educational purposes. I certainly wouldn't want to state XML support if I knew my implementation wasn't standards compliant, so I think you made the right choice by removing it.

[kb1]

DaniJ said:

These are not map compatibility options, they are user preferences. How can any of those "break" a map?

It's pretty obscure, but imagine a map where a G1 or GR switch was placed so high or low (or awkwardly) that you'd need to free-aim to shoot it. Come to think of it, it could definitely be left out of the spec, since even though it would "break" the map, you'd just turn the option on right then (as long as you're playing single-player, or your port allows option changes to be propagated across nodes in multi-player).

An option of this type would fall into the "required" or "don't care" categories, but boolean could handle it as "Freeaim required?"

Doesn't bother me to leave it out. I simply did a (rather hasty) sweep of my source looking for options I use, that could possibly affect the playability of a map. However, my list is incomplete, because I lump a lot of options into categories based on vanilla demo versions (Doom 1.2, Doom 1.666, Final Doom, etc.) Other ports break these up more suitably, and, if the "wad compatibility db" actually becomes a reality, I suppose I'll have to support the individual options as well.

[Vermil]

kb1 said:

It's pretty obscure, but imagine a map where a G1 or GR switch was placed so high or low.

Indeed, Doom2 Map16 features a G1 switch, to access a secret, that you have to use a lift to be able to shoot without look up/down.

There’s also of course Doom2 Map30 (and many other IOS variants).

Personally, as a player and modder, I think the database should go all the way and cover every game play difference to Vanilla Doom that it can. This includes additional or altered controls IMO; jumping, crouching, looking up/down, Heretic/HeXen non full looking up/down.

Even auto aim being on/off can change the game play; I have made map set ups where I expect auto aim to be on.

I'd imagine all ports will have options for the user to overrule the database on any or all compat options if they wish. Also, if a certain compat option really goes against a ports ‘ideals’, the port can choose not to support it.

That said, I'd imagine that discussion about what compat options should go on the database and how a port interprets them, is secondary to the actual setting up of the database.

[Graf Zahl]

Don't get too intrusive.

A compatibility database should specify what options are needed to play a map, not options that may affect gameplay.

Anyway, I think that this entire discussion has completely lost its focus. I don't see it going anywhere if it continues as it is now.

[DaniJ]

I quite agree. Attempting to leverage this mechanic to cover every possible option in any port which may in any way alter gameplay compatibility would be a self-defeating endeavour. Conceptually all options alter gameplay at some level or another.

What matters are the important, game changing options that fundamentally alter game behaviour in subtle but important ways, such as the use of BOOM's method for finding next nearest plane heights.

I think we may be overlooking an obvious path - limit the option set to those found in PrBoom which are required in order to return it to vanilla-exact behaviour, including those for demo support. The BOOM featureset has become almost universally accepted as a canonical evolution of the DOOM base but in the process of which, numerous incompatibilities resulted. If we can address this with a mechanism supported across all modern DOOM based ports I think we will have succeeded.

[Graf Zahl]

Quite right. But once that works one could still selectively add a few options that are important for other ports. For ZDoom there's precisely 2 additional ones that I consider important enough to put in such a database ('Use original soundtarget handling' and 'self referencing sectors don't block shots') None of the other ones is critical for vanilla/Boom maps.

[DaniJ]

Care to explain a little more about what those options actually do and why they became necessary?

[Graf Zahl]

Doom originally uses per-sector soundtarget storage. ZDoom at one point changed this to per-actor because the original method has issues when spawning monsters by script and makes them wake up immediately when spawning in a sector where any player has made a sound at any time.

This can cause problems with monsters that are teleported somewhere by a conveyor belt without being alerted by sound before. I admit that such Boom maps are rare but they exist.


The second option relaxes the hitscan trace code checks for self referencing sectors. Since id never considered such constructs the hitscan code never tested for them. ZDoom's code is a lot more thorough (it has to be for some stuff, like 3D floors) but it causes some issues with a few WAD tricks, for example Alien Vendetta MAP15.

[DaniJ]

From your explanations I think both options are rightly suited for inclusion. The sound target one in particular is something I've encountered myself for script related reasons.

We should take care to generalise as far as possible when defining each flag. In general a logical option which says "use vanilla logic for XYZ" should be sufficient regardless of what implementing that means for a given port.

The only thing I'm wary of going forward is whether this system will fall foul of being used as a crutch - "it doesn't matter if I change the playsim in portX because I can always request a new compat flag". I don't want us to find ourselves in a situation where the compat database includes dozens of flags specific to a particular port. Should there be a review board for this?

[Graf Zahl]

In ZDoom I only add new compatibility options when there's a serious reason or incompatibility with Boom/Vanilla (or one of the other games) that requires it - definitely not as a crutch for lazy mappers.

If absolutely unavoidable I will deal with such issues in the local compatibility.txt if they get reported. The last such issue was a malfunctioning voodoo doll effect in UAC Ultra. I fixed that one by manipulating a linedef special after loading the map. But such special options should definitely not be in the database. This malfunction was a result of sloppy testing, not a real compatibility concern.

[kb1]

Agree with PrBoom compatibility (but use PrBoom+ as it has more options to get it even closer). Also, the 2 Zdoom options make sense.
Still a few port authors have not commented. No reason to limit our list to just PrBoom+ and 2 ZDoom options. It would be nice to have more input on gameplay breaking options.

Now, the kicker:
I believe the 1-bit approach will be problematic, and yes, I understand and appreciate the beauty and simplicity of that approach.

Here's why:
1. Data entry issues - If you present an HTML form with, say, 16 checkboxes for 16 options, it's likely that, for a particular map, only a few are actually required, and, in fact, the reporter is probably submitting an entry because one option broke his/her game. That reporter may have no clue about the other options, and, in fact, they may not even be important. In a perfect world, we would be able to choose each option wisely and document each, but in practice, I think the data entry will be closer to the scenario above.

2. Defaults/Future options/old datasets - How do you initially set the bit string? All zeros? All ones?

If you've defined "0" to mean "go vanilla", then by default, every entry dictates vanilla compatibility. You wouldn't be able to have an uninitialized default entry in the db at all, if the map requires a Boom feature.

If you've defined "0" to mean "go boom", all vanilla maps would need "1" in each and every field. Would your HTML show "all checked" for a new entry?

And, what about possible future options being added? Do they default to "0"? If I download the latest version of a port, will my outdated local db turn off that port's new features?

3. Picky Users - I don't want a compatibility database setting EVERY option for EVERY map, only the options that are really required to get through the map. Even if you can change them in-game, my God, you might have to keep changing such options manually for each new map load, which sucks as bad as having a broken map.

4. Easy to fix - To alleviate this problem, I propose a option bit storage format as follows:

. When presented or stored in a db/file, the option string is in hex.
. For each 8 options, there will be 2 bytes. The first byte contains the 8 option states. The second byte is a mask that dictates which of those options are set.
. A "1" in the options means "use vanilla behavior".
. A "1" in the mask means "apply this option".
. Options are written to the byte in ascending bit order (option 1=bit 0, option 8=bit 7, etc.)
. Option Byte/Mask Byte sets are written left-to-right in memory and in the hex string. This allows easy expansion while letting older db downloads work as good as possible:

optionstring "213A82C2"


Byte 0: 2A = 0 0 1 0 1 0 0 1 (options 1, 4, and 6 set to vanilla)
Byte 1: 3A = 0 0 1 1 1 0 1 0 (mask 2, 4, 5, and 6 = apply option)

Result:    = ? ? 1 0 1 ? 0 ? (option 2+5=non-vanilla, 4+6 vanilla, others (?)=untouched)     

Byte 2: 82 = 1 0 0 0 0 0 1 0 (options 10, and 16 set to vanilla)
Byte 3: C2 = 1 1 0 0 0 0 1 0 (mask 10, 15, 16 = apply option)

Result:    = 1 0 ? ? ? ? 1 ? (option 15=non-vanilla, 16=vanilla, others (?)=untouched)     

I think this idea provides a great deal of flexibility, upwards compatibility, and is still elegant and easy to store. The interleaved byte/mask scheme allow for variable-length, compressed hex strings (you can truncate any trailing 0000's). This also suggests that the most damning options should be placed first in the list, to support small option strings. In this scheme, 32 characters gets you 128 options, and if you order them wisely, the strings can usually be much smaller.

The "HTML form with checkboxes" is still easy to implement:

[X]Apply this option: [X]Use doom stair building method

The checkboxes map directly to the option string.

[DaniJ]

kb1 said:

Data entry issues - If you present an HTML form with, say, 16 checkboxes for 16 options, it's likely that, for a particular map, only a few are actually required, and, in fact, the reporter is probably submitting an entry because one option broke his/her game. That reporter may have no clue about the other options, and, in fact, they may not even be important. In a perfect world, we would be able to choose each option wisely and document each, but in practice, I think the data entry will be closer to the scenario above.

This is not an issue with the format itself. Data entry issues should be addressed in the proper place, i.e., the form or HTTP request used to submit your change to the database. As such the method used for picking default values is outside the scope of the format. Although I would personally suggest a mask-based mechanic which allows the user to choose the bits they wish to change in addition to their new values.

Defaults/Future options/old datasets - How do you initially set the bit string? All zeros?

Yes, the initial state of a hash would be [SHA-1|mapidx|0000...) for a map not yet present in the database. Any new option added at a later date would have to conform to the convention that 0=vanilla behaviour.

You wouldn't be able to have an uninitialized default entry in the db at all, if the map requires a Boom feature.

An un-initialized state would not be needed with the convention that an unset bit means vanilla behaviour.

3. Picky Users - I don't want a compatibility database setting EVERY option for EVERY map, only the options that are really required to get through the map. Even if you can change them in-game, my God, you might have to keep changing such options manually for each new map load, which sucks as bad as having a broken map.

This is beyond the scope of the metadata format. Its an implementer-domain issue.

Edit: One of the benefits of a char-per-bit representation is that standard RLE compression can be applied to the hash in its original form. Given a map that is primarily vanilla-compatible, with the full form [SHA-1|mapidx|000000000000101) would end up looking something like; [SHA-1|mapidx|0^12.101), which is a decent 25% reduction in the size of the fully expanded form. Obviously the more thought we put into ordering the bits in the hash the better, as we can maximize compression potential by grouping options logically. With care we should be able to achieve at least this compression rate on 95% (figure plucked from ass but seems plausible) of the maps in existence. With the convention that a 0 bit=vanilla, then a map that is purely vanilla can be represented wholly in 192 bits.

Now, I know somemany might think I'm taking this all too far but as example; a million maps of which 80% are vanilla compatible could have their metadata condensed to a (guesstimated) 25.MB of storage using nothing more than first-order compression techniques.

I don't particularly mind the idea of packing bits into words but then as you noted, you end up interleaving metadata within the metadata, resulting in additional complication at runtime or the requirement to pre-process the hashes.

[kb1]

DaniJ said:

This is not an issue with the format itself.

It absolutely is, I think you misunderstood me. It's not about data entry errors, it's about making partial entry a valid option. Why should the record set dictate that I cannot have monsters get stuck in doortracks if I want them, and it doesn't affect gameplay?

It's not a data entry issue, it's a format issue. You're not allowing the flexibility I envisioned, by only supporting "turn it on or off". What about "leave it alone"?

DaniJ said:

...An un-initialized state would not be needed with the convention that an unset bit means vanilla behaviour.

That forces the entry to positively define each option as on or off. And, it imposes "vanilla" on any new options, whether or not the record defines, effectively killing any port's new abilities that get a dedicated bit for that ability, until EVERY db record can be modified to include said port's option bit, PROPERLY populated to accurately reflect the needs of any potential map.

DaniJ said:

This is beyond the scope of the metadata format. Its an implementer-domain issue.

Yes, but not beyond the scope of my proposed format, which supports fixing option that need fixing, and leaving alone options that don't.

DaniJ said:

Edit: One of the benefits of a char-per-bit representation is that standard RLE compression can be applied to the hash in its original form. Given a map that is primarily vanilla-compatible, with the full form [SHA-1|mapidx|000000000000101) would end up looking something like; [SHA-1|mapidx|0^12.101), which is a decent 25% reduction in the size of the fully expanded form. Obviously the more thought we put into ordering the bits in the hash the better, as we can maximize compression potential by grouping options logically. With care we should be able to achieve this compression rate on 95% (figure plucked from ass but seems plausible) of the maps in existence.

I don't particularly mind the idea of packing bits into words but then as you noted, you end up interleaving metadata within the metadata, resulting in additional complication at runtime or the requirement to pre-process the hashes.

I get the feeling that you've been intentionally argumentive during this thread (but I could be wrong)... Run-length-encoding? Are you kidding? I was talking about saving a few bytes basically for free, not requiring something over-engineered. What's wrong with an option byte followed by a mask? Please explain how that's complicated? All in all, I don't particularly care how it's represented, as long as the data is there. But, no, 4 hex chars-per-8-options-with-masking is NOT difficult, and I think you'll find it's more compressed in general than RLE anyway, when you drop trailing zeros.

Edit: In my format, the above would be represented:

03 03

If you felt had to use binary, you'd have to interleave bits in my format, making it:

11 01 01 left-most bit=option 1, 2nd bit=allow

4 or 6 chars vs. 8 :) (who cares?)


Hell, I already gave up more than half of what I had envisioned could be possible and simple:

. An .ini file matching the wad filename with one-line-per-option: "Hard to parse!" "Takes too much space" "Too many files". Ok??

. A combined .ini file with both filenames and hashes: "Too slow to read" "unwieldy" "Can't open in Excel". Ok??

. Inclusion of 'non-options' that would assist in preventing a broken map from being played, like "not_coop_friendly": "Too hard to define" Ok??

I let those things go, because the general goal could still be more or less satisfied in some fashion, as long as options could be set to 'don't change'.

Do you seriously believe that:
1. ...he who enters the data into the HTML form will have tested each and every option, and will have determined that each and every option must be set exactly this one way, otherwise the game experience will suck?

2. ...every conceivable compatibility option must be set to either Yes or No, otherwise the game will play incorrectly?

3. ...any future option that gets added should be set to 'off?' in a mandatory fashion?

4. ...users will appreciate having to change their favorite option on each and every map load, even if the game plays ok regardless?

Look, man, there's no hard feelings for me here - I know you write tons of sweet code and all that, so I'm wondering how you can't see the valid argument for masking options. For me, without it, it's a deal-breaker - I simply can't use the data if it doesn't accurately portray which options are absolutely required, and which are not. What am I missing? What possible valid reason is there for not including it, when there's so many reasons to have it?

[DaniJ]

Before I get around to addressing the points you raised let me just state; no, I have not been deliberately argumentative towards anyone in this thread. I have taken due time to consider each proposal on its own merit and personally, I think I've been pretty pragmatic.

However, I cannot remain silent in a technical discussion going wrong, whose entire purpose is to produce a product design we all intend to use. This isn't a competition to decide who can design the best format. Personal attachment to a given design shouldn't even factor.

kb1 said:

It absolutely is, I think you misunderstood me. It's not about data entry errors, it's about making partial entry a valid option. Why should the record set dictate that I cannot have monsters get stuck in doortracks if I want them, and it doesn't affect gameplay?

It's not a data entry issue, it's a format issue. You're not allowing the flexibility I envisioned, by only supporting "turn it on or off". What about "leave it alone"?

I don't think I misunderstood at all. As far as I can see, nobody here has stated that transactions with the central database would require the user to specify all properties every time they wish to make a change. This mechanic would naturally need to be integrated into the interface to said database but it doesn't need to be present in the metadata representation.

How can the metadata tell each implementer which options they should ignore and which they can't? This is entirely in the domain of the implementer and will vary for each application and each user.

And lastly; run-length-encoding is basically free. You can read from RLE compressed data in linear time with no need for masks, micro-metadata or other such mechanisms while still achieving comparable compression rates. In short, we can do this dynamically tens of thousands of times a second during live gameplay if need be. Furthermore, RLE-reads can be implemented sequentially using one continuous seek without having to stop and process the pieces. However the point I was trying to illustrate was not that we should use RLE, rather, that the "full" form presents more compression options.

I'm not trying to make this complicated. I am however trying to leverage what I feel are the right techniques in the right places to delicately balance the rather disparate set of requirements placed on it.

I realize you couldn't care less about the size of this data but you aren't the one who'll have to foot the bill for the overheads.

[kb1]

DaniJ said:

I have not been deliberately argumentative towards anyone in this thread. I have taken due time to consider each proposal on its own merit and personally, I think I've been pretty pragmatic.

Ok, sorry, fair enough.

DaniJ said:

However, I cannot remain silent in a technical discussion going wrong, whose entire purpose is to produce a product design we all intend to use. This isn't a competition to decide who can design the best format. Personal attachment to a given design shouldn't even factor.

Yes, but please be "right" when defining "wrong". No competition, but, I do value my ideas - I've designed a few pretty slick systems of my own in my >34 years of hacking. One things I do take pride in is knowing when a structure is missing something vital.

DaniJ said:

I don't think I misunderstood at all. As far as I can see, nobody here has stated that transactions with the central database would require the user to specify all properties every time they wish to make a change. This mechanic would naturally need to be integrated into the interface to said database but it doesn't need to be present in the metadata representation.

Yep, you missed it a bit :) It isn't just about the database transaction being incomplete. Specifying "this option does not need changing" is valid, useful data.

DaniJ said:

How can the metadata tell each implementer which options they should ignore and which they can't? This is entirely in the domain of the implementer and will vary for each application and each user.

Please give an example of an option that the implementer would want to disobey, given these 3 choices: "On", "Off", or "leave it alone". I can see a port not understanding a certain option, but we are discussing giving the port "as much information as possible to have a functioning game, and, hopefully, only that much information". The mask gives more power back to the port and the user - all it does is limit the number of flags that are deemed "critical". I'm pretty sure no user wants to get half-way through a game, to learn that some critical setting is preventing their progress. However, that same user will certainly have some preferences. The mask allows more of their preferences to be used, without having to override anything. That's the whole point.

Once again, my only hangup here, is that your format makes each option Mandatory Off or On (for all options the port understands, obeys, even though the port could allow a override...messy). But, that's more power than the metadata usually needs.

Use the "monsters stuck in doortracks" example. Assume that we've agreed that this is flag #3 (unlikely, but humor me). Let's say that, for a particular map, this option doesn't make a lot of difference - this particular map plays good with or without it being set. Let's examine some scenarios:

Scenario 1: Port 1 understands this flag, and expects your proposed metadata string (the boolean-only system). Port 1 can either (A) ignore the metadata and use the user-preference (no need to implement metadata at all), or (B) Unconditionally set the game flag to True or False, according to the metadata. Note that, the option is not important for this particular map, but it's been forced on or off anyway. Optionally, the port may allow some method to override. However, almost every map in the metadata set will define at least 3 flags, so this flag will almost always be defined, and, as a user, I have now lost control of my preference. Sure, Port 1 may allow me to override, maybe even "always override", but, for this particular flag, I've just rendered any benefit the metadata provides useless.

Scenario 2: Port 2 understands this flag, and expects my proposed metadata string (boolean+mask). Port 2 can, like Port 1, ignore the metadata and use the user-preference (once again, not very helpful), but instead, port B, seeing that the mask bit was not set, decides to leave this option the way the player wants it. This is ok, because the metadata stated that this flag has either never been populated in the metadata, or was deemed "not necessary" to set by those maintaining the central database.

That's what I've been trying to state. I don't know how to clarify it any more. I find it invaluable to make that distinction, and it actually makes partial data entry easier, and the ports automatically become less obtrusive at the same time, which is more inline with my original goals: An easy, non-offensive way to improve playability, without the user having to fiddle with tweaking this or that, if if you're a power-user.

DaniJ said:

And lastly; run-length-encoding is basically free. You can read from RLE compressed data in linear time with no need for masks, micro-metadata or other such mechanisms while still achieving comparable compression rates. In short, we can do this dynamically tens of thousands of times a second during live gameplay if need be. Furthermore, RLE-reads can be implemented sequentially using one continuous seek without having to stop and process the pieces. However the point I was trying to illustrate was not that we should use RLE, rather, that the "full" form presents more compression options.

RLE is just about as free. But, it's not really the "full" form anymore than the hex. And, it's bigger :) But, seeing the "ones and zeros" may help developers - I'll give you that. Once again, I was simply proposing a simple format that inherently has some slightly valuable benefits. Could be just as acceptable in a number of forms.

DaniJ said:

I'm not trying to make this complicated. I am however trying to leverage what I feel are the right techniques in the right places to delicately balance the rather disparate set of requirements placed on it.

Ok, I see that. I want it simple too. Just not too simple to get the job done. The mask allows a more-delicate approach, IMO, than setting every conceivable option each time. Please consider the cost/benefit/risk a bit more. I'm going to give it a rest for the weekend.

[DaniJ]

I now realize how this contention came about. It seems to me like you are hoping to solve two entirely different problems using the same data representation.

Problem A:
Compact yet runtime-useful representation that can be quickly parsed, indexed and sorted.

Problem B:
Managing user preferences governing the interpretation of problem A at implementer level.

Problem A is the only one considered by my proposal. I don't think it would be even practical to try to solve B) using the same format. What I personally had envisioned for B) is a lot more complex than this, involving cascading preference rulesets etc...

[kb1]

DaniJ said:

I now realize how this contention came about. It seems to me like you are hoping to solve two entirely different problems using the same data representation.

Problem A:
Compact yet runtime-useful representation that can be quickly parsed, indexed and sorted.

Problem B:
Managing user preferences governing the interpretation of problem A at implementer level.

Problem A is the only one considered by my proposal. I don't think it would be even practical to try to solve B) using the same format. What I personally had envisioned for B) is a lot more complex than this, involving cascading preference rulesets etc...

Really, all I wanted was to reduce the impact of Problem A - that's all. The "runtime-useful" data describes, say, 25 options, but, for Map X, it only needs to describe 10, for Map Y, it only needs to describe 2.

[DaniJ]

I can certainly appreciate that motivation, however I personally believe that a combined solution wouldn't be as robust and ultimately less flexible than if the two problems are handled separately.

By separating them it affords implementers the opportunity to decide how they want to tackle B), which they can tailor to needs of their application.

A map editor for example, wouldn't really be interested decoding from the representation used in the proposed solution for A). In this use case there is far more emphasis on end user usability and very little on data size. A possibly elaborate, project-like solution to B) would be preferred and instead, only supporting representation A) in the final bake-and-package stage of mod authoring.

[kb1]

But, how do you prevent unnecessary options being set? The initial .ini approach automatically handled this:

If only 2 compatibility options were necessary for a map, the .ini would contain 2 entries.

With your maskless approach, ALL compatibility options are set every time, by default. That's as bad as having no metadata at all.

Say we define 30 options. Say I play a map one day, and I realize that, for this map to play properly, "all lines passthru" needs to be set. So, wanting to do the right thing, I go to the wad-compatibility-db.com, type in the wad name, SHA-1 hash, and map number. How do I state that "all lines passthru" is required? More importantly, how do I state that, as far as I can tell, "all lines passthru" is the ONLY requirement?

[DaniJ]

That logic would be handled in the implementation of the web form (and HTTP request interpreter) presented to you by the interface to the compatibility database. You would first select the properties which you wish to change (possibly implemented as checkboxes) before then providing new values for them (a further set of checkboxes). The interface would then combine your changes with any previously existing hash, masking out the values for the properties you don't wish to change. The new complete-hash would then be inserted into the database*.

* Over simplification. This system would require some mechanism for resolving conflicts (perhaps with wiki-like community management sensibilities).

[kb1]

DaniJ said:

That logic would be handled in the implementation of the web form (and HTTP request interpreter) presented to you by the interface to the compatibility database.

No, that's not what I'm asking. How will the source port know that only one option is to be changed?

[DaniJ]

Perhaps I'm missing something but I don't understand what you mean. A sourceport would only ever be presented with a hash that is complete or foreshortened (thus complete by definition due to 0=vanilla).

In what scenario would a sourceport need to interpret a partial hash change other than if it provides a built-in mechanism to interface with the database via HTTP (in which case it can resolve the result of the change also)?

[kb1]

DaniJ said:

...A sourceport would only ever be presented with a hash that is complete or foreshortened (thus complete by definition due to 0=vanilla)...

That is exactly the entire issue I speak of. Why would I want the hash to dictate all options, when only a few may be needed to make the game play well? Hence the need for a mask. I only want the hash to dictate the subset of options that are really needed for a given map, not all of them. I can't state it any other way, I've given lots of scenarios above. It's a simple concept.

[DaniJ]

Then mask them out yourself. Only YOU as the implementer know which values are applicable to your application, therefore only you can define your compatibility mask(s).

[kb1]

Per map? The mask I speak of belongs with the database. Of course, as an implementor, I will decide if I support an option or not, and, if I do support an option, I want to only set it True or False if the hash says that it's important.

Ok, here's my last attempt at explaining: An analogy:

Directions for cooking toast, using my proposal:
1. Take 2 pieces of bread.
2. Put them in the toaster.
3. Turn toaster on.
4. When toast pops out, take them out.

Directions for cooking toast, using your proposal:
1. Take 2 pieces of bread.
2. Put them in the toaster.
3. Turn toaster on.
4. When toast pops out, take them out.
5. Do not add jelly.
6. Do not fold bread.
7. Do not cut bread.
8. Place bread on an 8 inch plate.
9. Eat immediately
10. Pour a glass of milk.
11. For every bite, wash down with said milk.

etc... Options 5 thru 11 are simply not necessary, and not applicable to all people. But, it can be decided that, if Options 1 thru 4 are not chosen, your toast experience will suck.

In your proposal, options 5 thru 11 MUST BE defined either Yes or No, even though they are not required. However, Option 5 *could* be required on a different set of directions, like cooking pizza.

In my proposal, I would mask out options 5 thru 11 in my toast cooking directions, which would leave them up to the cook to decide. If the cook doesn't have jelly, he can choose to omit it.

So, I ask, how can I cook toast, and have jelly on it at the same time? With your idea, I cannot, OR I'm forced to get it each time (unless I switch options manually each and every time, which defeats the purpose.)

With my idea, the "jelly" options stays the way the user wants it.

Back to the main discussion, No, I cannot "implement the mask myself", because the mask is different for each map.

[DaniJ]

Honestly I'm completely lost as to how this could be an issue for you.

Half the point in choosing a mask-able representation is to allow doing things like this in as few instructions as possible.

Edit: Is it just me? Is there anyone else reading this thread that can identify the problem kb1 is presenting?

Edit2: Huh? The mask belongs to the database? Since when? The whole point is that the "absolute" hash (i.e., same for all supporting sourceports) is in the database, while the mask(s) exist in the implementer's domain (sourceport, editor, whatever...).

[Moriarti]

Sorry, but this is just getting ridiculous. Honestly, I'm completely lost as to how you keep missing the point.

Suppose you have 10 compatibility options. Now suppose you have a map that needs one option to be set a certain way in order for it to function properly. The other nine options are completely irrelevant. They can be on or off, and the map will still work fine with any of their 512 possible combinations, so long as the one option that matters is set correctly.

What kb1 wants to know for this map is which one option needs to be set for the map to work. The port can then set that option as required. The other 9 will be left alone at whatever the user prefers.

What you propose is to completely disregard the users' preferences and force all 10 of those options a certain way, even though it's not necessary and completely overkill.

[kb1]

Heh...Text is a crappy way to try to communicate. Maybe you do get it, and I just can't tell.

Ok, if I was a Compatibility Option Database Entry, and some port wanted to load me up, I would want to say this to the port:
"Hey, random source port, you absolutely must set option 3 and 4 to 'Yes', and option 6 to 'No', but I don't care how you set the other options."

That's it. If this random port doesn't know what Option 6 is, so be it. If this random port wants to ignore the request, so be it (player be damned).
If the source port wants to try to honor the Entry, the port would set its Option 3 to Yes, 4 to Yes, and 6 to No, and leave the others as they were.

That's all I'm trying to say. If that doesn't make sense, someone please help clarify, and/or sorry to waste your time.