Easier way to watch demos

[Grazza]

Would you like to be able to browse through demos as easily as you might browse a collection of videos, photos, documents, etc.?

<font color=red size=4>The file to download is here. And you need Prboom-plus.</font>

I imagine most people who watch a lot of demos have a fairly streamlined way of doing so, but nevertheless it is sometimes a bit more fiddly than we'd like. Or you discover part way in that you've loaded the wrong version of the wad, or that plutonia needed to be used as a pwad in this case, etc., etc. And at best it normally takes at least a few seconds to sort out the right command line, or right-click action or whatever.

The aim here is to make it so most demos can be viewed simply by clicking on them, with the correct wads automatically loaded. This is done by an extensive list of patterns against which the file-name is matched.

I'll get straight to the point. You'll need the current or a recent version of Prboom-plus and the big list of patterns that can be found here. I must stress that it doesn't work at all with version 2.4.6.2 or earlier, or "regular" PrBoom - these versions lack the feature that it relies upon.

Here is the quick and dirty version of the instructions (more details in the txt that accompanies the patterns):

1. Check that the new version of Prboom-plus is finding your iwads and working OK (I always use my existing cfg with it and just tweak any new settings).

2. Open patterns.txt in Notepad (or similar). Copy & paste the list of patterns into your cfg in place of the list that is currently in there.

3. Open Prboom-plus's launcher and select (in the Commands window at the bottom):

Associate the current EXE ... with DOOM demos (-auto mode)

(-auto is the command-line option that activates the new autoloading feature)

Now when you click on a lmp (from inside a zip, from anywhere on your computer, or that you have opened from an online source), then as long as the pattern is recognized and it finds the wads, it will automatically load them as needed and play the demo.

This should work with most of the demos you'll find here, at compet-n, and elsewhere. Obviously you need to have any pwads that are required, and they need to be in the location the program is expecting to find them. This means either the same directory (folder) as the program itself, or in a %doomwaddir% that you have defined.

More info is given in the zip. As just one example of what it can do, consider that from a playback viewpoint there are four possible cases of compet-n demos on Evilution:
1. tnt.wad used as an iwad
2. tnt.wad used as an iwad and the tnt31 patch loaded as a pwad
3. tnt.wad used as a pwad
4. tnt.wad used as a pwad and the tnt31 patch loaded too
The patterns include sufficient information to distinguish between these four cases, for all 900+ tnt demos at compet-n (I hope; I'm pretty sure I tested them all). They also include a list of demos starting "et" that are Tysons on tnt.wad (rather than demos on Eternal.wad). And so on.

Anyway, I hope this proves useful. Feel free to extend it, but be careful, as a single misplaced bracket can (and often did) create an awful mess.

[updated: 28-September-2008; 1671 patterns]

[myk]

It just occurred to me, could something like this be engineered to work the other way around, as well? I mean, you record, and depending on the IWAD, PWAD (if any), and time, it names the demo. Of course, it would name any demos like speed runs or maxes, since otherwise it would become quite more complicated.

[Grazza]

Given that when recording, you need in any case to specify some command-line parameters (one way or another), any time saving would be of a smaller order of magnitude. You're saving a few characters of typing, rather than the need to fiddle at all.

Well, I had thought that maybe it could autoload wads depending on the lmp file-name that you had chosen to record with. So, e.g., you put in "-record ks13-xxx -auto" and it autoloads ksutra.wad (a saving of one character in this instance). In general, this would mean putting in more wildcard matching than I have used so far, and I could imagine it having some unexpected results.

As for autonaming, I think you'll need to rename manually in so many cases that this wouldn't really be much of a help. And there is perhaps more danger of overwriting existing demos. Anyway, it sounds a fiddly feature to implement, and with unclear benefits. If I were to request Andrey to extend this feature further, it would be to allow forcing of additional command-line options depending on the lmp file-name. That would make it possible, e.g., to view all TAS demos directly - currently some will fail without "-complevel 5" being supplied to them. This would also need additional work on the patterns, but it wouldn't be too arduous given that I've done a lot of the spadework already by testing TAS demos for desyncs.

[Hobbs]

I just tested this feature out, and I have to say I am quite impressed. Thank you very much, what a wonderful gift to the demo community (and everyone really) on Doom's 13th.

[Grazza]

Thanks. :)

I've uploaded an updated version (same link as before).

This adds:

** Improved support for HR and HR2 demos. Specifically:
* It now knows which HR2 demos were for a beta (and which beta).
* It can identify which HR demos were for the Episode 1 pre-release. However, some 100% naming clashes prevent them from being fully supported. Fortunately, the unsupported ones are inside the zip that contains the wad they require, so that should be fairly obvious.
* More HR demos with relatively ad hoc names are supported. I couldn't leave pn22-123.lmp out, could I?

** Rudimentary support for DM demos:
* The DM pwad demos at DSDA are all supported. The wads for those demos can all be found here in case you don't already have them. This was fiddly work for just 35 demos, but then again it is more than 9 hours' worth, and it will be easy to extend it to cover more DM demos from other places, as many of the popular DM wads are now in the list.
* I've tweaked the default patterns at the end of the list a little, as some Doom2 DM iwad demos were being "guessed" as Doom(1) demos.


As an aside, I noticed one really weird thing when doing this. If you play back hqr-4339 without loading hr.wad (i.e. so it starts playing in Doom2.wad and thus completely the wrong map), then the player actually exits map01 of Doom2. This happens about 6 minutes in. I wonder what the odds are against that, even considering that it is a small and simple map.

Edited May 28, 2023 by Grazza

[myk]

2. Open patterns.txt in Notepad (or similar) and replace all occurrences of "c:\doom" with the place where you have your Ultimate Doom pwads and "c:\doom2" with the place where you have your Doom2 pwads (it's OK if they are both the same place). Or if you have them in the same locations as I do, you don't need to change anything.

Perhaps a couple of variables, for DOOM and DOOM II, could simplify this. You'd just have to define each once, and on the entries it'd say something like %doom and %doom2. It could also be made to retain the ability to read specific directories (i.e., c:\doom\pwads\TCs\) in case someone likes to place wads in various directories for whatever reason.

For example:

demo_pattern35            "Equinox/eq\d\d.\d\d\d\.lmp/doom2.wad|%doom2\equinox.wad"

The %doom2 would appear once somewhere in patterns.txt, and would hold the actual directory string refrenced by each "%doom2".

[entryway]

prboom should search for wads, iwads in default dirs (current, exe, %doomwaddir%). it's bug if doesn't

[Grazza]

If I eliminate the paths, then it only works if the demo I am trying to play back is in the same directory as the wads, or is in a zip that is in that directory. The exe is in the same directory as the pwads that the autoloading is failing to find.

It appears to me that the autoloading function is only looking (when no path is given) in current-dir, not exe-dir or %doomwaddir%.

[Grazza]

The current version of the patterns (477 lines) is included in the Prboom-plus 2.4.8.1 release.

The patterns are already included in the glboom-plus.cfg in prboom-plus-2.4.8.1-win32.all-in-one.zip and entryway.cfg in prboom-plus-2.4.8.1-win32.zip, so if you'll only need to copy/paste them if you want to use your existing version of your cfg files.

Note that I have now eliminated the paths. I reckoned that the advantages of this outweighed the possible disadvantages, now that the program looks in all the locations it is meant to. This means that all the wads relevant to playing back demos need to be in either the same directory as the program or in a %doomwaddir% that you have defined.

The patterns will now match a lot more demos than the previous ones. The main ones that I know of that it won't match are DSDA demos from 2003 or earlier on wads that aren't popular for recording and whose file-name doesn't start with p-z. I think deathmatch demos are now pretty well supported. Feedback on any of this is welcome, particularly if you find some demos that ought to work with it but don't.

[Grazza]

Updated again, with a total of 975 patterns. I've now finished the bulk of this work, having completed the support for the demos in the DSDA archive.

If you find any compet-n or DSDA demos that it doesn't currently support, then either:
1. The naming is too conflictive to support.
2. It is one of the rare demos that needs an additional command-line option to be specified.
3. The demo is for Legacy or Zdoom, or some other type not supported by Prboom-plus.
4. You don't have the necessary pwad(s), or have the wrong version of it/them.
5. I have made a mistake (let me know).
6. I have overlooked something (let me know).

[VinceDSS]

does it include public dang demos ?

[Grazza]

If they have fairly regular naming, yes. I didn't test DANG demos systematically though - just some random testing to make sure it more or less worked with them.

For demos on the original iwads, the recognition works as follows:

• Near the start of the list, there are patterns for standard demo names, such as those used at compet-n. If it matches these, then it is played back with the appropriate one.
  
• If the demo name doesn't match those or any of the patterns for pwad demos, then it reaches the default patterns at the end (972-974 with the current numbering). If there is a sequence of the type [letter][digit 1-4][letter][digit 1-9] or something else that strongly suggests it is an (Ultimate) Doom demo, then it is played back with that (pattern 973). Otherwise it is played back with the Doom2 iwad (pattern 974).

[Searcher]

Grazza that is certainly going above and beyond the call to put all those together. Your hard work sure has made my life easier. Thanks.

[Donce]

I wonder, maybe PrBoom+ could write information about the wad into lmp file when recording a demo? Just like it can be done with lmpmast.exe, which I use quite often for my demos (just so I know after a few years which wads they were recorded on). This way, every demo recorded with PrBoom+ (and their number is growing) could be automatically played back whatever the filename.

[Grazza]

Updated once more. Primarily a maintenance release, with some improvements in the default patterns, fixing of some errors, improvements in the text-file (info on where to get the necessary wads), etc. Only a small increase in the total number of patterns (to 1013).

Question for anyone familiar with RegExps: how can you match a file specifically named (e.g.) "1.lmp" without also matching any that end "-1.lmp"?

Donce: interesting idea. I presume lmpmast.exe adds this information onto the end of the lmp file?

[Donce]

Grazza said:

Donce: interesting idea. I presume lmpmast.exe adds this information onto the end of the lmp file?

Yes, that's exactly what it does, it writes everything after the demo ending byte. Take, for example the demo from this zip. Its end, as copied from metapad.exe (text viewer similar to notepad, don't have a hex editor at hand) looks like this:

€psycho3.wad                                                           0046doom2+  waSEXE

That euro sign is the demo ending byte (in ASCII editor it would look like a french C) and then there's some info I entered with lmpmast.exe: that it was recorded in psycho3.wad with doom2+. Of course, this info is displayed in organized way in lmpmast's pseudographical interface.

So, PrBoom+ could support lmpmast's format (then it would always at least recognize my demos in which I have entered the information) or it could use some new format.

[Kristian Ronge]

Grazza: To match [digits].lmp and not [whatever]-[digits].lmp, can't you just use the RE (0|1|...|9)(0|1|...|9)* to match any number (at least one, though) of digits, and digits only?

[entryway]

Grazza said:

Question for anyone familiar with RegExps: how can you match a file specifically named (e.g.) "1.lmp" without also matching any that end "-1.lmp"?

It's a bug in prb+. I do not check bounds indexes of matched part. They should equal to indexes of first (0) and last (length-1) symbols for correct matching.

The current code will use a pattern even in following case:
aaabbbccc.lmp
demo_pattern0 "The Bug/bbb/doom2.wad|mm.wad"

Fixed.

[Schneelocke]

Grazza said:

Question for anyone familiar with RegExps: how can you match a file specifically named (e.g.) "1.lmp" without also matching any that end "-1.lmp"?

Is this what you're looking for?

m/^1.lmp$/   # matches "1.lmp"
m/^\d.lmp$/  # matches "1.lmp", "2.lmp" etc.
m/^\d+.lmp$/ # matches "1.lmp", "385.lmp" etc.

[entryway]

m/^1.lmp$/ # matches "1.lmp"

That will not work because by some reasons I use PCRE_NOTBOL flag in regexec(). This option specifies that first character of the string is not the beginning of a line, so the circumflex metacharacter should not match before it. I can remove this flag, but now it has no sense because I check bounds. (that's needed in any case)

[Grazza]

Great! Now I can remove all those "\b"s and add back a few things I had removed because they were causing problems.

Schneelocke: Thanks - useful information (which I ought to have known, but didn't), even though it isn't (wasn't?) the solution in this case.

[Grazza]

Grazza said:

Great! Now I can remove all those "\b"s and add back a few things that I had removed because they were causing problems.

Done, and uploaded. Note that this version shouldn't be used with anything earlier than the current test version, for reasons that will be obvious if you read the last few posts. I've also added a bunch of links to demo libraries and sites in the accompanying text-file, though they should mostly be pretty familiar to regulars here.

[Schneelocke]

Grazza said:

Great! Now I can remove all those "\b"s and add back a few things I had removed because they were causing problems.

Schneelocke: Thanks - useful information (which I ought to have known, but didn't), even though it isn't (wasn't?) the solution in this case.

You're welcome. I recommend this book for all your regexp needs, BTW. :)

[Grazza]

Updated once more. Mostly little corrections and improvements, and some obscure stuff added. For instance:

• More DM demos supported (e.g. Jason's)
  
• Some compet-n demos on doom.wad with irregular naming that I hadn't previously spotted (I searched a bit more systematically this time), or for the old E1M1
  
• sda/id - a few didn't work
  
• Mirra demos (only a couple of episode runs didn't work previously though)
  
• DSF demos (i.e. the ones in the DSF FTP space - some of those that weren't in demo packs, or had been renamed, were not supported)
  
• Some older demo packs that I had forgotten about
  
• A few demos from the last couple of weeks that weren't covered by the existing patterns
  
• ...and some other things that were lying around on my hard disk or on the internet.

[Opulent]

Grazza said:

• sda/id - a few didn't work
  

whoa, forgot all about that page

[Grazza]

Actually, a couple of demos from there (yours?) still have me baffled: 439.zip and loh8b.zip. Any ideas?

That reminds me of one other situation in which I hope this pattern-matching gizmo will be useful to people: if you have a demo with no textfile and no idea what it was recorded on. In that case you can just click on it, and, if recognized, the program will either play it back, or tell you which wads it requires if it doesn't find them.

[Opulent]

If I was to guess, they are in the wrong place.
probably my demos testing two people's maps -- like a beta of loh8(or something) and one of Espi's maps. ???
I usually record a first time through demo(so the author can see how it plays normally), then I play to complete it after knowing the map.
either way, these are probably trash.

[Donce]

That's a bit off-topic, but this "first time through" thing is what I really like and I sometimes record such demos myself in fakenet mode (just that in case I die, I don't need to restart all the map again), I don't keep them though. I think this is an interesting category - you virtually never see such demos.

[Grazza]

Another update. There are now 1120 patterns, and many of the existing ones have been improved. The additions are mostly for older or more obscure stuff, and quite a lot of DM demos too.

[emailking]

I've updated to the latest version. I have 32 old av maxdemos (as in avold.wad) in my doom2 directory. They are of the form avoldxx-uv.lmp. They are from some demopack I downloaded on May 21, 2002. All I have now are the demos and the included text file. So it's not some wacky naming convention I made up. When I double click on them, it just plays the demo in the corresponding doom2.wad map. I did
pr+ -file av.wad avold.wad -playdemo avoldxx-uv and then they play fine. It doesn't really matter to me as it's only a minor issue...I love what you've done. But I thought you might want to know.