Doom Neural Upscale 2X [v 1.0]

[elend]

I don‘t want to needlessly spam this topic with vapid comments, but I really love this. I tried the recent update with breach.wad and it simply looks amazing and fresh, without being too different from the  vanilla Doom experience. I‘ll replay countless of WADs with this!

 

Can‘t wait for the pixel optimization to begin. 

[Teder]

7 hours ago, Immorpher said:

Interesting note about noise! I wonder if it may help sometimes to add some minor noise before the neural upscale? Great work here and gives me hope about doing it for other 2.5D games.

Before I redraw sprites for my project I rescale them 4 times (2x height, 2x width) in bicubic, and then rescale to 100 times (10xheight, 10x width) in nearest neighbour.

Original:

Bicubic sharper:

Nearest:

Redraw:

 

But now I'm thinking to stop working on sprites (textures are almost done) because we got @hidfan work. 

Nvidia upscale still better than any other methods - waifu, bicubic or else. That would be interesting what could happen if we can use a little bit more upscaled sprite to upscale by neural nvidia stuff.

For example:

Original, Hidfan upscaled, bicubic sharper:

 

 

What could be happen if we use a bit rescaled (3rd arch on a pic above) version to nvidia upscale.

 

Barons shown by hidfan are good example. I like very much this third pic.

 

 

EDIT:

I like to play with graphics and I know my post is a bit to large for now, but I would like to share with this little experiment:

hidfan mastermind sprite resized in waifu. I like that "paint" look:

 

 

 

 

 

Edited November 12, 2018 by Teder

[Soundblock]

55 minutes ago, Teder said:

@hidfan[...]

 

Barons shown by hidfan are good example. I like very much this third pic.

 

 

At least to my eyes, the third has some additional interest in certain areas (leg muscles, elbow area), but overall is a lesser end result than the 2nd frame. The area I react most negatively to on the 3rd is the brow/eyes - compare it to the original frame, and Baron has taken on a distinctly different personality, looking less resolved/sophisticated. Not to mention that the 2nd retains the pixel art look. While interesting for further development, I think taking on too many additional ambitions (hi Goatlord) before the two games have been fully rendered in method 2, would be risky at this stage.

[Gifty]

I concur that the ideal look is one with one foot still planted in pixel-art aesthetic. The more painterly it starts to look, the more awkward and "washy" it seems.

[Koko Ricky]

Its really important that we're talking about maintaining Doom's aesthetic in remastering/HD projects. I memba in the early 2000s, how trendy it was to use trilinear filtering, HD texture resource packs and clunky 3D models. Those projects represented a complete dismissal of the mood created by Doom's pixelation. A 2x upscale is a far better approach. It makes me wonder if nueral networks could eventually automatically generate animated voxel sets, or at least generate most of it and only require touch-up. 

[DooM_RO]

The Archvile looks unbelievable. One of the best so far.

[NightFright]

What's the problem if the HUD weapons are not scaling properly (i.e. they are much larger than they should be)? Any setting that is required for this to work as intended?

[Gifty]

The zombies are looking MUCH better in the newest versions, I'm impressed yet again. After screwing around a bit with manual cleanup, I think just redefining the facial features would pretty much do the trick. I'm gonna give that a shot.

 

EDIT: another observation: I think with some graphics, converting them back to Doom-palette can make them look a little more convincing, as it has a natural sharpening effect that brings some blurrier details into focus, and reintroduce a bit of the "noise" effect folks were talking about earlier. At least trying it with the zombieman, indexed mode also looks closer to the original game's style of color shading.

Edited November 12, 2018 by Gifty

[Linguica]

11 hours ago, Teder said:

Original, Hidfan upscaled, bicubic sharper:

 

 

Looking at it side by side like this, it really is outrageous how much detail the algorithm "recovers" seemingly out of thin air. Like, the neural upscaling *looks* like the same archvile model captured at higher resolution, as if it had somehow pulled it out of a parallel universe where id had done that originally.

[Lollie]

6 hours ago, NightFright said:

What's the problem if the HUD weapons are not scaling properly (i.e. they are much larger than they should be)? Any setting that is required for this to work as intended?

You're probably loading another mod that affects the HUD weapons. I had this same issue, I forgot that I had the Doom Sprite-Fix mod set to auto-load. Make sure that you're only loading the Neural Upscale mod.

Edited November 12, 2018 by Lollie

[VGA]

Yeah, it's crazy how the algorithm seems to add detail, visible in the creature's musculature. It also took the liberty to shorten that unnatural weird-looking finger.

 

I wonder how well a neural network approach would work if the goal was to undo music quality loss, for music that's retrieved from an old cassette, or recorded with a cheap mic, or compressed into a low bitrate format etc. I bet the algorithm would make it sound crisp!

[NightFright]

7 hours ago, Lollie said:

You're probably loading another mod that affects the HUD weapons. I had this same issue, I forgot that I had the Doom Sprite-Fix mod set to auto-load. Make sure that you're only loading the Neural Upscale mod.

 

Right, I had forgotten about that. Thanks a lot for the hint!

[hidfan]

7 hours ago, VGA said:

Yeah, it's crazy how the algorithm seems to add detail, visible in the creature's musculature. It also took the liberty to shorten that unnatural weird-looking finger.

 

 

I did took the liberty to crop the finger :)

I also cropped the real human fingers holding the miniature for the first pain frame.

Masking is badly done by IA. Topaz AI GigaPixel (trial available) is the only one trying to do something. But I still have to blur+contrast to round it before downscaling

Then I re-round, resharp or refine details manually (add seethrough holes when it's possible ex background arm, ribs, spine,fingers, blood drops, flames). It can be very long (hello Vile ), but it adds a lot of HD feeling.

Because of me, there are also little inconstistency of alphas between frames where only a parts of the body is moving. That will need to be fixed.

 

[hidfan]

On 11/12/2018 at 6:49 AM, elend said:

I don‘t want to needlessly spam this topic with vapid comments, but I really love this. I tried the recent update with breach.wad and it simply looks amazing and fresh, without being too different from the  vanilla Doom experience. I‘ll replay countless of WADs with this!

 

Can‘t wait for the pixel optimization to begin. 

Your nice comments are perfect for marketing - advertising quotes :)

I'll upload my PSD sources soon !

[Batandy]

You know, i'd love to see Wolf 3d's 64x64 wall textures upscaled to 128x128 and then compared to Doom 2's upscaled textures in MAP31 and MAP32. Just to see how they'd compare to a manual edit.

[Antnee]

Request:  Can you run some of the Doom64 sprites through this?  Especially the Baron and Imp.  I'd love to see what that looks like.

 

What really strikes me as impressive:  Look how it "creates" the baron's forehead wrinkles out of thin air.  How the fuck

Edited November 13, 2018 by Antnee

[hidfan]

58 minutes ago, Batandy said:

You know, i'd love to see Wolf 3d's 64x64 wall textures upscaled to 128x128 and then compared to Doom 2's upscaled textures in MAP31 and MAP32. Just to see how they'd compare to a manual edit.

I processed the (doom2's)Wolf texture but the result wasn't very good, and not very usefull (the goal wasn't to be immersive, just a joke, so low pixel art style is almost mendatory).
They are not in the pk3

[Teder]

@hidfan 145 frames for archvile :D yes, arch is a pain in ass ;). I was repostioning him 30 minutes looking for original and add dobule x/y position to Your upscaled version. Monsters sprites are most time consuming graphics to work. There was a time when I reworked whole QUAKE 1 hires textures with bump/spec/etc. to QUAKE 4. Problem was that I manualy adjust brightness for heightmaps. It was about 1000 MB files (if I good remember). It was 10 years ago :D

 

EDIT: I do not touch wolf textures neither in my hires pak. I think they should be as they are.

Edited November 13, 2018 by Teder

[elend]

@Teder Also a big thanks to you, for sll the help and texture stuff you are doing! Greatly appreciated! :D

 

Regarding Doom 64; I am not sure the results would be anywhere near as good, probably because the original sprites already lack a bit „definitiv“ due to being „just resized renders“. It‘s worth a try of course. 

[hidfan]

Doom64 imp with ugly masking, and without the Topaz.AI Gigapixel 6x layer (my trial has expired)

As the input was not very sharp, the upscale isn't that good. (downsizing the upscaled back to original size improves a bit its sharpness (but I have a strange 1 pixel offset occuring)

I won't upscale D64 assets  (1- they're damn too dark and colorless, 2- their animations are uncanny 3- they're blurry, 4- I dislike the style)

[Soundblock]

I think a lot of the reason why these sprites upscale so well is down to Adrian's (and Kevin's) ability to tell a big story with humble means. Additional defintion wouldn't be possible if the original pixel configurations weren't pointing towards the current results. That, and hidfan having considerable pixel art skills to begin with, which makes him suited for configuring the outcome. Guess we might see more pixel art classics get this treatment in the future, once the thing's out of beta. I know I'd love to see Chaos Engine and/or Metal Slug with the additional fidelity, though I suppose they might require different settings set, if even compatible.

[DooM_RO]

Once you guys are done with this, I hope you won't stop at Doom. It would be great if Blood and Duke Nukem 3D got the same treatment.

[Koko Ricky]

Blood, being that it has higher resolution sprites, would look especially nice. I think it's the only Build-era game with sprite work comparable to Doom's in terms of detail.

[Mordeth]

38 minutes ago, GoatLord said:

Blood, being that it has higher resolution sprites, would look especially nice.

 

Redneck Rampage, (probably) the last classic Build engine game to be released, had the highest resolution sprites.

[Teder]

Yeah Redneck Rampage got very nice resolution. The most shock resolution for me in 90s was when I saw for the first time Witchaven (don't remember 1 or 2). It was photorealistic for me at that time.

 

 

[The Nate]

Looking very great! Just another problem here: The Archvile is too big on Doom 2's cast screen!

 

Spoiler

 

[elend]

This image will now haunt me forever! 

[KVELLER]

There seem to be some tiling issues with the blood:

 

 

BTW this looks amazing, yadda yadda, you've heard it a hundred times already :P

Edited November 14, 2018 by KVELLER

[DooM_RO]

One thing you need to be aware of is that some texture recreations might have extra noise that wasn't supposed to be there because of the upscaling. This can disrupt the look of textures that are supposed to be more smooth. 

 

I have always thought that this door and panel are made out of a smooth metal/plastic:

 

 

[Teder]

@Sgt Nate V that's simple - archvile is very important enemy - so that's why he is so big there ;) I used a trick to work arch, in game he looks normal, but on the end screen You're right he is double size. For sure some much more skilled than me fix it in the future.

 

@DooM_RO there is a solution, but this is additional manual work - retailing each texture :( For me it's fine, I like a lot all @hidfan work.