A simple method of scaling up 2D sprites with ESRGAN

[MrFlibble]

I've been experimenting with neural network upscaling of various old video game art assets for quite some time already, and recently found a way to produce quite acceptable results using ESRGAN without much manual editing, proprietary software or complex procedures. Here's a sample with some sprites from Strife scaled to 2x the original size (zoomed 2x mode with nearest neighbour for better visibility):

Spoiler

 

original image for comparison

original image for comparison

 

Below is a step-by-step description of how to produce similar results.

 

I. Setting up.

Spoiler

 

1. What you'll need:

• ESRGAN
• modified test.py (optional but very useful)
• Cartoon Painted and MyManga109 models by patrikspacek
• xBRZ Scaler Test tool (ScalerTest.zip)
• GIMP
• mtPaint

2. I will not explain here how to set up ESRGAN; there's a useful guide here, and also a front-end which however I never used so I can't really recommend it. The following steps assume that you have ESRGAN installed and understand its operation. Note that if you do not have an NVIDIA card or cannot install CUDA you can stick with CPU processing although that will greatly increase performance time.

 

3. You will need to produce interpolated models from the ones you downloaded from patrikspacek. First, put the models mymanga109_250000.pth and cartoonpainted_80000.pth into the models sub-folder of the ESRGAN main folder. Then open the file net_interp.py that comes with ESRGAN in any text editor of your choice, and find the line net_PSNR_path = and alter the following segment (replace model names) to:

net_PSNR_path = './models/RRDB_ESRGAN_x4.pth'
net_ESRGAN_path = './models/mymanga109_250000.pth'

Then run net_interp.py like this:

python3 net_interp.py 0.8

That will produce a new model called interp_08.pth. You can rename it to something like MyManga109_08.pth. The do the same with the cartoonpainted_80000.pth. (The above sample images are produced with the MyManga109 interpolated model).

 

That's it, you're ready to go. To use the new interpolated models, run test.py with the respective model name as an argument. The modified version of test.py I liked to above will append the model name to the output file name.

 

II. Preparing images for processing.

patrikspacek's models are versatile enough to handle raw 8-bit images more or less properly but I discovered that the following pre-processing method yields noticeable improvement that is quite crucial when it comes to sprite edges.

Spoiler

 

1. Assemble a sprite sheet that you want to upscale. Very large images are not recommended for processing, but individual sprites would be too cumbersome to process. In any case, take care that the sprites do not touch the edges of the image and there is at least one pixel thick border between any part of the sprite and adjacent sprites, and image edges. Preferably use a dark background colour, most crucially it must be a colour that is not similar to any colour used in the sprite itself.

 

2. Open the resulting image in xBRZ Scaler Test and scale it up to 4x the original size using the xBRZ scaler. Don't forget to turn off the "Show scaler name" option. Save the result as a separate image (I use a -z postfix but it can be anything). You can batch-process images using the script by Phredreeke available here.

 

3. Open the scaled-up sheet in GIMP. Apply Gaussian blur at 0.5 pixel radius. Scale the image down to the original resolution using Sinc interpolation. Alternatively, you can apply blur at 1.0 pixel radius and use Bicubic interpolation instead. Bicubic results in a sharper scaled-down image, so blurring somewhat compensates for it. This sharpness will translate to the ESRGAN upscale result so you can change this variable to get sharper upscales if you want. Save the result.

 

III. Put the pre-processed image(s) into the LR folder of the ESRGAN folder and run test.py with one of the interpolated models. I have generally found that the MyManga109 250k produces quite good results with various video games' graphics (here are some results of scaling up complete screenshots).

 

IV. Now let's produce the masks for the upscaled sprites. This is made easy because both patrikspacek's MyManga109 and Cartoon Painted models can handle sprite edges very well on their own, especially in pre-processed images.

Spoiler

 

1.Let's first process the mask for the 4x upscale, which is the easiest. Open the ESRGAN output file (if you used the modified test.py it should be called something like myimage_MyManga109_08.png depending on how you named the interpolated model and the input image) in mtPaint and save a copy. I usually add the KN postfix because what we're going to do here is apply Kuwahara-Nagao blur (available in the Effects menu) at 2 pixel radius.

 

2. After the KN blur, you need to convert the image to indexed palette mode. In mtPaint, you can load a palette straight from another image. There are two ways you can do that:

 

a) take a screenshot of the game (e.g. Strife) in DOSBox, preferably at default brightness. That will give you the entire palette at once.

 

b) However, I intuitively prefer a different method. Remember the sprite sheet you assembled? Open that up in mtPaint and select Image -> Convert to Indexed. Unless you put too many different sprites on the sheet it will offer you to create a palette that is an exact match of the present colours (if the number of colours is 256 or less). Save the result.

 

3. Now back to the Kuwahara-Nagao 4x image, load the indexed palette using the palette menu and select the image produced by step 2b. This will allow to convert the scaled up result to exactly the same colours as the source image. Now, select Convert to Indexed, and pick the following options:

• set palette to current palette
• set dithering mode to none
• click options and set colour space to RGB (default is sRGB)

Click OK to convert the KN image.

 

4. Open both the original ESRGAN 4x result and the KN indexed image. With the latter active, press Shift+O to activate GIMP's select by colour tool, and click anywhere on the background. This will only select the background colour (purple in the examples above).

 

5. Right-click the selection and from the dropdown menu pick Selection -> Grow. Increase the selection by 1, then copy the result over the ESRGAN 4x image. Export the combined image.

 

6. Now you have the sprite sheet with clean sprite edges. If you want you can palettise the result following the same steps as desctibed in paragraphs 2 and 3. To get a transparent background, open the image in GIMP, first select Layer -> Add alpha channel, then again use the select by colour tool to pick out the background and simply delete it.

 

V. If you want to get a 2x upscale, take the ESRGAN 4x output, make a copy and scale it down to 2x. The easiest is to pick any version of Bicubic that mtPaint offers. Then, make a copy and apply Kuwahara-Nagao blur, but this time at 1 pixel radius, and also select the option to protect details. After that, take the same steps as in 2-6 in section IV above. This is how I produced the Strife images shown at the top of the post.

 

More examples: Doom, Hexen, Daggerfall (4x).

[elend]

Very interesting read and a surprisingly high amount of steps to achieve these results. But those Strife sprites look really great. It seems that their original pixel art style lends itself nicely for upscaling. 

 

Great work, please continue and don‘t give up. 

[MrFlibble]

On 5/29/2019 at 4:29 PM, elend said:

But those Strife sprites look really great. It seems that their original pixel art style lends itself nicely for upscaling.

It isn't really related to the original art; it's how the model is trained. There are newer models compared to what I have used, which produce very good results, and without the need to pre-process images as I described in my previous post.

 

Here's what I got from an interpolation of Fatality, Rebout and DeToon models (details here):

And here's the same model applied to pre-rendered sprites from Doom 64:

[elend]

Wow, that does look very clean and awesome. Great stuff.

[Immorpher]

I fiddled with ESRGAN on Doom 64 a little bit a while ago and didn't like the results, but it looks like the models have improved quite a bit! I should give it a go again. 

[wheresthebeef]

Beautiful Strife art, not so good Doom 64 result.

 

Strife stuff just needs some small touchups like to the gun, muzzleflash, or that dudes neanderthal face.

Doom 64 would be better remaking the models and doing new captures from those.

[zen4040]

that's pretty big list of things I have to install

[MrFlibble]

On 8/29/2019 at 3:37 AM, wheresthebeef said:

Beautiful Strife art, not so good Doom 64 result.

What do you think is wrong with the D64 upscales? Usually pre-rendered sprites seem to come out somewhat better that hand-drawn art when neurally upscaled, but of course that varies.

Edited September 3, 2019 by MrFlibble

[Mordeth]

15 hours ago, MrFlibble said:

What do you think is wrong with the D64 upscales?

 

They have a 'stony' look and feel to them, instead of living flesh. It also hasn't filled in the implied details as well as with other sprite sets (notably the Imp's teeth, Mancubus mouth).

[hemersonkl]

Hello people. I know here is doom, but maybe they can help me I would like to make an upscale of frames of this game and so far I have not found any good model for this and waifull2x did not work. It seems very hard to find something for sprites once it has a good result. Do you experts have any tips?

[Immorpher]

@hemersonkl This is what Topaz gigapixel does

[hemersonkl]

1 hour ago, Immorpher said:

@hemersonkl This is what Topaz gigapixel does

yes i used topaz and waifull2x and Esrgan but none of then give me a good result. =/ 

[hemersonkl]

My best results.

[Immorpher]

Ya it would be nice if there was a neural upscaler trained specifically for sprites of 3D models.

[YukiHerz]

You're not going to find anything that'll do magic to such tiny sprites, sorry.

[hemersonkl]

Understood. =/ 

[MrFlibble]

I haven't posted here in a while but I've been experimenting with ESRGAN intermittently. The models get better and better, recently I tried out one called Rebout Blend by LyonHrt (available here) which was specifically trained to upscale character sprites and it produces some very good results.

Here's a small test I just ran with some Dark Forces character sprites:

This is already palettised, scaled down to 2x (the model upscales to 4x the original size) and with the mask applied. No pre-processing was required. Here's the 4x image the model originally produced:

Original sprites:

[MrFlibble]

I just found the ESRGAN fork by Joe Ballentine which can also auto-process alpha channel in images. Here's an attempt on Strife characters using the 2x_FakeFaith-Lite_105000_G.pth model (scaled up 2x witth nearest neighbour):

 

[Vic Vos]

Spoiler

Looking at most "AI-upscale everything" projects I've seen, it's definitely not a panacea. Sometimes, though, it does some impressive stuff, especially when upscaling some games' FMV's.

[MrFlibble]

Recently I realised that a model called 4x_ArtStation1000_Sharp generally does a very decent job with pre-rendered sprites:

You can compare this to previous attempts with Doom 64 sprites here and here.

 

Also when considering the "plastic" look of the results, please keep in mind that the original models from which the sprites were made also likely had a similar kind of appearance as well.

 

And just for fun, some characters from In Pursuit of Greed:

[DOEL]

I know this is an old thread but I noticed a Twitch stream using these assets while playing demos. Is there a complevel source port out there that supports this?

[Wiw]

Any time I try to generate an interpolated model, I get this error:

 

Traceback (most recent call last):
  File "C:\Users\Frank Tiberius\ESRGAN\net_interp.py", line 18, in <module>
    v_ESRGAN = net_ESRGAN[k]
KeyError: 'conv_first.weight'

 

What's going on here, why can't I get it to work like I want it to!?

[MrFlibble]

Which ESRGAN version are you using? Does it work correctly when upscaling images?

[Wiw]

I sorted it - turns out I had to use the old_arch models rather than the newer ones.