User:TechAngel85/Sandbox3

Texture Upscaling

A reference for upscaling Bethesda textures

by: Step Team  | Forum Topic

TechAngel85

This document describes the methodology, as well as, general knowledge of creating upscaled textures for Bethesda games.

The information contained within this resource was derived from first-hand experience. Countless hours have been spent upscaling and testing various models on various textures in an attempt to, objectively, determine what pre-trained models work best for specific texture types. New models are being developed all the time and not all models were tested, however, a large group was tested and narrowed down to a select few that worked well in various situations. It is entirely possible there are models that were missed that perform just as well. Finally, this information was relevant at the time of this writing, but may become irrelevant at any time in the future.

Setup

Complete system setup for upscaling will not be covered here. The Upscale Wiki has all the resources users will need to get set up and started.

Windows Setup: ESRGAN Installation Guide for Windows

Once setup is complete, users will need an interface for ESRGAN. It can be ran from command line, but most users will want a GUI.

Step Recommends: Cupscale

However, Image Enhancement Utility is also very good.

Take some time to get to know the interface of choice. Users should be sure they visit the settings and apply any desired changes.

After the GUI of choice is installed, models will need to be downloaded and installed. These can be found across the web using a good search query, however, the Upscale Wiki hosts a fairly large database of models with links. As always, follow safe internet practices and scan any files before use.

BC/DXT Decompression Model Reference

These models were used for removing artifacts from previous compressions. This processing inherently causes some detail loss in the range from very little up to a medium loss, depending on the model used. The images are blown up sections of textures. They show how each model handles the compression artifacts on a small section of fur on a normal map and the eye section of a diffuse map. Results will vary based on the texture content, but users can get the general sense.

General Purpose Decompression Models

These are general purpose models, which work on most textures including alpha channels, normal maps and diffuse maps.

DEDXT

Use for textures with light artifacting. This is a model for light-weight cleanup and is often used for noisy textures.

Color:	nearly 1:1
Color Correction:
Detail:	nearly 1:1
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

BC1 Smooth

Use for textures with medium artifacting. This is a middle-ground model and is used often. It can cause a tiny, tiny tone shift, however, the shift is so small its rarely worth correcting.

Color:	nearly 1:1, nearly inconceivable tone shift.
Color Correction:
Detail:	smoothing / blurring / light to medium loss of detail
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

Artifacts BC1 Free Alsa

Use for textures with medium to heavy artifacting. This is a stronger model and not used often.

Color:	causes color shift in the reds
Color Correction:	Levels > Red Channel > adjust as necessary
Detail:	smoothing / blurring / medium loss of detail
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

DXTless Source Engine 170000

Use for textures with heavy artifacting. This is the strongest of these models and is rarely used.

Color:	causes color shift as it's designed to remove green tint
Color Correction:	Levels > Red/Green Channels > adjust as necessary
Detail:	smoothing / blurring / medium loss of detail
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

DXTLess

DXTLess

Upscaling Model Reference

These models were used for upscaling original or cleaned textures. They are all 4x models, meaning they scale the texture to four times its original size, and produce various results. The images are blown up sections of textures. They show how each model handles the compression artifacts on a small section of bone-like, fur-like and leather-like textures.

General Purpose Upscale Models

These are general purpose models, which work on most textures including alpha channels, normal maps and diffuse maps. Results will vary depending on model used.

Deviance DigitalArt

A general purpose model that works well with many texture types. It's particularly good on fur/hair, but is used often for many other textures.

Color:	causes color shift in the reds
Color Correction:	Levels > Red Channel > -1 to Midtone
Detail:	smoothing / blurring / light to medium noise reduction / produces lines from some noise
Detail Correction:	use Smart Sharpen to remove Gaussian

ESRGAN Skyrim NonTiled RGB

This is a fairly good model that was designed for wood textures, however, it causes a color shift that can be difficult to correct. As such, it's rarely used.

Color:	causes color shift
Color Correction:
Detail:	sharpening
Detail Correction:

GameAI

This model is designed for various game textures, however, has a heavy hand with sharpening. As such, it's not used very often.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	heavy sharpening
Detail Correction:

NMKD Superscale SP

This model is model designed for real photos, but does well for some texture types. It works well for noisy textures and textures like: leather, skin, scales, metal, etc.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	moderate sharpening
Detail Correction:

SkyrimTexV2

This model was designed for general Skyrim textures, but works best on metal, stone and wood-like textures or textures with rough patterns like leather and skin.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	smoothing + sharpening
Detail Correction:

Textures Skyrim Armory Alsa

This model is designed for Skyrim armors and as such, is great for most noisy and grainy textures, such as: leather, metal, skin, wood, etc.

Color:	causes color shift in the reds
Color Correction:	Levels > Red Channel > adjust as necessary
Detail:	sharpening
Detail Correction:	add very small amount of Gaussian blur, if needed

UniscaleV2 Sharp

This is a general purpose model good for most texture types, however, as the name suggests it's quite heavy on the sharpening. As such, it's most often used in interpolations of 10-20% to provide some sharpening, as needed.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	heavy sharpening
Detail Correction:	add very small amount of Gaussian blur, if needed

Alpha Channels

The models below are designed for and work best on alpha channels. Results will vary depending on model used. However, whether a general model from above is used or a specific one for alphas below, if the alpha channel is simplistic then the results will generally be comparable across all models.

FireAlpha

This model is the lightest model that can be used for alphas and produces upscales that most closely resemble their originals; for better or worse. Since the results are nearly 1:1, this model is used most often for upscaling the diffuse alpha channels.

Color:	1:1
Color Correction:
Detail:	nearly 1:1
Detail Correction:

Methodology

Preparation

1. Open original texture in PS or image editor of choice that is DDS/BC7 format capable
   • BC7 format is only applicable to Skyrim SE/AE and Fallout 4
   • If Alpha exists, choose to load it as transparency
2. If applicable, create a new layer from the alpha channel
3. Exam layers to note the type of texture and any issues, such as compression artifacts, to determine the best models to run.
   • It's common to use 3-5x zoom to visual artifacting and other potential issues.
4. Save Color Map and Alpha, if applicable, in a lossless format
   • The format has to be supported by the ESRGAN GUI
   • The format should be: lossless / uncompressed / RGBRGBRGB / Linear (not all these will be available, just covering basis)
   • Step recommends using PNG, lossless format. TIFF can usually also be used, but do not save layers.

Running ESRGAN

1. Open saved images, one at a time, in ESRGAN GUI (i.e., Cupscale)
2. If compression artifact cleanup is warranted, run desired model; else skip
   • If this was completed, it's recommended to compare against the original to verify the result is desired
   • If compression artifact cleanup was completed, open the cleaned texture in ESRGAN GUI for upscaling
3. Run desired model for upscaling

Handling Heavy Artifacting

Removing heavy artifacting, like what is found on many normal maps, can be difficult to accomplish. Unfortunately, users are unlikely able to remove it all utilizing models, alone; although sometimes very close results are achievable. Since the goals are to clean textures while retaining, at least, vanilla level of details, using models that would likely correct this artifacting is not an option. These types of models typically cause either serious loss of detail or too much dithering (noise) to be useful; both resulting in less accurate upscales.

To attempt to eliminate heavy artifacts without losing too much detail:

1. Separate out the color channels into layers and save them individually.
2. Process to clean each channel, individually, using various models until a general to moderate reduction of artifacts is achieved
   • Don't worry about eliminating all artifacting at this stage.
3. Once all three are processed, recombine the channels and resave the whole texture in a lossless format again
4. Finally, process this recombined texture again to allow the models to work on the recombined channels at once

Post-Processing

Color Correcting Tip

---

Correcting color changes can be rather tricky and frustrating. A tip when a non-black or non-white, solid background is present is:

1. Using Color Picker, set foreground color in PS the same as the original texture's background
   • Note the original RGB values
2. Using the foreground color and Color Picker, select the upscaled texture's background
   • Note the changes to the RGB values when the new background is selected.
3. Go to Levels. Adjust the individual color channels up/down according that what is required to match the original RGB values.
4. Test again and repeat Level adjustments until the values are no more than 1 digit off
   • Often it can be adjusted as close as only a single value being off by a single digit.

Open the upscaled Color Map and Alpha, if applicable, in PS and compare against original to verify the result is desired

Complete any post-processing required (i.e., color correction, sharpening, etc.)

• To ensure alphas are black and white, users can open Hue/Saturation in PS and remove all saturation.
• Color Maps most often require minor adjustments to correct color changes from the upscaling. This can be achieved via Levels in PS.

Resize the texture to the desired dimensions

• Look out for transparent edges as the images have have developed them by this point! If they appear, simply create to copies of the original layer and merge down.

If applicable, recombine the Color Map with its Alpha.

Save using Nvidia Texture Tools to DDS format using BC7 / Color Map / RGBA

• Set mipmaps to Kaiser and quality Normal

  Production quality can be used, but it takes an incredible amount of CPU power. Most users can not run it without each texture taking 6+ hours to save.

  Obviously other formats can be used, and should be used if the game doesn't support BC7. However if the game does support the format, BC7 is highly recommended!

Open saved file to verify integrity

NOTE

If the alpha channel is blank (black or white) it will sometimes refuse to save. To force this save, simply make a 1x1 pixel in the corner of the alpha a dark/lighter shade. For example, if the alpha channel is all white and will not save. Paint a 1x1 pixel square in a corner with 90% brightness. The alpha should then be recognized as having data (is present) and will be saved.

Specific Texture Recommendations

Fur/Hair

Fur and hair can be difficult to handle without losing too much detail or it causing hashing or other issues. Generally, the Deviance DigitalArt model tends to work best for these types of textures, but some experimentation may be warranted.

Noisy Textures

(creature scales/shells, some armor, some landscape, etc)

BC/DXT Model Recommended: DEDXT

Heavily artifacted textures can use BC1 Smooth at the cost of some detail loss

Recommended for Diffuse:

Interpolate:

1. Skyrim Armory Alsa @ 5%
2. Deviance DigitalArt @ 95%

   This interpolation will limit color shifting and provide a very small about of sharpening to the result, since Deviance tends to blur a bit.

Corrections:

-2 Luminosity to correct over-brightness

Recommended for Alpha Channel: Fire Alpha or GameAI

Creature/Human Skin Diffuse

These textures typically have noise and finer details. The combination below will clean them up a bit and upscale them, while leaving most the noise and sharpening details.

BC/DXT Model Recommended: DEDXT

Recommended for Diffuse: NMKD Superscale SP