User:TechAngel85/Sandbox3

Texture Upscaling

A reference for upscaling Bethesda textures

by: Step Team  | Forum Topic

TechAngel85

This document describes the methodology as well as general knowledge of creating upscaled textures for Bethesda games.

What makes TEVT different?

The majority of upscales available have been bulk produced. This means the majority (or all) of the textures where treated equally by applying the exact same process to all textures. This works in general, however, not all textures are equal. Some may have less or more artifacting, some may have issues never corrected by Bethesda and other may require scarcely any to no processing. Then take into account that some models work better, or more often worse, for specific types of texture vs other models.

To get consistent quality and results, TEVT handles each texture individually. A lot of testing was put into figuring out what models work well for specific texture types. Thus, each texture was examined and processed according to what past results proved to the best method for the texture type. However, each step is examined and the process adjusted for each texture, as needed. This meant a lot of experience was gained throughout the process of creating the mod. As the work was completed, that knowledge and process was documented to create this informational resource.

Setup

Complete system setup for upscaling will not be covered here. The Upscale Wiki has all the resources users will need to get set up and started.

Windows Setup: ESRGAN Installation Guide for Windows

Once setup is complete, users will need an interface for ESRGAN. It can be ran from command line, but most users will want a GUI.

Step Recommends: Cupscale

However, Image Enhancement Utility is also very good.

Take some time to get to know the interface of choice. Users should be sure they visit the settings and apply any desired changes.

After the GUI of choice is installed, models will need to be downloaded and installed. These can be found across the web using a good search query, however, the Upscale Wiki hosts a fairly large database of models with links. As always, follow safe internet practices and scan any files before use.

BC/DXT Decompression Model Reference

These models were used for removing artifacts from previous compressions. This processing inherently causes some detail loss in the range from very little up to a medium loss, depending on the model used. The images are blown up sections of textures. They show how each model handles the compression artifacts on a small section of fur on a normal map and the eye section of a diffuse map. Results will vary based on the texture content, but users can get the general sense.

General Purpose Decompression Models

These are general purpose models, which work on most textures including alpha channels, normal maps and diffuse maps.

Original

Original

DEDXT

Use for textures with light artifacting. This is a model for light-weight cleanup and is often used for noisy textures.

Color:	nearly 1:1
Color Correction:
Detail:	nearly 1:1
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

DEDXT

DEDXT

BC1 Smooth

Use for textures with medium artifacting. This is a middle-ground model and is used often. It can cause a tiny, tiny tone shift, however, the shift is so small its rarely worth correcting.

Color:	nearly 1:1, nearly inconceivable tone shift.
Color Correction:
Detail:	smoothing / blurring / light to medium loss of detail
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

BC1 Smooth

BC1 Smooth

Artifacts BC1 Free Alsa

Use for textures with medium to heavy artifacting. This is a stronger model and not used often.

Color:	causes color shift in the reds
Color Correction:	Levels > Red Channel > adjust as necessary
Detail:	smoothing / blurring / medium loss of detail
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

Artifacts BC1 Free Alsa

Artifacts BC1 Free Alsa

DXTless Source Engine 170000

Use for textures with heavy artifacting. This is the strongest of these models and is rarely used.

Color:	causes color shift as it's designed to remove green tint
Color Correction:	Levels > Red/Green Channels > adjust as necessary
Detail:	smoothing / blurring / medium loss of detail
Detail Correction:	upscale or use Smart Sharpen to remove Gaussian

DXTLess

DXTLess

Upscaling Model Reference

These models were used for upscaling original or cleaned textures. They are all 4x models, meaning they scale the texture to four times its original size, and produce various results. The images are blown up sections of textures. They show how each model handles the compression artifacts on a small section of bone-like, fur-like and leather-like textures.

General Purpose Upscale Models

These are general purpose models, which work on most textures including alpha channels, normal maps and diffuse maps. Results will vary depending on model used.

Original

Deviance DigitalArt

A general purpose model that works well with many texture types. It's particularly good on fur/hair, but is used often for many other textures.

Color:	causes color shift in the reds
Color Correction:	Levels > Red Channel > -1 to Midtone
Detail:	smoothing / blurring / light to medium noise reduction / produces lines from some noise
Detail Correction:	use Smart Sharpen to remove Gaussian

Deviance DigitalArt

ESRGAN Skyrim NonTiled RGB

This is a fairly good model that was designed for wood textures, however, it causes a color shift that can be difficult to correct. As such, it's rarely used.

Color:	causes color shift
Color Correction:
Detail:	sharpening
Detail Correction:

ESRGAN Skyrim NonTiled RGB

GameAI

This model is designed for various game textures, however, has a heavy hand with sharpening. As such, it's not used very often.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	heavy sharpening
Detail Correction:

GameAI

SkyrimTexV2

This model was designed for general Skyrim textures, but works best on metal, stone and wood-like textures or textures with rough patterns like leather and skin.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	smoothing + sharpening
Detail Correction:

SkyrimTexV2

Textures Skyrim Armory Alsa

This model is designed for Skyrim armors and as such, is great for most noisy and grainy textures, such as: leather, metal, skin, wood, etc.

Color:	causes color shift in the reds
Color Correction:	Levels > Red Channel > adjust as necessary
Detail:	sharpening
Detail Correction:	add very small amount of Gaussian blur, if needed

Textures Skyrim Armory Alsa

UniscaleV2 Sharp

This is a general purpose model good for most texture types, however, as the name suggests it's quite heavy on the sharpening. As such, it's most often used in interpolations of 10-20% to provide some sharpening, as needed.

Color:	causes color shift
Color Correction:	Levels > adjust as necessary
Detail:	heavy sharpening
Detail Correction:	add very small amount of Gaussian blur, if needed

Alpha Channels

The models below are designed for and work best on alpha channels. Results will vary depending on model used. However, whether a general model from above is used or a specific one for alphas below, if the alpha channel is simplistic then the results will generally be comparable across all models.

FireAlpha

This model is the lightest model that can be used for alphas and produces upscales that most closely resemble their originals; for better or worse. Since the results are nearly 1:1, this model is used most often for upscaling the diffuse alpha channels.

Color:	1:1
Color Correction:
Detail:	nearly 1:1
Detail Correction:

Methodology

Preparation

1. Open original texture in PS or image editor of choice that is DDS/BC7 format capable
   • BC7 format is only applicable to Skyrim SE/AE and Fallout 4
   • If Alpha exists, choose to load it as transparency
2. If applicable, create a new layer from the alpha channel
3. Exam layers to note the type of texture and any issues, such as compression artifacts, to determine the best models to run.
   • It's common to use 3-5x zoom to visual artifacting and other potential issues.
4. Save Color Map and Alpha, if applicable, in a lossless format
   • The format has to be supported by the ESRGAN GUI
   • The format should be: lossless / uncompressed / RGBRGBRGB / Linear (not all these will be available, just covering basis)
   • Step recommends using PNG, lossless format. TIFF can usually also be used, but do not save layers.

Running ESRGAN

1. Open saved images, one at a time, in ESRGAN GUI (i.e., Cupscale)
2. If compression artifact cleanup is warranted, run desired model; else skip
   • If this was completed, it's recommended to compare against the original to verify the result is desired
   • If compression artifact cleanup was completed, open the cleaned texture in ESRGAN GUI for upscaling
3. Run desired model for upscaling

Post-Processing

1. Open the upscaled Color Map and Alpha, if applicable, in PS and compare against original to verify the result is desired
2. Complete any post-processing required (i.e., color correction, sharpening, etc.)
   • To ensure alphas are black and white, users can open Hue/Saturation in PS and remove all saturation.
   • Color Maps most often require adjustments to the red channel for corrections, which can be performed via Levels in PS.
3. Resize the texture to the desired dimensions
   • Look out for transparent edges as the images have have developed them by this point! If they appear, simply create to copies of the original layer and merge down.
4. If applicable, recombine the Color Map with its Alpha.
5. Save using Nvidia Texture Tools to DDS format using BC7 / Color Map / RGBA
   • Set mipmaps to Kaiser and quality Normal

     Production quality can be used, but it takes an incredible amount of CPU power. Most users can not run it without each texture taking 6+ hours to save.

     Obviously other formats can be used, and should be used if the game doesn't support BC7. However if the game does support the format, BC7 is highly recommended!

6. Open saved file to verify integrity

NOTE

If the alpha channel is blank (black or white) it will sometimes refuse to save. To force this save, simply make a 1x1 pixel in the corner of the alpha a dark/lighter shade. For example, if the alpha channel is all white and will not save. Paint a 1x1 pixel square in a corner with 90% brightness. The alpha should then be recognized as having data (is present) and will be saved.

Specific Texture Recommendations

Fur/Hair

Fur and hair can be difficult to handle without losing too much detail or it causing hashing or other issues. Generally, the Deviance DigitalArt model tends to work best for these types of textures, but some experimentation may be warranted.

Noisy Textures

(creature scales/shells, some armor, some landscape, etc)

BC/DXT Model Recommended: DEDXT

Heavily artifacted textures can use BC1 Smooth at the cost of some detail loss

Recommended for Diffuse:

Interpolate:

1. Skyrim Armory Alsa @ 5%
2. Deviance DigitalArt @ 95%

   This interpolation will limit color shifting and provide a very small about of sharpening to the result, since Deviance tends to blur a bit.

Corrections:

-2 Luminosity to correct over-brightness

Recommended for Alpha Channel: Fire Alpha or GameAI

Human Skin Textures

BC/DXT Model Recommended: DEDXT

Recommended for Diffuse: NMKD Superscale SP