User:Z929669/Sandbox/HeaderTabs Functionality

WIP

This guide is currently under heavy revision. The information in some sections is accurate, but linked information may be out of 'order' or out of context with the current guide format.

A simplified guide to using DDSopt to improve texture quality & performance -- by z929669 & Kelmych & S.T.E.P. Team

Updated: 21:13:38 25 July 2021 (UTC)

GUIDE FORUM THREAD

---

Introduction

Using DDSopt

Appendix

[edit]

Textures are game assets that wrap around object models in games like Skyrim. These textures provide the colors and details of all objects. Think of textures as the paint and models (meshes) as the canvas. In short, textures play a major and fundamental role in making games like Skyrim aesthetically enjoyable. Textures can be quite large, however, and they are often primarily responsible for performance limits. Unfortunately, many of the fantastic textures produced by Bethesda artists and modders for Skyrim do not exist in an optimal format. There are trade-offs for quality and performance, and there is quite a lot of room for improvement in this area. Thankfully, we have DDSopt to rectify these issues.

What is DDSopt?

DDSopt (created by Ethatron) offers optimization of texture files by eliminating redundancies, ensuring proper compression format, and recalculating scale-corrected mipmaps using a variety of custom algorithms. Several different types of texture files are used in object modeling in Skyrim, and DDSopt can recognize most types and be told how to treat unknowns using a configuration file.

Although powerful and useful, DDSopt (and its little brother BSAopt) is a beta program that is no longer under development. As such, it lacks some bells and whistles and is not so simple to use without some instruction. Nevertheless, no other application exists with the functionality of DDSopt, and it is simply the best application for optimizing, compressing and packaging texture assets for Skyrim, Oblivion and Fallout, period.

Modded texture packs are often composed of textures saved in an inefficient compression format in relation to the texture function. This translates into textures often being 2x larger than necessary or worse due to inclusion of redundancies that add no useful information. Like unnecessary white space in this guide, the net effect is that more graphic memory (VRAM) is required to render these 'bloated' textures, which in turn costs valuable resources and hampers performance. In addition to bloat, improperly-formatted textures may also lack important information in the form of mipmap levels (see Q1) required for scaling the textures in game. Visit the DDSopt page on the OBGE Wiki to learn more about texture compression and technical details of DDSopt, and the DDSopt Nexus page for more information on DDSopt functionality.

Purpose of this Guide

The available web documentation on DDSopt is highly technical and not very extensive. This guide provides the bulk of available documentation on how to use DDSopt. The guide includes information aimed at users across a wide range of backgrounds from mod developers with significant experience with Solid Modeling and the associated tools for creating and editing these models to new users with limited understanding of these models. For much of the material in the guide it is currently the only available source. The guide includes some step-by-step instructions along with explanations. This fulfills one of the goals of the guide which is to provide insight in using DDSopt so it can also be used for situations not described in detail in the specific steps listed in the guide.

As suggested by Martigen, it is hoped that mod authors will adopt "best practices" that are put forth in this guide as standard procedure when creating textures for the modding community. The main goal is to facilitate proper implementation of DDSopt for modders and mod authors without requiring too much pain or need for hours of research and preparation. This guide's authors are not graphics experts or artists. The STEP community has modders with enough experience to recognize that DDSopt is quite useful for the purpose of improving performance and quality, and empirical evidence is presented here to support this claim. Thus, aside from being a practical guide to using DDSopt and some effective benefits of doing so, this guide also serves as a review and "DDSopt-imization" of some of the major texture overhauls including Bethesda's Hi-Res Texture Pack DLC, Vano89's Hi_Res DLC Optimized, Nebula's Skyrim HD, Starac's Skyrim Realistic Overhaul, Z4G4's Serious HD Retexture Skyrim and Insanity's Texture Pack.

[edit]

Initial Configuration

To reduce potential issues on Windows OS, the following should be considered:

• The user must be in the Administrators group on the system so that DDSopt GUI settings changes will be preserved upon exit.
• Applications installed within User Access Controlled (UAC) directories (e.g., Program Files, Program Files (x86)) may not behave as expected, so consider installing DDSopt and "Steam" into other directory locations (e.g., D:\Games\DDSopt and D:\Games\Steam).

1. Before using DDSopt for the first time, launch the game at least once so all the game configuration settings and some important registry values are initialized.
2. Make sure the 2012 Visual C++ redistributable from Microsoft is installed on the system; the current 32 bit and 64 bit versions are available. When using a 64 bit version of windows installing both the 32 and 64 bit versions is recommended.
3. Download DDSopt 0.8 - pre-release - update 4 from the Nexus, and create a shortcut to the executable.
4. Right click on the shortcut, go to the "Compatibility" tab, and check the box labeled "Run this program as an administrator", then click "Apply" and "OK". This ensures that DDSopt is run with administrative privileges so the chosen settings will "stick" between sessions. NOTE: If an error occurs stating that "d3dx9_43.dll is missing" when launching for the first time, then install or reinstall DirectX to resolve.
5. Be sure to update to the latest version of DDSopt.ini, and place it into the same folder as the executable, overwriting the older version of this file. The DDSopt.ini file, described in more detail in the Appendix, is used to configure texture-specific processing. Unlike some INI files, it does not affect the GUI or default GUI settings.

The Interface

The GUI settings for DDSopt control aspects of texture optimization, and settings presented here are based on Ethatron's recommendations, being an expert in texture compression and optimization in general. When first installing DDSopt, the GUI settings need to be updated. The screens below show the recommended settings based on experience with the vanilla Skyrim textures, but custom modded textures may require alterations from the "vanilla baseline" settings presented here.

Main Menu Settings

WARNING

This template is deprecated. Please revise markup to use Template:GallerySimple instead.

WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Behave


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Apply


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Ignore


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Compression


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Settings


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Constraints


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Alternate Constraints

Behave Menu Settings

WARNING

This template is deprecated. Please revise markup to use Template:GallerySimple instead.

WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Behave - Normal Maps


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Behave - Foliage Maps


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Behave - Color Maps


WARNING
This template is deprecated. Please revise markup to use Template:GallerySimple instead.
• 

  Behave - Alpha Maps

Main Menu Setting Descriptions:

{{{title}}}

---

{{{text}}}

The following explanation of settings are taken largely from Ehatron's OBGE page and augmented with information provided by Ethatron or gleaned ourselves, so it is incomplete and may even have some misinformation. Revisions to inaccuracies are always welcome for that to which we remain uncertain ... these speculations are highlighted in yellow.

• Game: Indicates the game for which texture optimization applies. Behavior is assumed to be dependent upon this selection. Select Skyrim to be safe.
• Behave: This is a complex menu with many options. Since it only has one child (Textures), which is broken down into a submenu, it is clear that either more 'children' were to be incorporated in later development of DDSopt, or this is a relic from NIFopt from which DDSopt was ported.
  • Produce partial derivative tangent-space normal-maps: Ensures that Ethatron's custom algorithm is applied when converting tangent-space normal maps. This should always be checked.
  • Raise normal-map steepness each mip-level: Recovers some normal-map contrast (curvature) when it becomes washed out from mip-map down-scaling.
  • Raise foliage-map opacity each mip-level: We think that this increases opacity of foliage edges (small leaves and branches) that tend to disappear from alpha mip-map down-scaling. This tweak works nicely for vanilla foliage textures, but is not appropriate for mods that already compensate for this, like Skyrim Flora Overhaul.
  • Gamma-correct mip-map filtering for color-maps: Enforces resampling of mip-levels in gamma space.
  • Contrast-correct mip-map filtering for alpha-maps: This may be similar to "Raise foliage-map opacity each mip-level", but for non-foliage textures. This may effectively sharpen the edges to increase contrast in some textures.
  • Normal-map steepness raise: Magnitude setting for "Raise normal-map steepness each mip-level".
  • Foliage-map opacity raise: Magnitude setting for "Raise foliage-map opacity each mip-level".
  • Color-map gamma: Magnitude setting for "Gamma-correct mip-map filtering for color-maps".
  • Alpha-map contrast: Magnitude setting for "Contrast-correct mip-map filtering for alpha-maps".
  • Don't eliminate lower mip-levels: This setting appears to be broken in pre-release version 4 (the recommended release) This tells DDSopt not to eliminate 1x1 mip levels, even though 1x1 mip-maps are totally redundant (this setting is for experimental purposes and is otherwise not useful).
  • Ignore borders on optimizing: Causes the optimizer not to consider/look at border regions (of the configured size) of the textures. This allows DDSopt to optimize textures which have an obvious artifacts from rescaling. It is also prevalent in the vanilla Skyrim-textures.
• Apply: These are global settings corresponding to the texture-type settings on the Constraints tab. Uncheck both of these items to extract BSAs without altering any textures.
  • Re-compress and re-mip all textures: Globally forces DDS-files to be revisited, optimizations like mip-mapping and channel-reductions only happen when this is on. This setting "turns on" the texture-type check boxes on the Constraints tab.
  • Compress all kown texture-formats: Enables conversion from fe. PNGs to DDS-DXT. This setting "turns on" the texture-type compression format context menus on the Constraints tab.
• Ignore: These settings conditionally prevent DDSopt resampling and processing.
  • Don't process any high dynamic range images (DDS, HDR, etc.): We assume that this means that DDSopt can recognize certain HDR texture types (presumably by use of the alpha channel), and avoids resampling them (but Resolution-limit constraints should still be effective).
  • Don't process any already processed files (DDS only): Recognizes existing files in the destination as already DDSopt'ed files and skips those ... especially helpful for creating down-scaled versions of optimized textures, as those won't be re-evaluated again. In other words, none of the Constraints tab settings will be applied EXCEPT FOR the Resolution-limit settings for textures already present in the destination.

  NOTE: the following two items may be checked to extract BSAs without processing similar to above under Apply.

  • Don't process any of the known file types at all (DDS, PNG, etc.): Prevents processing of "every known file-type". This puts DDSopt essentially into BSAopt-mode.
  • Copy unprocessed and ignored files (passthrough):Forces DDSopt to copy all assets (whether resampled or not) into the destination.
• Compression: Defines the compression-strength. "0" turns compression off. "9" is the normal maximum of zlib. "10 (quick)" will do a quick search for a better compression, "10 (maximum)" will do an exhaustive search for the best compression and usually takes longer. By default compression will be turned off when the compression is not very high (>95%). Compression can be forced, regardless the efficiency via "Forced".
• Settings: These are general settings that conditionally affect processing of files.
  • Skip existing files: Never overwrite a file in the destination directory (BSAs are considered directories),
  • Skip older files: Compares the timestamps of the source and the destination and overwrites only if the source is newer (BSAs define the timestamps of their contents).
  • Skip hidden files: Skips all hidden files. Changing this settings will automatically be reflected in the file-list.
  • Skip hash-check: Skips the sanity-check for BSAs (when recovering a broken BSA, for example).
  • Skip broken files: Suppresses applicable event dialog if some files can't be decompressed.
  • Write logfile: Writes processing events to a log file adjacent to the destination directory.

Tab Settings & Descriptions

Browser Tab

This is where source and destination directories are chosen and other operations are performed.

Source

This is the directory/BSA which DDSopt will scan recursively and display in the tree-window below. BSAs in directories will be scanned recursively. Click [Browse] to invoke the browser pop-up. To operate on folders, click on the directory to highlight, and click on [Use folder].

NOTE:

DDSopt automatically remembers the most recent directory locations chosen and will re-scan them the next time DDSopt is started, so when working with large directories it may be prudent to reset to smaller directories when all operations have been performed before exiting DDSopt.

Directory Tree Browser

Displays all sub-directories/BSA-paths of the overlayed selected source & destinations (i.e., this is a composite of source and destination folders/files). Click to expand or collapse directories in the tree or click on a directory to view only that directory's contents in the lower pane. The lower pane lists the path and file information of both source and destination directories at the node selected (see below).

File/Path Browser Toolbar

GUI tools

• The 'check' button selects all visible files.
• The 'X' button deselects all visible files.
• "Show recursive" makes DDSopt display full path information prepended to file names.

File/Path Browser Filter

This is the context menu and [Apply] button that exist between the upper and lower panes of the Browser tab. From the context menu, several pre-configured filters can be chosen, or entirely new filters can be typed into the field itself. Wildcards (*) are allowed. Filters are only applied once the [Apply] button is clicked. Examples:

• interface - anything with the word 'interface' within the file/path
• \\interface - any path/file containing '\interface'

NOTE:

The filter tool is very handy when combined with selection via the directory tree above it and the check/uncheck tools beside it. Used properly, this is much simpler and more customizable than using the batch script to differentially apply treatments to textures by location and name.

File/Path Browser

Lists the files/paths based on selections made in the directory-tree browser and filter above. Double clicking on a file/path will load the asset into the Preview tab. Relationships between source and destination are indicated by the color-coded path/file names in the lower pane:

dark green — unprocessed textures (exist only in source location)

blue — files that will only be copied from source to destination 9(passthrough). These will be optimized by DDSopt, due to non-image file type or settings chosen that would restrict processing of image files.

gray — file existing only in the destination

purple — ignored textures based on the Ignore menu setting (see above)

black-strikethrough — files that have already been processed by DDSopt (as configured in the Settings menu options)

italic — files are compressed in the source (i.e., the source is a BSA)

Destination

The destination directory or BSA. use the [Browse] button same as above.

NOTE:

In order to package the output into a BSA archive, append the archive name with extention as PATH\MyOptimizedTextures.bsa.

Preview Tab

This tab is used to preview individual textures and their properties. Invoke by double-clicking on a texture in the lower window of the Browser tab.

Path

The path of the texture being previewed.

Channel Context Menu

Used to select a texture channel (primary or alpha) and the diffs among source texture and destination (optimized) texture.

Preview

This shows a render of the texture. Cube-maps will be rendered as cross-probes. Normal-maps will be normalized before display.

Infos

Texture properties. The purpose(s) for the game, the dimensions and mip-levels as well as the format and compression the texture is saved in.

Format & SaveAs

Quick save the active image in a specific format. Only a minimal set of optimizations is applied.

Constraints Tab

Used to fine-tune the behavior of texture optimization based upon texture type (as defined by the 'known use cases': The following use-cases are known (for Oblivion and Skyrim):

• in a "menu"-folder: User-interface element, alpha is element-mask
• in a "landscapelod"- or "terrain"-folder: World-space normal-map for terrain, no alpha
• "_n"-suffix: Tangent-space normal-map, alpha is specularity
• "_msn"-suffix: Model-space normal-map, no alpha
• "_g"-, "_glow"- or "_emit"-suffix: Glow-map, alpha is a mask
• "_hh"-suffix: Gloss-map for hair, no alpha
• "_hl"-suffix: Detail-map for hair, alpha is opacity
• "_m"-suffix: Reflectivity-map for light-sources, no alpha
• "_em"- or "_envmap"-suffix: Reflectivity-map for environment-maps, no alpha
• "_e"-suffix: Environment-map (some are planar, some are cube-maps), no alpha
• "_b"- or "_bl"-suffix: Backlight-map, no alpha
• "_s"-suffix: Specularity-map for skins, no alpha
• "_sk"-suffix: Tone-map for skins, no alpha
• "_p"-suffix: Parallax-map, no alpha
• "_d"-suffix: Diffuse-map, alpha is opacity
• "_h"-suffix: Haze-map, alpha is unknown

All others are color-map, alpha is opacity or parallax

Optimize / Compress

The check box of each texture-type tells DDSopt to optimize that type. This is the individual equivalent to the menu option "Re-compress and re-mip all textures". If this option is not checked (or is greyed out) when a texture is already compressed or not scheduled for compression, nothing happens. These individual optimization settings do not seem to have any impact on processing results, so this feature may not be fully or properly implemented.

The context menu drop-down then tells DDSopt which format to compress when an uncompressed texture or raw image is scheduled for compression.

Resolution-limit

Cap the texture resolution independently for compressed or uncompressed formats after the texture has been optimized and possibly compressed.

Size-limit

This option is not configurable as far as we know.

Settings Tab

This area simply lists DDSopt-supported/unsupported file types and texture formats.

DDSopt Initialization File

The DDSopt INI file is currently under testing to confirm the expected behavior as described below and on the INI Settings page.
The DDSopt INI allows custom specification of the channel contents of specified images (base and alpha). There are three possible *.ini files that DDSopt can interpret:

• DDSopt.ini
• DDSopt-Oblivion.ini
• DDSopt-Skyrim.ini

The first is always read in, the other are read in dependent on the game you selected in the menu.

NOTE:

Changes to the INI file impact the current DDSopt session upon save.

This feature is practical when textures can not be matched by suffixes (like _n for tangent-space normal-maps). Also, the default behavior for Oblivion is to assume the alpha-channel can be anything, but for Skyrim it assumes by default that the alpha-channel is used for opacity. The INI allows specification of the base texture type and its use of the alpha channel (if present and used). DDSopt automatically applies specific treatments to Skyrim textures based on file directory location and/or file-name suffix, so the INI serves as a means to specify exceptions to these built-in rules.

INI File Specifications

The INI allows the user to list textures by relative path names in order to apply custom treatment during optimization. The purpose of the INI sections is two-fold:

1. Defining the compression format on the 'base' channel. Base channels are always used.
2. Defining the content and purpose of the 'alpha' channel. The alpha channel is often not used.

NOTE:

The 'base' usually contains 3 channels, which can mean 3 independent channels, or color or a vector, or xy + height etc.

Compression Formats

subdirectoryoftextures/texturename.png=DXT -> compress to a suitable DXT-format (DXT1, DXT3, etc.)

subdirectoryoftextures/texturename.bmp=BIT -> compress to a suitable bitfield-format (565, 4444, etc.)

subdirectoryoftextures/texturename.dds=RAW -> don't compress if uncompressed, otherwise continue as usual

subdirectoryoftextures/texturename.png=SKP -> don't do anything at all with the texture

Syntax

- Exclude the "texture/"-path level (it will be stripped anyway if included)

- Where applicable, the compression format specification may be omitted to indicate a specific file-type that otherwise wouldn't be recognized by DDSopt.

INI file and GUI behavior is currently under testing

PRELIMINARY FINDINGS:

• Compression-format specifications work as applied within the 'base'-related sections
• Special treatment of textures is handled by the GUI if the GUI is instructed to do so. This means that specific textures are accounted for within DDSopt's processing instructions (added by Ethatron)
• Textures that do not follow the known Bethesda naming conventions AND are not explicitly accounted for within DDSopt's instructions may be listed in the INI under the appropriate section to notify DDSopt to apply special processing instructions.
• Foliage alpha opacity settings are confirmed to work when either INI or GUI, whichever applies (see previous), is set accordingly; however, the magnitude setting seems to have no effect, regardless of INI or GUI settings (this has not been exhaustively tested though)

Supported 'Base' Types

(compression format IS supported here)

• [NormalsTangentSpace]

  - mark textures to be tangent-space normal-maps

  - enables specific compression taking unit-halfsphere property into account

• [NormalsModelSpace]

  - mark textures to be model-space normal-maps

  - enables specific compression taking unit-halfsphere property into account

• [DiffuseColor]

  - mark textures to be color converted

• [DiffuseGrey]

  - mark textures to be greyscale converted

Supported 'Alpha' Types

(compression format IS NOT supported here)

• [AlphaOpacity]

  - mark textures to have an alpha-channel with opacity, enables specific compression taking this into account

• [AlphaFoilage]

  - mark textures to have an alpha-channel with foliage, enables specific compression taking this into account

• [AlphaCustom]

  - mark textures to have an alpha-channel with something custom, disables specific compression only working for opacity

All of them (base + alpha) have their own processing. Normals are normalized, tangent-space vectors are possibly stored partially derivative, opacity gives importance to color (the more transparent the less important the exact color-value), foliage is possibly made sharper. The menu settings correspond exactly to each of these types. The menu settings to which these sections correspond remains unclear.

Launching an Optimization Pass

Before processing any textures through DDSopt, the program must first be pointed to a valid location containing recognized image files. DDSopt can process several types of textures (DDS) or image files (e.g., *.PNG, *.BMP, etc.). A "valid location" is any directory location containing valid image-file formats. This "directory" can also be a BSA file; BSAs, like any archive file format (e.g., TAR, CAB, 7z, ZIP, RAR, etc.), are really directories with path information contained within an 'archive' that may or may not be compressed. The file itself acts as the ROOT directory. Once the source and destination directories have been selected, optimization can begin.

[edit]

DDSopt INI

Frequently-Asked Questions & Answers

DDSopt Usage Examples

Benchmarks & Performance Examples

Quick-Start Guides

Vanilla Skyrim Texture Optimization

Vanilla Fallout Texture Optimization

Texture Viewers

AMD's Compressonator - useful diff tool for texture comparisons of textures before and after processing through DDSopt.

Irfanview - A very compact but useful image viewer with support for DDS file types and other advanced features.