GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Neovalen]

Yeah seem to be getting quite a few blacks with the latest nexus release.

[Daagar]

z... way back on post #430, you have some suggested DDSopt settings in the form of screenshots. It seems that the last picture isn't displaying for some reason. And I just _know_ it is going to be the most vital setting :) Can you repost or put a text description of what that picture shows? Thanks!

[z929669]

z... way back on post #430' date=' you have some suggested DDSopt settings in the form of screenshots. It seems that the last picture isn't displaying for some reason. And I just _know_ it is going to be the most vital setting :) Can you repost or put a text description of what that picture shows? Thanks![/quote']

See the OP for details

[Neovalen]

Here is another black texture (latest on nexus):

textures/trees/treepineforestaddon01*.dds

 

This is the Mora Tapinella file. (totally not where I thought it would be)

[Kelmych]

This is how optimized textures look on my end. I used the 0.8 prerelease from Nexus.

I looked in more detail at the inn sign texture I was using (from Bumpy Inn and Shop Signs) and found that it had been optimized using DDSopt 0.8 pre5b, the version before the new update (DDSopt 0.8 update 1) on Nexus. When I run the texture file through the newest release from Nexus the texture also turns black for me when viewed in DDSopt preview mode and ingame. The normal map is fine, but the texture isn't.

[Kelmych]

Screens are never unappreciated by the likes of me ;)

I processed the uncompressed LOD using DDSopt v080 update 1 from Nexus, and looked the LOD texture with several texture viewers (Irfanview, Paint.net, Windows Texture Viewer, and the preview mode in DDSopt; I haven't been successful installing Compressonator on my system). It looks different in each viewer, and I can't decide which one I should post here. There seem to be two layers, the tree LODs from the original LOD texture and "in front" of that is another partly transparent texture. The partially transparent layer is the part that looks different in each view.

 

I then repeated the optimization using DDSopt v080 pre5b, the version prior to the update 1 posted on Nexus. The LOD texture looks fine; there is no partially transparent layer "in front" of the texture.

[PCG4m3r]

I also am noticing the same black textures as mentioned in previous posts, using the latest 8.0 pre-release update 1 from the Nexus. However, when I compared the same textures that I processed using 8.0 pre5b everything looked fine.

 

I found another texture (mushrooms on stumps) that appears black in game after using pre- update 1. I wasn't sure what texture file it was coming from though (guessing one of the files in \textures\plants). Below is a link to my imgur album that shows the before/after images. I apologize, but I couldn't figure out how to link directly to the images themselves or how to upload to the wiki so that they would be embedded in this post.

https://pcg4m3r.imgur.com/

 

Also, I'm following the recommended DDSopt settings per the OP.

[Daagar]

I tried re-converting the riverwood inn sign that uLoop used using 0.8 pre5b and it works fine there. I was able to duplicate the 'black' error with update 1. I haven't tried the other black textures yet, but would assume the same results there.

 

EDIT: Beaten by PCGamer above. Doh!

[z929669]

Screens are never unappreciated by the likes of me ;)

I processed the uncompressed LOD using DDSopt v080 update 1 from Nexus, and looked the LOD texture with several texture viewers (Irfanview, Paint.net, Windows Texture Viewer, and the preview mode in DDSopt; I haven't been successful installing Compressonator on my system). It looks different in each viewer, and I can't decide which one I should post here. There seem to be two layers, the tree LODs from the original LOD texture and "in front" of that is another partly transparent texture. The partially transparent layer is the part that looks different in each view.

 

I then repeated the optimization using DDSopt v080 pre5b, the version prior to the update 1 posted on Nexus. The LOD texture looks fine; there is no partially transparent layer "in front" of the texture.

Only the 32-bit installer of the compressionator works. the 64-bit is broken. unfortunately, I don't think that any of those other programs will let you get much out of viewing normal maps.

[z929669]

It looks like in addressing issues with pre5b (the original vesion of 8.0 on the nexus I think), Ethatron may have broken something else. I would refrain from using the latest release until we get this ?bug? fixed.

[Kelmych]

Only the 32-bit installer of the compressionator works. the 64-bit is broken. unfortunately' date=' I don't think that any of those other programs will let you get much out of viewing normal maps.[/quote']

Thanks.  I wish they mentioned that on the AMD site and in the threads I read where others were unable to install the 64bit version.

[Ethatron]

@Ethatron

 

Q.2: In the DDSopt output, there are several lines that make up the Final Report. Ex:

 

Final report "D:\Skyrim Mods\Mod_Working\DDSopt\out":

 processed files: 48

 modified textures: 27

 skipped textures: 0

 fixed textures: 0

 fixed files: 0

 broken files: 0 (0 without fixed ones)

 planar (1x1) textures: 0

 changed texture formats: 16

 i/o delta: -36413610 bytes

 tex delta: -21670756 bytes

 

The information contained in certain of these is pretty obvious; however, a couple of variables are somewhat unclear as to what can be gleaned (highlighted). Could you briefly describe these and provide an example or two of values that might raise and eyebrow versus those you would expect (e.g., a previously "optimized" texture set might yield a very different result than an "un-optimized" texture set or even a result from applying constraints). I have seen very large positive bytes for i/o as an example.

A.2: That's tough. Let's try:

• "fixed textures": The number of corrupt textures that have been fixed, this are currently textures which are too short (few bytes missing at the end). I found one or two of those, I don't know which software creates these truncated textures.
• "fixed files": A remnant of NIFopt, will always be 0. Refers to repaired models mostly.
• "planar (1x1) textures": The number of textures which have only one distinct value. These textures may be converted to 1x1 size without loss. Though seems not all hardware likes that. Haven't yet understood what's the real cause.
• "i/o delta": This is the global incoming to outgoing delta. Say you process a BSA, then it takes the size of a file inside the BSA (possibly compressed, then it takes the compressed size), and does the processing and writes it into another BSA. The sum of those differences is that value. If a DDS is not touched it still is possible to get i/o deltas != 0 because the zipper in BSAopt/DDSopt is about 10% better than Bethesda's. If you decompress a compressed BSA to directories, this delta is likely positive (out is bigger than in).
• "tex delta": This is only the recorded change in size from changes to texture-formats, resolution and mip-maps.

EDIT: also want to know if the following resources will help us to understand some of the basics of texture compression (in order to better equip us to maximize our comprehension of your responses :P ):

 

https://mines.lumpylumpy.com/Electronics/Computers/Software/Cpp/Graphics/index.php

 

https://doc.spatial.com/index.php/Texture_Mapping

 

https://www.gamasutra.com/view/feature/2496/texture_compression_techniques_and_.php

The Gamasutra is okayish for understanding DXT, the others are unrelated.

This one's extensive for modelers:

https://wiki.polycount.com/NormalMap/

 

What exactly do you want to understand? The DXT artifacts (DXT3 vs. DXT5)? Why normal maps in DXT sucks?

[Ethatron]

Found a texture that is causing issues when optimized:

 

 

textures/clutter/scrollworktrim02.dds (causes black and white blotches in game on the standing stones)

There is a glow-map instead of an opacity map in the alpha channel. Added to the exeption list.

DDSopt corrupts sign textures' date=' for example textures\clutter\signage\riverwood\riverwoodsleepinggiantinn01.dds' date=' it is almost completely black ingame. Other similar signs (inns, traders, etc.) could also be affected, but I have not tested that yet.[/quote'']

There are spec.-maps instead of a opacity maps in the alpha channel. Added to the exeption list.

Hm, these are also buggy. Fe. "mrksignalchemyshop01" is correct, it really is opacity in the alpha-channel.

[AOF HD Tree LODs] The resulting texture looks very different than both the original uncompressed and the 2K compressed LOD (which are quite similar).

Confirmed' date=' another 5 character stupidity (wrong default compression format, was ATI2, LOL).

Here is another black texture (latest on nexus):

textures/trees/treepineforestaddon01*.dds

 

This is the Mora Tapinella file. (totally not where I thought it would be)

Puh, this are actually buggy Bethesda textures ...

These textures are shared between models which do not use an alpha-channel (mushroom fe.) and models which do (the moss is a stump decal). Now fully transparent stuff is ... nothing of course. So the mushroom becomes nothingness. If someone would paint the solid area white in the alpha-channel, it could pass as a foilage-texture type. The wood-block ("treepineforestcut01") is done correctly, the shrubs too (tundrashrub*.dds). Sometimes I almost think I can distinguish the texture-artists working at Bethesda by the care given to the textures. Some textures are just so wrong, and you can group them by wrongness, and I expect each group is made made the same guy.

Added to the exeption list.

[AlphaCustom]
; Vanilla Skyrim (spec. maps, not opacity)
clutter\signage\riverwood\riverwoodsleepinggiantinn01.dds
clutter\signage\riverwood\riverwoodsleepinggiantinn01.dds

; Vanilla Skyrim (glow maps, not opacity)
clutter\scrollworktrim02.dds

; Vanilla Skyrim (this is a shared map, preferrably the mushroom needs a white alpha)
landscape\trees\treeaspenaddon01.dds
landscape\trees\treepineforestaddon01.dds
landscape\trees\treereachaddon01.dds

[Ethatron]

It looks like in addressing issues with pre5b (the original vesion of 8.0 on the nexus I think)' date=' Ethatron may have broken something else. I would refrain from using the latest release until we get this ?bug? fixed.[/quote']

We just need a bit more rigorous classified lists of textures from now on. It's actually rather easy to check for those special maps which have something else than opacity in the alpha-channel (normal-map alpha-channels are never! treated as opacity, so no problem there):

IrfanView - by default - assumes alpha-channel being opacity, if you use the thumbnail browser (press T) you may identify strange looking black color-textures where you'd assume it really should not be black really.

The DDS-thumbnail extension for the explorer - by default - never even applies the alpha-channel. Having both side by side makes it easy to detect the strangers/outliers if you have a good instinct what a transparency-mask should do, and what it likely shouldn't (fe. eliminating mushrooms).

[z929669]

What exactly do you want to understand? The DXT artifacts (DXT3 vs. DXT5)? Why normal maps in DXT sucks?

Everything' date=' just point to whatever you think is a good starting point for someone like me trying to understand the basics for getting the most out of someone like you ;)

 

Introductory stuff that you think is pertinent to our purposes in modding and optimizing textures, including prerequisite topics that you might think are building blocks to understanding the relevance of various compressed (and uncompressed) texture formats.

 

the links you have already provided are a great start.

 

& thanks for A2. that is helpful ;)[hr']

It looks like in addressing issues with pre5b (the original vesion of 8.0 on the nexus I think)' date=' Ethatron may have broken something else. I would refrain from using the latest release until we get this ?bug? fixed.[/quote']

We just need a bit more rigorous classified lists of textures from now on. It's actually rather easy to check for those special maps which have something else than opacity in the alpha-channel (normal-map alpha-channels are never! treated as opacity' date=' so no problem there):

IrfanView - by default - assumes alpha-channel being opacity, if you use the thumbnail browser (press T) you may identify strange looking black color-textures where you'd assume it really should not be black really.

The DDS-thumbnail extension for the explorer - by default - never even applies the alpha-channel. Having both side by side makes it easy to detect the strangers/outliers if you have a good instinct what a transparency-mask should do, and what it likely shouldn't (fe. eliminating mushrooms).[/quote']

So you are asking that we examine the source file of all *problem* textures and identify what channels are being used for each and for what purpose?

 

Sounds like a good hands-on learning experience. Can you point to some basic reference for the standards?