GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[tony971]

That is all in turmoil right now and under construction/destruction. If you use the following constraints prior to optimizing, you will be fine:

 

Use the following filters (copy/paste into filter box on [browse] tab and click [Apply] to activate a filter before ticking/unticking files in the window). Filter masks changes to anything other than what the filter currently has applied, so if you want to see the overall result, click the root and select the *.* or blank filter and hit [Apply]:

 

*facemods

*facetint

*tintmasks

interface

coastbeach0*_n.dds

dirt0*_n.dds

dirtpath0*_n.dds

dirtsnowpath0*_n.dds

snow0*_n.dds

tundra0*_n.dds

Optimized Vanilla Textures has been re-based on these suggestions. The only change is that any textures that were collapsed to 1x1 by DDSopt were replaced by ones collapsed to 16x16 or 16x4 by Optimizer Textures. This is why. The "true" unique thought of Optimized Vanilla Textures is the installation method. The conversation has changed from "loose vs BSA" to "plugin or no plugin." Both options have all other texture mods override them, including the unofficial patches. Not all tools were equipped with proper handling for the plugin-less installation method, though, so I had to campaign to add functionality.

This is the current state of compatibility with the plugin-less method.

[z929669]

Thanks for the info ... the link you provided indicates power of two ... so that would be 2, 4, 16, 64, etc ... so why are 1x1 replaced by 16x16 or 16x4?

 

Can you provide an example of one of these textures and how it appears in game when 1x1 versus 16x16(4)? I have yet to find any issues with 1x1 in game, but I don't doubt that there could be issues. Will look into the OP of this thread to see if we ever asked Ethatron.

 

EDIT: Ethatron did address this some time ago in response to our first question in the OP. He prefers to throw away all mip levels under 4x4 ... so that is fine, but I see plenty of 2x2 and 1x1 mip levels still (in DDSopt'ed textures) ... need to test other versions of DDSopt I guess, since the "Don't discard lower mip levels" does not seem to be working in pre-release 4.

[tony971]

16x16(4) just seemed like a good number, as objects won't be smaller than that. The difference won't appear in-game. It's all performance-related. alt3rn1ty has a lot to say on the subject if his attention is drawn to this thread. Here are all of the un-collapsed textures if you want them.

[z929669]

So Optimizer Textures allows one to strip off lower mips according to user-supplied definition? Ethatron spoke of his 4x4 rule, but I don't think DDSopt strips anything other than 1x1 on 2:1 sized textures.

 

let's get alt3rn1ty in here ...

[tony971]

Optimizer Textures auto-collapses patterned textures to 16x16 (or smaller if you want) and then you can user-define to cut the other textures' sizes in half until you get them to your desired size.

Edited October 13, 2014 by tony971

[z929669]

Wait, I am talking about mip-map levels here ... are you talking about the native texture dimensions? It would be very nice if we could eliminate all mips below 4x4 if possible.

[tony971]

Oh I went through every single dds in the game after running DDSopt and checked for 1x1 textures. Doesn't matter if they were a mip or native texture. If DDSopt collapsed them, I replaced them.

[z929669]

OK, so you went through the logs I am guessing looking for "to 1x1" string or something like that? you won't ever get rid of small mip levels that way, but you will effectively up-size the small textures (but they will still get the 1x1 mip, presumably when upscaled to 16x16 if re-mipped)

 

Example:

D:Skyrim ModsMod_WorkingOptimizationSkyrim Vanilla Textures (cleaned source)texturesactorscharacterargonianfemaleblankdetailmap.ddsprocessing: Format		   : DXT1 to A8R8G8B8 Dimensions	   : 256x256 to 1x1 - 9 to 1 levelsnotes: Planar image detected, collapsing to size 1x1. Texture was compressed.delta:		    43700 bytes less

I recall Ethatron saying something about this (which is not addressed in the OP; we are referring to mip-map levels there). The idea is that the original in the above case was 256x256 and monochromatic, so collapsing to 1x1 is theoretically no different in appearance when tiled (stretched?) onto whatever object would use that. The original parent object was 256 and also had all the mip levels (including 1x1, 2x2, etc.), so since there is no scaling issue for monochromatic, it makes sense to reduce.

 

My problem is that I am not sure if Skyrim will even use 1x1 ('native' or a mip-map of the parent texture), and if it does, I wonder if tiling (and sampling) a bunch of these is detrimental to GPU performance even though it would seem beneficial to GPU memory performance. Or maybe they are just stretched? Stretching a 1x1 would seem to be hugely beneficial to using a single 256x256.

 

I am just trying to determine once and for all if the above examples really are a problem and, if so, what is the effect in game? Same goes for tiny mip-maps below 4x4 (i.e., when viewing an object in game from 'far away' or at a very steep angle)

[tony971]

No, I went through every texture in the output directory and arranged by file size. It took forever. I think the best way to test performance would be to make a test zone in which a lot of the collapsed textures are present. Test it once with the 1x1 textures made by DDSopt and then again with the 16x16 textures I provided earlier.

Edited October 13, 2014 by tony971

[z929669]

I guess I am more interested in the appearance of said 1x1 textures in game. Performance is also interesting, but that is secondary. i just want to confirm that DDSopt-collapsed textures present qualitative issues in game.

 

Has anyone seen this?

[z929669]

In additionon to the 1x1 texture issue, I am trying to confirm my findings regarding the DDSopt INI.

 

In my testing, I am finding that the INI entries do not confer any treatment changes unless one adds one of the four treatments to the texture path entries (e.g., ==BIT, =RAW, etc.)

 

For example,

landscapetreesreachtreebranch01.dds

 

... is an entry Ethatron placed under [AlphaFoliage], but even if I comment that entry (closing/restarting DDSopt between INI changes), there is no change to the behavior of texture processing. The result is identical either way. I have tried this with "Raise foliage-map opacity each mip -level" both checked and unchecked. (that setting does not seem to have any effect either way, so both the menu setting and the INI entry have no effect on the processed result as far as I can tell comparing diffs in Compresonator at all mip-levels)

 

Can anyone confirm? (using pre-release v4)

[hellanios]

WOW, all these changes! It's really nice to see that you still seek the best even when things seem "stable".

[Kelmych]

The comment by Ethatron about monochromatic textures is:

• planar (1x1) textures - The number of textures which have only one distinct color value (monochromatic). These textures may be converted to 1x1 size without loss, which reduces size and should increase performance, although not all hardware likes 1x1 (reasons unknown at this time).

For me the issue with collapsed textures is whether they change the game display, and I haven't seen any evidence one way or another from a carefully setup test. What we should do is some testing, although with such small textures it might be hard to see any significant differences.

 

There was a comment that collapsing caused a problem with facial textures, and perhaps there is a problem for facial textures. I'm personally more interested in how the thousands of terrain textures in Skyrim, many of which are monochromatic or nearly so, are displayed when collapsed vs. their original size. In the vanilla Skyrim textures the only texture types with a lot of monochromatic (or nearly monochromatic) textures are terrain textures and facial tint textures.

 

I don't see how the KatsBits article is directly relevant to this. The comments might be correct for large textures, but it's hard for me to extrapolate from what is assumed to happen with large textures to a special case of a collapsed or partially collapsed texture. A much better understanding of exactly how a graphics card responds to collapsed or partially collapsed textures would be needed. I've never seen any comments from a graphics card developer about this.

[z929669]

There are hardly any truncated textures in Skyrim, but Ethatron pointed to a few. These might be affected as stated by the KatBits article, but I don't see how that article relates to 1x1 textures (other than 1x1 is not a power of 2). The point is that 1x1, being monochromatic, will not be distorted as the KatBits article suggests (although that article does not explain the issue very well), and sampling missing info from a monochromatic pixel seems elementary.

 

Ethatron is really an expert on these things, and his statement is what I would go with. He states that downscaling monochromatic to 1x1 should increase performance (GPU memory), and the KatBits article seems to indicate that it decreases performance (GPU processor). Based on what Ethatron said, the GPU memory gain is probably significantly higher than any speculated processor cost. The only issue is the hardware.

 

I don't think it will really make a difference, and most of the exceptions we have in the INI and that we impose on certain texture types (e.g., tintmasks), don't seem to have any solid evidence as being the best approach. I am trying to get some actual before/after screen examples of 1x1 texture issues.

 

Now ... how the heck do we find objects in the game that use these textures to any noticeable degree? It sounds like faces are a good start, but we will need to verify somehow. maybe before/after screens of various faces. But we need to find the others outside of faces too I think.

---

EDIT: The best test is probably to be as conservative as possible and optimize as much as possible and let user feedback help us to isolate the true use cases and make exceptions based on real evidence.

[tony971]

I don't believe that there would be any visual regressions. Even if it's stretched weirdly, it's monochromatic. There's not a lot to distort. The only issue would be hardware (and the Creation Kit). CK users have reported many times that if the facetints are touched, the CK freaks out and they end up with gray face. The difference between 1x1 and 16x16 is negligible, though. If using 16x16 textures potentially avoids hardware issues, I don't really see a downside to using them.

Edited October 15, 2014 by tony971