GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[TechAngel85]

Hey guys,

 

You may what to take a look at this: https://forum.step-project.com/showthread.php?tid=1799

 

Seems like some sky textures are missing from the INI? It's possible that the user didn't copy it correctly though. Just deserves a second look.

[Kelmych]

When Ethatron looked at this issue with DDSopt 0.8.8 release 3 he made changes in DDSopt itself (DDSopt 0.8.8 release 4) and removed the sky textures from the DDSopt.ini. We haven't checked all the previous issues, such as the sky issue, yet to see whether they were solved by his changes in DDSopt release 4.

[Kelmych]

z and I have made some changes in the DDSopt and Texture Overhaul guide to try to make it easier to read (using a new template for sidebars, primarily for ancillary information so that it is closer to where it is needed in the processing steps) and adding some material based on recent questions in the forum.

[z929669]

I am still working slowly through all of the guides ... mostly addressing cohesion and articulations, but getting caught up in detail nonetheless :P it's gonna be awhile

[z929669]

Hey guys,

 

You may what to take a look at this: https://forum.step-project.com/showthread.php?tid=1799

 

Seems like some sky textures are missing from the INI? It's possible that the user didn't copy it correctly though. Just deserves a second look.

I responded with the solution to this issue on THIS POST

[z929669]

z and I have made some changes in the DDSopt and Texture Overhaul guide to try to make it easier to read (using a new template for sidebars' date=' primarily for ancillary information so that it is closer to where it is needed in the processing steps) and adding some material based on recent questions in the forum.[/quote']

I have yet to read your new edits, but I am really liking the Boxes. Looks good.

[Kelmych]

I added a new Note 6 in the STEP mod table to indicate mods that are most likely to provide game performance improvements when reduced size normal maps are used. I put the note on some of the mods. As we do more testing  we will hopefully be able to find more mods that yield reduced VRAM use using this technique.

[z929669]

Good call

[EssArrBee]

The 'n' is missing from the *_.dds -> *_n.dds.

 

I tried to find if the engine reduces the mip-map level by 1 to match the footprint of DXT1, since half size uncompressed normals look better than full size compressed normals. I know that a few other engines do this reduction automatically, Unreal comes to mind. I mention this because if you reduce the uncompressed normals and then the engine does it again, then the normals resolution is now reduced even further and the resolution can only be reduced so much before the it does look worse in game.

[z929669]

The 'n' is missing from the *_.dds -> *_n.dds.

 

I tried to find if the engine reduces the mip-map level by 1 to match the footprint of DXT1, since half size uncompressed normals look better than full size compressed normals. I know that a few other engines do this reduction automatically, Unreal comes to mind. I mention this because if you reduce the uncompressed normals and then the engine does it again, then the normals resolution is now reduced even further and the resolution can only be reduced so much before the it does look worse in game.

All of this can be done, but you have to configure each run independently. That level of automation does not exist at this point. You would also only apply this to source image files.

[Kelmych]

There is now an initial version of a QuickStart tab in the guide with a brief one page overview and roadmap of the steps needed for texture optimization.

[Kelmych]

In Question 3 of the recent Q&A with Ethatron (available in the DDSopt Technical FAQ tab in the guide) Ethatron mentioned that compressing normal map textures using R5G5B5 at half resolution was better than using DirectX compression at half resolution if there is enough noise (no smooth regions) in the texture. How can someone tell by looking at images with the DDSopt Preview viewer of the main texture (primary and alpha channels) and the associated normal map (primary and alpha channels) whether or not this is the case? What are the best clues for this?

[z929669]

The compression type should be indicated in the preview tab (at bottom under format: EX. DirectX texture; A8R8G8B8 (converted to DXT5)). You can also play around with converting single textures here I believe (have not played much with that though).

 

This also applies specifically to tangent-space normal maps (model space normals are always recommended by Ethatron to be R5G6B5 compressed). Normally, the advice is to use half-sized uncompressed TSN maps, but with noisy textures, the RGB compression format does work nicely (try it on Terrain Bump as an example) ... and reduces the size by half again ;)

 

Questions:

Once a texture is compressed, it cannot be "uncompresed" without losing the original fidelity, can it?

 

Also, is it possible to convert compression formats without loss of fidelity?

 

IOW, is compression like a ratchet effect, or can it be reversed?

[Kelmych]

There have not been any comments about my post that DDSopt reduces the quality of the textures from Psychosteve's Golden Shrine mod. The original textures have a lot of very small imperfections that are lost when compressing with DDSopt. The alpha channel has only one value. I don't see any of the specific exception categories in DDSopt.ini that would apply to the problem with these textures, so we would need to add these textures into the DDSopt.ini with a SKP entry. I suggest we add these to DDSopt.ini.

[Kelmych]

The compression type should be indicated in the preview tab (at bottom under format: EX. DirectX texture; A8R8G8B8 (converted to DXT5)). You can also play around with converting single textures here I believe (have not played much with that though).

 

This also applies specifically to tangent-space normal maps (model space normals are always recommended by Ethatron to be R5G6B5 compressed). Normally, the advice is to use half-sized uncompressed TSN maps, but with noisy textures, the RGB compression format does work nicely (try it on Terrain Bump as an example) ... and reduces the size by half again ;)

 

Questions:

Once a texture is compressed, it cannot be "uncompresed" without losing the original fidelity, can it?

 

Also, is it possible to convert compression formats without loss of fidelity?

 

IOW, is compression like a ratchet effect, or can it be reversed?

Lossy compression cannot be undone. You can transform between lossless formats (e.g., between the various lossless audio formats such as FLAC and Apple Lossless) but once you apply lossy compression (e.g., MP3) there is no way to recover the original.

 

My question was about whether by looking at an image before compressing the tangent space normal maps there are some clues that would indicate which type of TSN compression to use; e.g., if an image has a lot of detail in most places but has regions that are fairly smooth is it "noisy"? If we need to try both TSN compression formats and look at the comparative results in the game it is very time consuming; I'm looking for an easier approach.