GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Nearox]

I modified the first 4 columns in the STEP mod table in the guide. These are now:

• Did DDSopt improve the textures by noticeably reducing total texture size or adding missing mipmaps
• Is DDSopt recommended to improve textures for this mod (No means the author has already optimized it sufficiently, so use DDSopt only if needed to get a reduced resolution texture that isn't available in the mod).
• Does the mod include exterior textures; when VRAM reduction is needed these are usually the best candidates for size reduction, particularly the tangent space normal maps
• 4th column is TBD

The new first column combines the results of the original columns 1-4.

 

The intent of the table is that DDSopt should always be used if column 1 is Yes, it is recommended unless Column 2 is No (in which case it should be used carefully if used at all), and is often particularly useful for VRAM reduction for mods with Yes in column 3.

 

Some additional recommendations need to be added about handling uncompressed and compressed normal maps.

 

Let us know if there are disagreements about entries or columns in the revised table; this is the initial version.

Awesome work! :)

 

May I just suggest that somewhere it could be noted - if the information is available - how big of a VRAM reduction it leads to/could potentially lead to. This does not need to be a precise number but can be an indication. E.g. 3 levels:

 

Minor (1-10mb average potential/reported VRAM savings in a Skyrim area where the mod in question is heavily present). E.g. Ingredients/small armor mods

Moderate (11-25mb... ".....") . this could apply to e.g. mods like cloaks of skyrim, mods that alter creature textures etc.

High (26 and up mb... ".....") this could apply to e.g. larger texture packs

[z929669]

Here are screen images for different versions of snow01_n.dds . The same Winterhold save file that z provided was used, but the images here are looking away from Winterhold. The view toward Winterhold has too much windblown snow in the foreground so it's hard to compare images. There is a noticeable difference between the images for vanilla and the optimized vanilla normal map textures, but no significant difference that I can see for the original/optimized SRO textures.

 

vanilla TSN            optimized vanilla TSN             SRO TSN                              optimized SRO TSN

These are not at all consistent with my results using the BAT procedure (tech has the same issue as I have with the optimized texture ... unoptimized has very little tiling effect).

 

I think that you need to stick with the original screen shot, as this will capture more of the lower mip levels, where I think the problem is exacerbated. The optimized version should be much worse.

 

Still not sure if SRO will have this problem or not, but I think that the lack of any subtle tiling effect in the close-up unoptimized version shows promise (note that unoptimized vanilla shows signs of tiling).

 

Are you certain that your vanilla/vanilla optimized images aren't flipped?

[Kelmych]

Awesome work! :)

 

May I just suggest that somewhere it could be noted - if the information is available - how big of a VRAM reduction it leads to/could potentially lead to. This does not need to be a precise number but can be an indication. E.g. 3 levels:

 

Minor (1-10mb average potential/reported VRAM savings in a Skyrim area where the mod in question is heavily present). E.g. Ingredients/small armor mods

Moderate (11-25mb... ".....") . this could apply to e.g. mods like cloaks of skyrim, mods that alter creature textures etc.

High (26 and up mb... ".....") this could apply to e.g. larger texture packs

I'm not sure I know how to determine how much VRAM is decreased in the game without testing. DDSopt includes the file size decrease or increase in the logs, but I'm not sure how that translates to actual VRAM use. In the benchmarks referenced in my signature I reported several VRAM statistics; I am not sure which of these is best to include in the DDSopt table; my suggestion is the peak and average of a running short term averaged value of VFRAM use

---

[z929669]

Peak and average dedicated VRAM use is probably best (see GPU-z). If the two are close, then the ceiling is likely being reached, and others with 3-4 GB video cards should probably assess. Also, testing VRAM should only apply to mods that we know are pretty VRAM intensive (Sec F-G mods in particular).

 

I just purchased a 7970 with 3 GB, so I will be testing over the long term.

[Kelmych]

Since z suggested using the original Winterhold view here are screenshots of this view with different versions of the snow01_n.dds normal map.

 

   

Vanilla           Optimized vanilla          SRO        Optimized SRO

 

 

 

Unoptimized Serious HD           Terrain Bump

 

The striping in the snow in the midground with Vanilla optimized and Serious HD was surprising, so I did these twice to make sure they were correct. I got a similar bad result with the original snow01_n.dds texture from Skyrim HD, by the way. The optimized version of this texture for Skyrim HD and Serious HD showed a very similar tiling effect to the unoptimized texture from these mods.

 

My very tentative conclusion is that for this one normal map texture optimizing with DDSopt introduces new problems only with the vanilla texture. It would probably be more accurate to say that DDSopt makes the small "rippling" effect present in vanilla much worse. This would suggest that we should keep DDSopt from optimizing the vanilla landscape normal map textures for tiled, smooth ground textures using exclusions in the batch files, but not necessarily change DDSopt.ini to skip these textures for all mods.

 

I've tentatively identified 8 normal map textures to exclude in the batch processing in textures\landscape .

[z929669]

Since z suggested using the original Winterhold view here are screenshots of this view with different versions of the snow01_n.dds normal map.

 

   

Vanilla           Optimized vanilla          SRO        Optimized SRO

 

 

 

Unoptimized Serious HD           Terrain Bump

 

The striping in the snow in the midground with Vanilla optimized and Serious HD was surprising, so I did these twice to make sure they were correct. I got a similar bad result with the original snow01_n.dds texture from Skyrim HD, by the way.

 

My very tentative conclusion is that for this one normal map texture optimizing with DDSopt introduces new problems only with the vanilla texture. This would suggest that we should keep DDSopt from optimizing the vanilla landscape normal map textures for tiled, smooth ground textures but not necessarily change DDSopt.ini to skip these textures for all mods.

 

I've tentatively identified 8 normal map textures to exclude in the batch processing in textures\landscape .

Perfect verification now. I agree totally with your assessment.

 

I would also add that it would seem that Skyrim HD and Serious HD both used the vanilla source texture and mipped either with DDSopt or using a very similar algorithm. SRO must have used the vanilla texture only as a guide and maybe made it larger from a fresh source and using a compatible mip-mapping algorithm.

[Kelmych]

I expect Terrain Bump will be last mod loaded with these textures so it's textures will be the ones actually used in the game. Nonetheless, I'll have the batch file provide the unaltered vanilla textures in case Terrain Bump is not in use. I've modified the batch file and I'm testing it.

[Kelmych]

The batch files for optimizing vanilla textures have been changed to support bypassing optimization for the vanilla textures discussed in the previous posts. As mentioned in the last post, Terrain Bump replaces all these normal map textures, so this won't affect what STEP users see on screen.

 

For those who have already finished the optimization of the vanilla textures using the batch files, it is fairly easy to achieve what this new change achieves without rerunning the optimization. Copy the following files from the textures\landscape directory in the extracted version of Skyrim - textures (the STD folder in Vanilla Extracted in the DDSopt guide) to overwrite the equivalent textures in the textures\landscape directory in the archive file of the optimized version of these vanilla textures:

coastbeach01_n.dds

coastbeach02_n.dds

dirt01_n.dds

dirt02_n.dds

dirtpath01_n.dds

dirtsnowpath01_n.dds

snow01_n.dds

snow02_n.dds

tundra01_n.dds

tundra02_n.dds

[EssArrBee]

How do I force DDSopt to make the _sk.dds textures into 8888 format? I'm doing a little testing and can't figure out which constraint is the right one.

[Kelmych]

The DDSopt constraints do not include specified handling for special map textures (*_sk.dds, *_g.dds, etc.) other than normal maps. You will need to select these texture files using the filter window and then select the desired format in the Color-maps and/or Color-maps-with-alpha values, then process these files.

[hellanios]

Is there a way to make a command line or a new batch file which will separate the ''indoors'' (e.g.caves, dungeons, houses) textures from the ''outdoors'' textures? I would like to use different resolution limits, 1k for ''outdoors'' and 2k for ''indoors''. Of course this should only be done(if it can) while keeping your current work intact.

[phazer11]

Oh it's possible, it'd just be time consuming and I don't think Zed has much time for it not sure about Kelmych. The staff have been busy bees with all the restructuring going on.

You could manually deselect them in DDSOpt though before processing.

[Kelmych]

Is there a way to make a command line or a new batch file which will separate the ''indoors'' (e.g.caves' date=' dungeons, houses) textures from the ''outdoors'' textures? I would like to use different resolution limits, 1k for ''outdoors'' and 2k for ''indoors''. Of course this should only be done(if it can) while keeping your current work intact.[/quote']

Yes, using a batch file it is fairly easy to separate the indoor and outdoor textures from extracted mod files since they are in different directories, and separating normal maps (tangent space and model space) from color maps based on file names is also easy. These take only a few batch commands. I've also been planning a slightly more complicated batch file that would work with mods that have archive files at multiple resolutions to take the exterior files from the lower resolution archive file and other textures from the higher resolution archive file. This only adds a few batch commands relative to the first batch file I mentioned.

 

The hardest part is separating uncompressed textures from compressed ones since there isn't any general way to do this inside a batch file. Since few mods have uncompressed textures (less than 10 mods in STEP 2.2.6 and a similar number, mostly body-related mods, in Skyrim Revisited), I was planning on using knowledge of the archive file names for those mods that have some uncompressed textures (e.g. using the Nexus ID) to separate out the uncompressed textures when they exist using mod-specific batch commands. When the batch file sees these mods it would use additional mod-specific batch commands to move the uncompressed textures (typically normal maps) into a different folder (as was done for the vanilla textures) so a different DDSopt run can be used for these.

 

I've been waiting for the next update of STEP to release these batch files, although it could actually be done any time. The first two batch files I mentioned don't have any mod-specific dependencies so they are the easiest to write. They would look a lot like the existing 2_Pre-optimization.bat batch file, with some additional commands to copy the non-texture files to the output folder.

[hellanios]

Is there a way to make a command line or a new batch file which will separate the ''indoors'' (e.g.caves' date=' dungeons' date=' houses) textures from the ''outdoors'' textures? I would like to use different resolution limits, 1k for ''outdoors'' and 2k for ''indoors''. Of course this should only be done(if it can) while keeping your current work intact.[/quote'']

Yes, using a batch file it is fairly easy to separate the indoor and outdoor textures from extracted mod files since they are in different directories, and separating normal maps (tangent space and model space) from color maps based on file names is also easy. These take only a few batch commands. I've also been planning a slightly more complicated batch file that would work with mods that have archive files at multiple resolutions to take the exterior files from the lower resolution archive file and other textures from the higher resolution archive file. This only adds a few batch commands relative to the first batch file I mentioned.

 

The hardest part is separating uncompressed textures from compressed ones since there isn't any general way to do this inside a batch file. Since few mods have uncompressed textures (less than 10 mods in STEP 2.2.6 and a similar number, mostly body-related mods, in Skyrim Revisited), I was planning on using knowledge of the archive file names for those mods that have some uncompressed textures (e.g. using the Nexus ID) to separate out the uncompressed textures when they exist using mod-specific batch commands. When the batch file sees these mods it would use additional mod-specific batch commands to move the uncompressed textures (typically normal maps) into a different folder (as was done for the vanilla textures) so a different DDSopt run can be used for these.

 

I've been waiting for the next update of STEP to release these batch files, although it could actually be done any time. The first two batch files I mentioned don't have any mod-specific dependencies so they are the easiest to write. They would look a lot like the existing 2_Pre-optimization.bat batch file, with some additional commands to copy the non-texture files to the output folder.

That's great news! From what I understand this is easy for vanilla-bethesda's textures(the ones I had in mind). I'm looking forward to it.

[Kelmych]

The comments I made on batch files to separate exterior textures are for mods rather than vanilla. There isn't really any value in separating exterior textures for vanilla. The max resolution of almost all the HRDLC files in 1Kx1K; there are very few 2kx2k textures in the HRDLC. The 2Kx2K exterior textures that are present are 3 *\landscape\trees files in HRDLC2 and 3 in HRDLC3 which are later overwriten by STEP mods (primarily Skyrim Flora Overhaul), and a handful of normal maps that typically get reduced to 1kx1k or less using the existing vanilla batch files and the associated recommended constraints tab parameters for DDSopt.