GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Kelmych]

In addition to dirt, sand, tundra, mud, and snow, the landscape root includes textures for rocks, grass, moss, leaves, trees,and ice. Would the TSN for all of these be problematic or only for the dirt, sand, tundra, mud, and snow textures? It's possible to be fairly specific in the batch file when doing this.

 

Would the TSN for these textures in mods that replace some of the relevant vanilla textures (e.g., some of the mods in section F) also have this problem?

[z929669]

In addition to dirt, sand, tundra, mud, and snow, the landscape root includes textures for rocks, grass, moss, leaves, trees,and ice. Would the TSN for all of these be problematic or only for the dirt, sand, tundra, mud, and snow textures? It's possible to be fairly specific in the batch file when doing this.

 

Would the TSN for these textures in mods that replace some of the relevant vanilla textures (e.g., some of the mods in section F) also have this problem?

I think that we only need to be concerned with tiled, smooth ground textures. Rocks, grass, plants, roads, etc should be OK. I would be careful with these textures for mods as well, but I have not verified. A great litmus would be to run snow01_n from SFO through DDSopt.

[Nearox]

simple answer: for vanilla textures I'll change the settings in the pictures but it won't actually affect what you get. For non-vanilla textures it isn't as simple and depends on the mod. Are there some representative mods that you are interested in optimizing?

Thank you again Kelmych.

 

I believe I understand quite a bit more now. Actually used the preview option in DDSOpt to had a look myself. Now I finally understand something about what all the different settings mean, though still not much.

 

Just to ask for a final clarification, if I may (really don't want to take up too much of your time, sorry about that). 

 

- Parallax-maps should be set to DXt permanently, regardless of using it on vanilla (even though it is moot) or mod textures?

 

- Tint/tone-maps for vanilla should be set to DXt. For other mods, I will have to look at their original compression and should keep similar compression in tin/tone- maps. E.g. if a mod is in R8G8B8 then it should be compressed with R5G56B5. If a mod has any version of DXt compression then it should be compressed with DXtx settings. Is this correct?

 

 The model space normal maps in the vanilla textures are all uncompressed so we tried to keep them uncompressed with a little file size reduction. This isn't true for all mods; some use DXT compression for MSN textures. The MSN setting for other mods would depend on the format the mod uses for these textures.

- I understand what you are saying here. However, in the DDSOpt guide, the setting for MSN is R5G56B5 but when I preview the MSNs from Skyrim - Textures.bsa (in DDSOpt) they are all in DXT1 format. Shouldn't they too be in the same format? I feel kinda stupid, I'm probably missing something (obvious) here... :(

EDIT: Had a look. From what I can tell the MSNs from Skyrim - Texures.bsa are all in DXT1 format while the MSNs from HighResTexturePack03.bsa are all in R8G8B8 format. There are few MSN files in the other bsa's and it appears there are like less than 100 or so MSN files in total, mostly affecting body/face.

 

I want to optimized most exteriors to 1k textures/.5k normals and most interiors to 2k textures/1k normals. The mods I am interested in optimizing are (using already the Lite versions of these mods):

 

Skyrim HD 2K

Serious HD Retexture Skyrim

Skyrim Flora Overhaul

Ultimate Lush Overhaul

Hybrids HD Plants and Herbs Rextexture

SMIM (not sure if this is a good idea, or even necessary)

Re-defined dungeons

Langelys workshop

And a few other smaller texture mods.

Possibly also face/body/armor but that will come as a last resort (aMidianborn etc.).

 

I'm already using SRO as the basic texture pack. The plan is to build a very basic version of Skyrim Revisited, with no more than 50mods on the list. 

[Kelmych]

Parallax maps: All the ones I've seen are DirectX compressed so set DDSopt for DXTx.

 

Tint/tone maps: no universal answer; it depends. If you're optimizing vanilla textures, which are low res DXTx, use DXTx. If you're optimizing mods with uncompressed tint/tone maps like Better Makeup for SKSE (a Skyrim Revisited mod), use an uncompressed format (R8G8B8 or R5G6B5). If you're optimizing mods with high resolution DXTx compression such as XCE - Warpaint and dirt (a Skyrim Revisited mod) then it depends on the resolution you want to end up with. If you're not changing resolution use DXTx. If you are reducing resolution then you can choose DXTx or an uncompressed format; we haven't seen enough examples to know what is best for this situation .

 

Model Space Normal Maps (used for some body textures) and Tangent Space Normal Maps: Use the concepts described for tint/tone aps, but whenever there is a choice uncompressed maps are much better. Avoid changing an uncompressed normal map to one with DirectX compression when reducing resolution. DirectX compression is much worse with normal maps than with color maps (DDSopt does decrease the loss when reducing normal map resolution with DirectX compression). When reducing resolution by a factor of 2 from a DXTx normal map, uncompressed R5G6B5 gives better fidelity than DXTx at the same resolution, but the difference might be small - it depends :? . This is a subject of much discussion on this and some other threads in these forums.

 

Color Maps: If the color maps were created by a very good graphic artist (e.g., Cabal's textures) then you don't want to use optimization if you can avoid it (typically you use optimization only to reduce resolution if the author didn't provide the resolution you need or when needed to get improved mipmaps). I created a table in the DDSopt guide about this but most users have commented that it's too complicated for them to use :yucky:, so I am working on a simpler table with more specific recommendations.

 

Again, a lot of information but we didn't say it was trivial :facepalm:. The reason that the batch files separate the vanilla textures into 3 folders is so we can provide definitive guidance on the parameters for each set. We are working on a similar approach for the STEP mods :D. The landscape and clutter mods in your list other than SFO can typically get benefit from DDSopt optimization, if only for better mipmaps.

[phazer11]

I've noticed no difference in the file sizes of the vanilla textures using R5G6B5 compression versus DXTx so I'd go for it for the vanilla textures for sure. I'd imagine they use about the same amount of RAM/VRAM at least for the vanilla textures. I don't know how they load into the RAM in comparison to each other yet however so who knows.

[Kelmych]

Textures in VRAM are uncompressed. My understanding is that the ones in RAM can still be compressed.

[EssArrBee]

Well R5G6B5 is the format the texture is saved in while the DXT is a compression that is used once it's saved. And VRAM has full DXT textures because DX9 hardware has to have native support of any S3TC algorithms. This allows the whole process to be done in very few cycles, the entire thread block can be done in an average of 100 cycles per thread or less.

[Kelmych]

In addition to dirt, sand, tundra, mud, and snow, the landscape root includes textures for rocks, grass, moss, leaves, trees,and ice. Would the TSN for all of these be problematic or only for the dirt, sand, tundra, mud, and snow textures? It's possible to be fairly specific in the batch file when doing this.

 

Would the TSN for these textures in mods that replace some of the relevant vanilla textures (e.g., some of the mods in section F) also have this problem?

I think that we only need to be concerned with tiled, smooth ground textures. Rocks, grass, plants, roads, etc should be OK. I would be careful with these textures for mods as well, but I have not verified. A great litmus would be to run snow01_n from SFO through DDSopt.

I optimized snow01_n.dds from three mods with 2K versions: Serious HD, SRO, and HD2K. Using the preview mode of DDSopt to compare textures, the results after optimization were similar to those with the vanilla Skyrim textures version. At full resolution there are almost no differences between the original texture(s) and the optimized one(s), but at lower mip levels the differences increase.

 

Are there any mod testing savegames that use this texture in the view at the point where the game was saved so I can see if it is still purple with the optimized textures from the other mods?

[z929669]

 

In addition to dirt, sand, tundra, mud, and snow, the landscape root includes textures for rocks, grass, moss, leaves, trees,and ice. Would the TSN for all of these be problematic or only for the dirt, sand, tundra, mud, and snow textures? It's possible to be fairly specific in the batch file when doing this.

 

Would the TSN for these textures in mods that replace some of the relevant vanilla textures (e.g., some of the mods in section F) also have this problem?

I think that we only need to be concerned with tiled, smooth ground textures. Rocks, grass, plants, roads, etc should be OK. I would be careful with these textures for mods as well, but I have not verified. A great litmus would be to run snow01_n from SFO through DDSopt.

I optimized snow01_n.dds from three mods with 2K versions: Serious HD, SRO, and HD2K. Using the preview mode of DDSopt to compare textures, the results after optimization were similar to those with the vanilla Skyrim textures version. At full resolution there are almost no differences between the original texture(s) and the optimized one(s), but at lower mip levels the differences increase.

 

Are there any mod testing savegames that use this texture in the view at the point where the game was saved so I can see if it is still purple with the optimized textures from the other mods?

The save I posted that you used on my help thread has bott he tiled snow textures as well as the purplish planar LOD textures.

 

These are two different issues though. snow01_n optimization exacerbates the apparent tiling/striping effect in the non-distant snow textures in that save. Way out on the horizon, there are also two LOD missing color maps, which is a different issue and the texture is unknown.

---

EDIT: here is that savegame.

[Kelmych]

Here are two images with the save you posted. I moved the camera a little from the original view in the save to emphasize the nearby snow. I can also now see that the abrupt texture change I mentioned before was a different mountain occluding the other mountain. The first image is with the original vanilla unoptimized snow01_n.dds, the 2nd with an optimized SRO version of the normal map (using the original SRO 2K texture). DDSopt.ini was reverted to not include bypassing this texture. This includes some STEP mods through section G, and includes DTTL and the modified SFO 1.81, but no Terrain Bump.

 

                 

Original unoptimized vanilla normal map         Optimized SRO normal map

[z929669]

So you ran SRO snow01_n through DDSopt, re-mipped using all of the Behave options for mipping, and the reult is fine? It might be good to include your own optimized version of vanilla and another of SRO side by side in order to test that you get the tiled snow pattern at least on vanilla. Right now, I am not sure if we can say that SRO optimized was subjected to the same method as vanilla optimized (demonstrating that optimization breaks vanilla at least with respect to this texture).

[Nearox]

Parallax maps: All the ones I've seen are DirectX compressed so set DDSopt for DXTx.

 

Tint/tone maps: no universal answer; it depends. If you're optimizing vanilla textures, which are low res DXTx, use DXTx. If you're optimizing mods with uncompressed tint/tone maps like Better Makeup for SKSE (a Skyrim Revisited mod), use an uncompressed format (R8G8B8 or R5G6B5). If you're optimizing mods with high resolution DXTx compression such as XCE - Warpaint and dirt (a Skyrim Revisited mod) then it depends on the resolution you want to end up with. If you're not changing resolution use DXTx. If you are reducing resolution then you can choose DXTx or an uncompressed format; we haven't seen enough examples to know what is best for this situation .

 

Model Space Normal Maps (used for some body textures) and Tangent Space Normal Maps: Use the concepts described for tint/tone aps, but whenever there is a choice uncompressed maps are much better. Avoid changing an uncompressed normal map to one with DirectX compression when reducing resolution. DirectX compression is much worse with normal maps than with color maps (DDSopt does decrease the loss when reducing normal map resolution with DirectX compression). When reducing resolution by a factor of 2 from a DXTx normal map, uncompressed R5G6B5 gives better fidelity than DXTx at the same resolution, but the difference might be small - it depends :? . This is a subject of much discussion on this and some other threads in these forums.

 

Color Maps: If the color maps were created by a very good graphic artist (e.g., Cabal's textures) then you don't want to use optimization if you can avoid it (typically you use optimization only to reduce resolution if the author didn't provide the resolution you need or when needed to get improved mipmaps). I created a table in the DDSopt guide about this but most users have commented that it's too complicated for them to use :yucky:, so I am working on a simpler table with more specific recommendations.

 

Again, a lot of information but we didn't say it was trivial :facepalm:. The reason that the batch files separate the vanilla textures into 3 folders is so we can provide definitive guidance on the parameters for each set. We are working on a similar approach for the STEP mods :D. The landscape and clutter mods in your list other than SFO can typically get benefit from DDSopt optimization, if only for better mipmaps.

Wow, thank you for trying to epxlain it all to me. It is now pretty clear how to continue with DDSOpt! I actually followed Neo's instructions on DDSOpting the vanilla textures, which does not use the batch file. What is the different result to be expected from this approach?

 

And yes, the table is too complicated. I can give you some tips from the point of view of a noob/regular user (aka me) maybe it can help you to better structure it:

 

- As you mentioned yourself, the lack of recommendations means that the table has little guidance. After all, it is called a DDSOpt 'guide' :P

 

- The column labels are rather incomprehensible for the average user that is looking for a few optimizations to reduce VRAM. What does it mean when it says no/yes in mip-maps? That the mod package is missing some mip-maps? Stuff like that could be explained at the beginning of the guide in a legend or in a seperate textbox. 

 

- Mip-maps, errors, compression and size could be combined in one column, or at least reduce the column width/labels to make space for new columns... read further below :)

 

I think, but maybe I'm thinking too big, that it would be extremely helpful for the entire modding community if this became a reference table with information based on testing and/or reported user experiences: 

 

- A column labeled 'potential VRAM savings':The VRAM savings could be contributed by testing via the community. Perhaps open a new thread calling for people to test the difference between a DDSOpt and non-DDSOpt version of the mod. You could limit included results to 1680x1050 or 1080p resolutions. Attach a note saying that VRAM savings can vary greatly between systems. 

 

 - Another column labeled 'potential FPS savings'. Because not all users are interested in saving VRAM. Quite some people I know have issues with FPS instead of VRAM. For instance, DDSOpting Skyrim Flora Overhaul  (and I did not decrease resolution) reduced my VRAM use by only 20mb but it strangely enough also increased FPS in areas like the Rift by 4-6 with no noticeable quality loss. 

 

- Another column labeled 'potential and/or known graphical complications from DDSOpt' (or a shorter title) could indicate problems that may/have been found to arise from DDSOpting the specific mod. E.g. DDSopting normals to half-size can sometimes increase shimmering. An extensive lsit of notes at the end of the table can explain texture 

 

- Develop a benchmark standard. E.G.: 1) mod must be run in a vanilla environment with no other mods enabled 2) 'high' default settings must be used 3) ini must be default ini with no specific tweaks 4) the benchmark goes like: standing stones - riverwood - whiterun - fast travel to ivarstead - run to riften. Just some examples :)

 

 - Develop a DDSopt standard (e.g. exteriors to 1k textures/.5k normal and interiors to 2k textures/.5k normals) or develop multiple ones (e.g. low, medium & high res)

 

Of course, these columns will not be filled with data in a day. But this way, a database will gradually be built as users/step people contribute to it. Information can be extracted from the existing forum threads too, of course. The table should have a warning saying that the values are indications and/or estimates and therefor may vary widely per system. 

 

I already wrote down VRAM savings for every mod that I DDSOpted (about 10) so that could be a start, although I may have used less than optimal settings reading from your post of explanations. I realize that such a table can never be definite, or produce the same result for any user, but to have a roadsign is better than to be lost in the wilds :)

 

I'm just spitting out some ideas, that's all :P I'm willing to assist should you decide to embark on such a project.

[Kelmych]

So you ran SRO snow01_n through DDSopt' date=' re-mipped using all of the Behave options for mipping, and the reult is fine? It might be good to include your own optimized version of vanilla and another of SRO side by side in order to test that you get the tiled snow pattern at least on vanilla. Right now, I am not sure if we can say that SRO optimized was subjected to the same method as vanilla optimized (demonstrating that optimization breaks vanilla at least with respect to this texture).[/quote']

I started to make the additional screenshots, but looking at the scene I see that windblown snow is affecting the screenshots. Is there a way to temporarily turn it off?

[phazer11]

Best way to do that screenshot is to turn off your menus using the ~tm command and try your best to get the snow in the same position (hope you have good reflexes old man :)). Other than that the only things I can think of to change it is using that Thunder or Real Rain mod I can't remember which it is that let's you control weather. The only problem I see with it is I don't think it counts as a weather though.

 

Edit: What does STATUS say is the weather condition.

[z929669]

Parallax maps: All the ones I've seen are DirectX compressed so set DDSopt for DXTx.

 

Tint/tone maps: no universal answer; it depends. If you're optimizing vanilla textures, which are low res DXTx, use DXTx. If you're optimizing mods with uncompressed tint/tone maps like Better Makeup for SKSE (a Skyrim Revisited mod), use an uncompressed format (R8G8B8 or R5G6B5). If you're optimizing mods with high resolution DXTx compression such as XCE - Warpaint and dirt (a Skyrim Revisited mod) then it depends on the resolution you want to end up with. If you're not changing resolution use DXTx. If you are reducing resolution then you can choose DXTx or an uncompressed format; we haven't seen enough examples to know what is best for this situation .

 

Model Space Normal Maps (used for some body textures) and Tangent Space Normal Maps: Use the concepts described for tint/tone aps, but whenever there is a choice uncompressed maps are much better. Avoid changing an uncompressed normal map to one with DirectX compression when reducing resolution. DirectX compression is much worse with normal maps than with color maps (DDSopt does decrease the loss when reducing normal map resolution with DirectX compression). When reducing resolution by a factor of 2 from a DXTx normal map, uncompressed R5G6B5 gives better fidelity than DXTx at the same resolution, but the difference might be small - it depends :? . This is a subject of much discussion on this and some other threads in these forums.

 

Color Maps: If the color maps were created by a very good graphic artist (e.g., Cabal's textures) then you don't want to use optimization if you can avoid it (typically you use optimization only to reduce resolution if the author didn't provide the resolution you need or when needed to get improved mipmaps). I created a table in the DDSopt guide about this but most users have commented that it's too complicated for them to use :yucky:, so I am working on a simpler table with more specific recommendations.

 

Again, a lot of information but we didn't say it was trivial :facepalm:. The reason that the batch files separate the vanilla textures into 3 folders is so we can provide definitive guidance on the parameters for each set. We are working on a similar approach for the STEP mods :D. The landscape and clutter mods in your list other than SFO can typically get benefit from DDSopt optimization, if only for better mipmaps.

Wow, thank you for trying to epxlain it all to me. It is now pretty clear how to continue with DDSOpt! I actually followed Neo's instructions on DDSOpting the vanilla textures, which does not use the batch file. What is the different result to be expected from this approach?

 

And yes, the table is too complicated. I can give you some tips from the point of view of a noob/regular user (aka me) maybe it can help you to better structure it:

 

- As you mentioned yourself, the lack of recommendations means that the table has little guidance. After all, it is called a DDSOpt 'guide' :P

 

- The column labels are rather incomprehensible for the average user that is looking for a few optimizations to reduce VRAM. What does it mean when it says no/yes in mip-maps? That the mod package is missing some mip-maps? Stuff like that could be explained at the beginning of the guide in a legend or in a seperate textbox. 

 

- Mip-maps, errors, compression and size could be combined in one column, or at least reduce the column width/labels to make space for new columns... read further below :)

 

I think, but maybe I'm thinking too big, that it would be extremely helpful for the entire modding community if this became a reference table with information based on testing and/or reported user experiences: 

 

- A column labeled 'potential VRAM savings':The VRAM savings could be contributed by testing via the community. Perhaps open a new thread calling for people to test the difference between a DDSOpt and non-DDSOpt version of the mod. You could limit included results to 1680x1050 or 1080p resolutions. Attach a note saying that VRAM savings can vary greatly between systems. 

 

 - Another column labeled 'potential FPS savings'. Because not all users are interested in saving VRAM. Quite some people I know have issues with FPS instead of VRAM. For instance, DDSOpting Skyrim Flora Overhaul  (and I did not decrease resolution) reduced my VRAM use by only 20mb but it strangely enough also increased FPS in areas like the Rift by 4-6 with no noticeable quality loss. 

 

- Another column labeled 'potential and/or known graphical complications from DDSOpt' (or a shorter title) could indicate problems that may/have been found to arise from DDSOpting the specific mod. E.g. DDSopting normals to half-size can sometimes increase shimmering. An extensive lsit of notes at the end of the table can explain texture 

 

- Develop a benchmark standard. E.G.: 1) mod must be run in a vanilla environment with no other mods enabled 2) 'high' default settings must be used 3) ini must be default ini with no specific tweaks 4) the benchmark goes like: standing stones - riverwood - whiterun - fast travel to ivarstead - run to riften. Just some examples :)

 

 - Develop a DDSopt standard (e.g. exteriors to 1k textures/.5k normal and interiors to 2k textures/.5k normals) or develop multiple ones (e.g. low, medium & high res)

 

Of course, these columns will not be filled with data in a day. But this way, a database will gradually be built as users/step people contribute to it. Information can be extracted from the existing forum threads too, of course. The table should have a warning saying that the values are indications and/or estimates and therefor may vary widely per system. 

 

I already wrote down VRAM savings for every mod that I DDSOpted (about 10) so that could be a start, although I may have used less than optimal settings reading from your post of explanations. I realize that such a table can never be definite, or produce the same result for any user, but to have a roadsign is better than to be lost in the wilds :)

 

I'm just spitting out some ideas, that's all :P I'm willing to assist should you decide to embark on such a project.

It sounds like you just need to get out there and start editing ;) We'll clean up anything that gets messy if necessary. The  important thing is that more users get out on the wiki and contribute.