GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Kelmych]

Ethatron, ive been having a lot of crashes with DDSopt lately. For instance after constraining the _n.dds files from Serious HD to 50% resolution, when i click on Primary (Destination) in the preview tab it crashes. This happens with all the _n.dds files from Serious HD.

 

I dont know if its just me or if someone else can replicate the bug?

Is this with pre-release update 3 or with pre-release update 4 that was posted today by Ethatron?

[Garpov]

It was with update 3, but ive just ran the _n.dds files through update 4 and the same thing happens.

[Ethatron]

Thanks for updating the .ini and the executable . Did you also address the processing for the Riverwood Trader sign we were discussing a while ago?

Yes' date=' but's just 2 of the 3 in the folder. First is okay as-is.

It was with update 3' date=' but ive just ran the _n.dds files through update 4 and the same thing happens.[/quote']

Can be, mixed resolutions (the preselected input resolution doesn't exist for the output when you downed to 50%), you probably can go one mip down with the drop-down and then see the output texture while I look at it.

[z929669]

I was reviewing the discussion and I changed the following:

* "effects/" by default is excluded from semantical processing, I added a few files back into the .ini though which I know contain transparency in the alpha

When you say "transparency in the alpha", do you mean 0% opacity? Like the alpha being used as a mask rather than for variations in opacity? Is the term "100% transparency" redundant, in your opinion?

* I added all the files in "sky/" to the .ini which don't contain transparency (aurora and constellation)

FYI, we are maintaining a working INI on the wiki that essentially will include our findings. We will append the updates to the end of each applicable section, and these will reflect our hacks to get a working result. From this, you will likely know the proper solution.

* I let DDSopt choose a more pleasing matte-colour when transparency is 100%, though this may just obstruct the "problem" / the necessity to clearly identify special or buggy textures ... if you want to do wrong-transparency-assumption hunting you can continue with "update 3"

 

Just check the textures with the updates and look if I didn't find all exceptions. As said, effects+sky shouldn't be an issue anymore.

I will check with the new update, but the new INI does not account for one case I found what I believe is the alpha being used as a mask for a glow map. This results in the black horn candelabras I think, since it appears as though the alpha masks the primary channel with regard to all but the glow. Can you confirm that I am interpreting properly?

clutter\candles\horncandles01.dds

Also, there are a number of issues with landscape textures, where pieces of the primary channel itself are masked. I have no idea why this happens, as I cannot see anything strange happening in the alpha on these:

; Serious HD - Some spaces between rocks & particles are black
landscape\rocks01.dds
landscape\rocksedgetrim01.dds
landscape\tundrarocks01.dds
landscape\fieldgrass01.dds
landscape\fieldgrass02.dds
landscape\icefloes.dds
landscape\volcanictundragravel01.dds
landscape\dirt02.dds

; Skyrim HD - Some spaces between rocks & particles are black
landscape\riverbededge.dds
landscape\riverbededge1.dds

Does it make sense for us to control for these still?

[z929669]

Thanks for updating the .ini and the executable . Did you also address the processing for the Riverwood Trader sign we were discussing a while ago?

Yes' date=' but's just 2 of the 3 in the folder. First is okay as-is.

It was with update 3' date=' but ive just ran the _n.dds files through update 4 and the same thing happens.[/quote']

Can be, mixed resolutions (the preselected input resolution doesn't exist for the output when you downed to 50%), you probably can go one mip down with the drop-down and then see the output texture while I look at it.

OK, I have never once even noticed that the very bottom of the dropdown gives % options ... :P

 

I am guessing the fix is to never use this method (or don't choose any 2:1 constraints)

[Kelmych]

OK, I have never once even noticed that the very bottom of the dropdown gives % options ... :P

 

I am guessing the fix is to never use this method (or don't choose any 2:1 constraints)

I saw them but I've been using fixed maximum sizes for each dimension (e.g., 1024x1024) when I wanted to reduce normal map resolution. If there happened to be a small normal map in a set of textures I wan't sure I wanted to reduce the size of the existing one as it wouldn't save very much processing and might cause problems with the graphic quality.

[PCG4m3r]

OK, I have never once even noticed that the very bottom of the dropdown gives % options ... :P

 

I am guessing the fix is to never use this method (or don't choose any 2:1 constraints)

I saw them but I've been using fixed maximum sizes for each dimension (e.g., 1024x1024) when I wanted to reduce normal map resolution. If there happened to be a small normal map in a set of textures I wan't sure I wanted to reduce the size of the existing one as it wouldn't save very much processing and might cause problems with the graphic quality.

I believe I can confirm that reducing the size of already small normals does impact the texture quality more noticeably in game than reducing larger normals. I just posted some images at the following URL (see soulgem images "NODDSopt" vs. "HalfSizeNM").

 

https://imgur.com/a/OKapQ#4

 

The "HalfSizeNM" were reduced 50% normals (i.e. 512 to 256 (common / grand) ...and... 128 to 64 (soulgems01_n.dds)).

 

Also in that same album are some comparisons of mountain textures with reduced textures and/or normals, which I wrote about the results here (cyan text at bottom):

 

https://forum.step-project.com/showthread.php?tid=1392&pid=22130#pid22130

 

 

I have been using Neovalen's "Skyrim Revisited" guide (all his mods installed). After I had them all setup, I ran DDSopt 8 upd 3 on every mod that had textures; first pass on all *.dds files except *_n.dds; second pass on all *_n.dds to constrain by 50% with NO compression/optimization. I did this regardless of the mod or original texture size. This was so that I can go through the game to see what was impacted texture-wise for these other, non-vanilla/HRDLC mods, but also to see if the reduced normals improved anything. I posted my initial "gameplay" impression here (performance noticeably improved):

 

https://forum.step-project.com/showthread.php?tid=1392&pid=22105#pid22105

 

I will post other findings here and in Neo's thread in case this is helpful to anyone.

[PCG4m3r]

@Ethatron:

 

First wanted to say thanks for DDSopt. This is a excellent/powerful tool and your efforts are truly appreciated. :)

 

I wanted to ask about optimization of model-space normal maps, based on a problem I observed. I had unintentionally ran optimization on all *_msn.dds files for every mod I have installed, which includes several body/face mods. I meant to reduce the msn files by 50% with no compression/optimization, but it was an oversight. I noticed that DDSopt handled some msn files ok, depending on the mod, but others looked noticeably bad.

 

I posted my initial findings (some specific mods mentioned) here:

https://forum.step-project.com/showthread.php?tid=1392&pid=22105#pid22105

 

And here are two example images "NO DDSopt" and "WITH DDSopt":

https://imgur.com/a/PMZE3#1

 

Those images were from the UNP Blessed Body mod ver 2.4. I had used the recommended settings per the STEP DDSopt Guide.

 

Questions:

• Do we need to configure DDSopt settings differently for these types of files?
• Or should we exclude them from any optimization...period?
• Or is DDSopt simply not coded to handle these properly yet, but may in a future update, thus we still should exclude for now?

[z929669]

@Ethatron:

 

First wanted to say thanks for DDSopt. This is a excellent/powerful tool and your efforts are truly appreciated. :)

 

I wanted to ask about optimization of model-space normal maps, based on a problem I observed. I had unintentionally ran optimization on all *_msn.dds files for every mod I have installed, which includes several body/face mods. I meant to reduce the msn files by 50% with no compression/optimization, but it was an oversight. I noticed that DDSopt handled some msn files ok, depending on the mod, but others looked noticeably bad.

 

I posted my initial findings (some specific mods mentioned) here:

https://forum.step-project.com/showthread.php?tid=1392&pid=22105#pid22105

 

And here are two example images "NO DDSopt" and "WITH DDSopt":

https://imgur.com/a/PMZE3#1

 

Those images were from the UNP Blessed Body mod ver 2.4. I had used the recommended settings per the STEP DDSopt Guide.

 

Questions:

• Do we need to configure DDSopt settings differently for these types of files?
• Or should we exclude them from any optimization...period?
• Or is DDSopt simply not coded to handle these properly yet, but may in a future update, thus we still should exclude for now?

What you are seeing with these normals is that they have been converted but not re-normalized, thus, this is 'de'-normalized and looks strange. You will need to use an application that can normalize the image to see the result that will appear in game (Gimp or Photoshop have NM plugin viewers).

[TechAngel85]

Found another one!

 

wrplasterdecals.dds in Skyrim HD - Towns

[Kelmych]

There seems to be some consistency in some of these problems. The rocks that didn't get transformed properly also had an area that was much smoother and a different color than the rest of the rock; the primary channel for those textures turned black in the smooth area when processed in DDSopt.

 

I haven't updated the DDSop.ini since Ethatron released DDSopt pre-release 4 today. I was waiting if there was an answer to z about the best way to hande the problematic rock textures (and now the plaster decals texture you noticed). If I don't hear I'll add them to DDSopt.ini .

[TechAngel85]

And another...

 

From Skyrim - Textures.bsa (used as loose files)

It's no wonder those things look so noisy in-game.

 

I'm not using the high res DLC so I don't know if those are replaced or not.

---

EDIT:

And another...

 

From Skyrim - Textures.bsa (used as loose files)

[PCG4m3r]

@Ethatron:

 

First wanted to say thanks for DDSopt. This is a excellent/powerful tool and your efforts are truly appreciated. :)

 

I wanted to ask about optimization of model-space normal maps, based on a problem I observed. I had unintentionally ran optimization on all *_msn.dds files for every mod I have installed, which includes several body/face mods. I meant to reduce the msn files by 50% with no compression/optimization, but it was an oversight. I noticed that DDSopt handled some msn files ok, depending on the mod, but others looked noticeably bad.

 

I posted my initial findings (some specific mods mentioned) here:

https://forum.step-project.com/showthread.php?tid=1392&pid=22105#pid22105

 

And here are two example images "NO DDSopt" and "WITH DDSopt":

https://imgur.com/a/PMZE3#1

 

Those images were from the UNP Blessed Body mod ver 2.4. I had used the recommended settings per the STEP DDSopt Guide.

 

Questions:

• Do we need to configure DDSopt settings differently for these types of files?
• Or should we exclude them from any optimization...period?
• Or is DDSopt simply not coded to handle these properly yet, but may in a future update, thus we still should exclude for now?

What you are seeing with these normals is that they have been converted but not re-normalized, thus, this is 'de'-normalized and looks strange. You will need to use an application that can normalize the image to see the result that will appear in game (Gimp or Photoshop have NM plugin viewers).

I actually notice a problem in-game though. I first noticed the problem on the body, what I can only describe as some kind of color banding with apparent, hard transitions on the skin. After replacing with the unoptimized file, the body texture looked much smoother and normal.

 

I looked at the good/bad msn files using the NVidia NormalMapFilter plugin in Photoshop. When I use the 3D Preview in the plugin the good file looks smooth, while the bad file is not smooth at all. I also use SageThumbs to initially view thumbnails in Windows Explorer and Windows Texture Viewer as the default viewer for all .dds files; with those tools, I see the same good/bad results. I haven't checked with Compressionator yet, because I don't have it installed.

[Ethatron]

I was reviewing the discussion and I changed the following:

* "effects/" by default is excluded from semantical processing' date=' I added a few files back into the .ini though which I know contain transparency in the alpha[/quote']

When you say "transparency in the alpha", do you mean 0% opacity? Like the alpha being used as a mask rather than for variations in opacity? Is the term "100% transparency" redundant, in your opinion?

Well, non-native english-speaker here. I have no clue which exactly is the semantic difference in unsing "opacity" and "transparency", I guess "opacity" gives more emphasis on the fact that something is opaque by default, though for example skin could be not really be called transparent, but calling it half-opaque also sounds strange ... The one extreme is "it's transparent" the other is "it's opaque", if you use one or the other word to go into the other extreme's direction isn't sufficiently standardized I suppose.

In 3D we basically always say "transparency", in 2D (images, PNG, GIF etc) we also say "transparency", in web-design and with browsers they started using "opacity" (in CSS fe.), Photoshop and alike use "alpha" as a synonym for "transparency", it's a big mess ...

* I added all the files in "sky/" to the .ini which don't contain transparency (aurora and constellation)

FYI' date=' we are maintaining a working INI on the wiki that essentially will include our findings. We will append the updates to the end of each applicable section, and these will reflect our hacks to get a working result. From this, you will likely know the proper solution.[/quote']

Sure.

* I let DDSopt choose a more pleasing matte-colour when transparency is 100%' date=' though this may just obstruct the "problem" / the necessity to clearly identify special or buggy textures ... if you want to do wrong-transparency-assumption hunting you can continue with "update 3"

 

Just check the textures with the updates and look if I didn't find all exceptions. As said, effects+sky shouldn't be an issue anymore.[/quote']

I will check with the new update, but the new INI does not account for one case I found what I believe is the alpha being used as a mask for a glow map. This results in the black horn candelabras I think, since it appears as though the alpha masks the primary channel with regard to all but the glow. Can you confirm that I am interpreting properly?

clutter\candles\horncandles01.dds

Ah, yes. I forgot to reread the previous discussion-page. :)

Also' date=' there are a number of issues with landscape textures, where pieces of the primary channel itself are masked. I have no idea why this happens, as I cannot see anything strange happening in the alpha on these:

; Serious HD - Some spaces between rocks & particles are black
landscape\rocks01.dds
landscape\rocksedgetrim01.dds
landscape\tundrarocks01.dds
landscape\fieldgrass01.dds
landscape\fieldgrass02.dds
landscape\icefloes.dds
landscape\volcanictundragravel01.dds
landscape\dirt02.dds

; Skyrim HD - Some spaces between rocks & particles are black
landscape\riverbededge.dds
landscape\riverbededge1.dds

Does it make sense for us to control for these still?[/quote']

Well, I would be surprised if there would be two distinct types of landscape-textures. It's possible that the alpha-channel is just filtered in such a way that normally unused parts of the texture are accessed nevertheless they are invisible according to the transparency-map. The matte-colour that DDSopt currently assigns could already have fixed that. Previously the matte-colour was black. The basic idea is, that when the compression takes place and the DXT only allows a palette of 4 colours per 4x4 block, the compressor gives less weight to colours you're not going to see that clear anyway; if you have a fully transparent pixel you wouldn't see the colour at all, if you open such a DDS in Irfanview it looks totally fine, and if Skyrim would follow suit it would be totally fine in-game as well (if alpha==transparency, otherwise not). But Skyrim being a black-box we have to predict the behaviour of it by trial-and-error. It would be possible to check the NIFs for the meaning of the textures, but then you couldn't process any textures without all (!) used NIFs available.

OK' date=' I have never once even noticed that the very bottom of the dropdown gives % options ... :P

 

I am guessing the fix is to never use this method (or don't choose any 2:1 constraints)[/quote']

Maybe someone wants all textures equally blurry. You may also have no intuition about what a regular size is or what oversized, so when you want it smaller, just go for a %.

I noticed that DDSopt handled some msn files ok' date=' depending on the mod, but others looked noticeably bad.

 

Questions:

• Do we need to configure DDSopt settings differently for these types of files?
• Or should we exclude them from any optimization...period?
• Or is DDSopt simply not coded to handle these properly yet, but may in a future update, thus we still should exclude for now?

[/quote']

The normals are okay like that. It's possible some of them got DXT-compressed, or R5G6B5 - smooth regions get banding with this setting, non-smooth MSNs won't have problems with this - your's look pretty much like 565.

What you are seeing with these normals is that they have been converted but not re-normalized' date=' thus' date=' this is 'de'-normalized and looks strange. You will need to use an application that can normalize the image to see the result that will appear in game (Gimp or Photoshop have NM plugin viewers).[/quote'']

I actually notice a problem in-game though. I first noticed the problem on the body, what I can only describe as some kind of color banding with apparent, hard transitions on the skin. After replacing with the unoptimized file, the body texture looked much smoother and normal.

 

I looked at the good/bad msn files using the NVidia NormalMapFilter plugin in Photoshop. When I use the 3D Preview in the plugin the good file looks smooth, while the bad file is not smooth at all. I also use SageThumbs to initially view thumbnails in Windows Explorer and Windows Texture Viewer as the default viewer for all .dds files; with those tools, I see the same good/bad results. I haven't checked with Compressionator yet, because I don't have it installed.

You have to renormalize with the Normal-filter in the filter-menu to really compare them unapplied. As I said, 565 can give you banding.

[Ethatron]

It looks like Serious HD rocksedgetrim01's alpha is a height-map, while Vanilla Skyrim rocksedgetrim01's alpha is a transparency-map ... Is this a modder misunderstanding? Or are there special NIFs floating around?

Vanilla Skyrim rocks01 seems to be a height-map. Vanilla volcanictundraminerals looks super weird, looks more than a specular map than anything else, or an erronous texture.

I guess all the files directly in landscape are actual height-map carrying, except the trim-file, all the sub-folders have transparency-maps.