GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Neovalen]

 

 

Here's what I did:

 

1) Extracted all vanilla + HRDLC textures using BSAopt to separate folders.

2) Followed Z's instructions for fixing HRDLC.

3) Merged the fixed HRDLC textures with vanilla textures.

4) Ran DDSopt v8 on ALL textures using the Recommended settings from the OP.

5) Overwritten the incorrectly optimised textures from the DDSopt guide and Neo's guide with the unoptimised textures from the vanilla folder.

6) Archived using LZMA2 Ultra compression that gets the textures down to something like 3.1 GB (from 5.65 GB).

7) Disabled HRDLC from loading in Data Files.

 

This pretty much gives you a base that won't change any time soon, unless Bethesda updates their textures for some reason.

The only thing I would qualify is that, for Wrye Bash users, when packing up into the archive, use non-solid compression. This may slightly decrease the compression ratio, but the benefit is huge when you are (un)installing upstream packages, re-unpacking from the huge vanilla archive ... extraction goes much faster.

If this isn't in the WB guide, it needs to be. I'll re-archive mods even with no changes for this very reason.

Steps 1-4 are in the revised (as of today) DDSopt guide. We are still discussing how to handle the incorrect textures mentioned in step 5; z and Ethatron prefer to handle this via the ddsopt.ini file. Does STEP want to recommend specific compression to use for Skyrim files as in step 6? If so some users might appreciate it especially if we provide some rationale and perhaps a few quantitative comparisons with representative Skyrim data. Should step 7 be in WB guide or the installation guide?

 

By the way, now that a number of corrections have been made (and are ongoing) to the DDSOpt guide, there are a couple of inconsistencies in the Skyrim Installation guide in the part that mentions the HR DLC.

I would consider compressing it to be an optional step. If your using something like MO you wouldn't compress it. I would think a custom ini file is the route to go between Ethatron's updates.

True, but WB needs compressed files so that it does not have to scan an entire directory each load. It scans the archive once and then uses the CRC to verify subsequently. I assume that this may be why MO ?duplicates? packages?

 

Compression is not really optional for WB

If so then step 6 would be in WB guide... and step 7 in STEP itself as generally applicable (imo). Step 5 would be handled by the custom ini.

[Garpov]

Hi guys, ive been watching this thread some time, finally have something to contribute.

 

When using DDSopt ( pre-release 3-5755-0-8-0-upd3) on the vanilla textures from textures.bsa, ive noticed several corruptions after optimization.

 

effect\cloudtilefrostsmall.dds

effect\fxcloudroundtile.dds

effect\fxcloudroundtilelite.dds

effect\fxcloudroundtilestrip.dds

 

Also every file in the sky\ directory containing the word "cloud". 13 files in total + skyrimcloudsfill.dds which has already been mentioned as turning the horizon purple.

 

When processing the same files with DDSopt pre 5b there is no corruption (except skyrimcloudsfill.dds).

[TechAngel85]

The entire textures/effects/ and textures/effect/gradient/ folders need to be looked at. I've actually just when ahead and replaced the /gradients/ folder with the unoptimized textures because so many of them are causing issues. :P

[Kelmych]

Here is a list of files in /textures/effect/gradient that I'm fairly convinced are not properly processed by DDSopt

[z929669]

I just posted my DDSopt INI on the wiki. This lists all of the textures underneath effects/ and sky/ (many, if not most have custom alphas that get messed up, so just listed them all). I confirmed that this fixes, but Ethatron may want to explicitly handle these within the application at some point, so it would be great if we could get a complete list of all textures that have weird alpha channels.

[Neovalen]

I think my blood problem was related to effects, I took out effects and I'm having much better results in game.

[z929669]

 

Here's what I did:

 

1) Extracted all vanilla + HRDLC textures using BSAopt to separate folders.

2) Followed Z's instructions for fixing HRDLC.

3) Merged the fixed HRDLC textures with vanilla textures.

4) Ran DDSopt v8 on ALL textures using the Recommended settings from the OP.

5) Overwritten the incorrectly optimised textures from the DDSopt guide and Neo's guide with the unoptimised textures from the vanilla folder.

6) Archived using LZMA2 Ultra compression that gets the textures down to something like 3.1 GB (from 5.65 GB).

7) Disabled HRDLC from loading in Data Files.

 

This pretty much gives you a base that won't change any time soon, unless Bethesda updates their textures for some reason.

The only thing I would qualify is that, for Wrye Bash users, when packing up into the archive, use non-solid compression. This may slightly decrease the compression ratio, but the benefit is huge when you are (un)installing upstream packages, re-unpacking from the huge vanilla archive ... extraction goes much faster.

If this isn't in the WB guide, it needs to be. I'll re-archive mods even with no changes for this very reason.

Updated the WB Guide to include this. Also updated a lot of other stuff on that one.

[Besidilo]

I just posted my DDSopt INI on the wiki. This lists all of the textures underneath effects/ and sky/ (many' date=' if not most have custom alphas that get messed up, so just listed them all). I confirmed that this fixes, but Ethatron may want to explicitly handle these within the application at some point, so it would be great if we could get a complete list of all textures that have weird alpha channels.[/quote']

This is great, thanks a lot for editing the INI, I'll report on the results ASAP.

[PCG4m3r]

I just posted my DDSopt INI on the wiki. This lists all of the textures underneath effects/ and sky/ (many' date=' if not most have custom alphas that get messed up, so just listed them all). I confirmed that this fixes, but Ethatron may want to explicitly handle these within the application at some point, so it would be great if we could get a complete list of all textures that have weird alpha channels.[/quote']

Thanks z for the ini! Much appreciated! :)

 

I noticed there were a couple lines in there that only listed directory paths, but no DDS files:

 

effects\cockroach

effects\trailerfx

 

I wasn't sure if that was intentional or just an oversight? The actual DDS files that exist in those directories are included on their own lines in the ini, so it probably wouldn't hurt anything with those extras in there. I just wanted to point that out.

 

Thanks again for making that available!

[z929669]

Yep, Looks like I left a couple of directory paths in there. I pulled the paths out of a log file ... I'll get rid of them

[PCG4m3r]

Hey z,

 

Regarding all the new ini and all the effects and sky textures that were added, I'm wondering should they all end with "=SKP" ? The reason I'm asking is I made sure that new ini was in my DDSopt path and ran ver 8.0 update 3 against all Vanilla textures and even with that ini I still get issues like the purple sky problem due to modification to:

 

        sky\skyrimcloudsfill.dds

 

I made sure my setup was configured per the DDSopt Guide on the wiki, but it seems like DDSopt is still changing some (not sure if all) of those files in a bad way. I'm also seeing other textures that are 32x32 changed to 1x1, which is a problem I noticed before the new ini file. 

 

Am I doing something wrong or should we just set all those to be skipped with "=SKP" at the end of each line?

[z929669]

Hey z,

 

Regarding all the new ini and all the effects and sky textures that were added, I'm wondering should they all end with "=SKP" ? The reason I'm asking is I made sure that new ini was in my DDSopt path and ran ver 8.0 update 3 against all Vanilla textures and even with that ini I still get issues like the purple sky problem due to modification to:

 

        sky\skyrimcloudsfill.dds

 

I made sure my setup was configured per the DDSopt Guide on the wiki, but it seems like DDSopt is still changing some (not sure if all) of those files in a bad way. I'm also seeing other textures that are 32x32 changed to 1x1, which is a problem I noticed before the new ini file. 

 

Am I doing something wrong or should we just set all those to be skipped with "=SKP" at the end of each line?

Adding SKP just prevents processing altogether.

 

Have you verified the purple sky is associated with this texture and that you can fix it by replacing this file?

 

I have verified using the Compressionator that adding these files to the INI indeed changes the result as evidenced by the diff. I do notice the purple sky only on the very distant horizon from high altitude. I see this even with unoptimized vanilla.

 

Can you post screens of your purple (and corrected) sky issue?

 

Also, DDSopt will change 32x32 to 1x1 if there is no variation in the texture (i.e., it is a single color). This is a good thing, and Ethatron mentions it somewhere around here ...

[Garpov]

The pre 5b version of DDSopt handles almost all of the sky\ and effects\ textures correctly, what are the advantages of using the nexus version?

[PCG4m3r]

Hey z,

 

Regarding all the new ini and all the effects and sky textures that were added, I'm wondering should they all end with "=SKP" ? The reason I'm asking is I made sure that new ini was in my DDSopt path and ran ver 8.0 update 3 against all Vanilla textures and even with that ini I still get issues like the purple sky problem due to modification to:

 

        sky\skyrimcloudsfill.dds

 

I made sure my setup was configured per the DDSopt Guide on the wiki, but it seems like DDSopt is still changing some (not sure if all) of those files in a bad way. I'm also seeing other textures that are 32x32 changed to 1x1, which is a problem I noticed before the new ini file. 

 

Am I doing something wrong or should we just set all those to be skipped with "=SKP" at the end of each line?

Adding SKP just prevents processing altogether.

 

Have you verified the purple sky is associated with this texture and that you can fix it by replacing this file?

 

I have verified using the Compressionator that adding these files to the INI indeed changes the result as evidenced by the diff. I do notice the purple sky only on the very distant horizon from high altitude. I see this even with unoptimized vanilla.

 

Can you post screens of your purple (and corrected) sky issue?

 

Also, DDSopt will change 32x32 to 1x1 if there is no variation in the texture (i.e., it is a single color). This is a good thing, and Ethatron mentions it somewhere around here ...

Yes, after I noticed the purple sky, I copied over the original Vanilla textures (no DDSopt processing) and the sky reverted back to normal. The purple is very distracting when in game and may be more noticable at certain times of the day.

 

I uploaded a couple images to my imgur album here:

        https://imgur.com/a/lCIep#0

 

If that URL doesn't take you directly to the album, go here:

        https://pcg4m3r.imgur.com/

 

        then look in album: "DDSopt_v080preUpd3_with_Z_ini"

 

 

The images were taken with a complete vanilla game (no other mods and no HighRes DLC textures). The purple sky is the result of using DDSopt 8 update 3 with your custom ini and the other image is after I copied over all \sky textures from the Vanilla BSA. Time in game was 7:45am and location was near Ivarstead in case you were wondering.

 

EDIT: And forgot to add: I'm certain the purple is due to skyrimcloudsfill.dds. I copied just that one DDSopt-imized file over and the sky reverted back to purple. I recalled that one causing the same issue in an older post, so knew right where to look. It also happens to be one of the files that DDSopt changes from 32x32 to 1x1 size.

[JudgmentJay]

I uploaded a couple images to my imgur album here:

        https://imgur.com/a/lCIep#0

Kind of off-topic, but I'm wondering how you got the orange horizon color all the way down to the water. Is that normal? I always have a grey-ish horizon at a high enough altitude, as seen here:

 

 

That's a 100% vanilla game, .inis and all.