GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Kelmych]

For vanilla texture installation my concept is the following (some details are left out):

• Copy all relevant vanilla BSAs into a working folder (vanilla textures, 3 HRDLC (if being used), and any DLC BSAs).
• Use BSAopt (preferred) or DDSopt  to extract all into the Vanilla Extracted directory (single process step).
• Run the revised batch file to repair the HRDLC (if HRDLC is present)
• Run a new batch file (using XXcopy) moves .png files to "Vanilla Optimized" and removes all non-texture data
• Use DDSopt to create optimizes the texture files putting them in "Vanilla Optimized" (single process step, no fiddling with filters)
• Run a new batch file (using 7z.exe with the proper command line flags) to archive the resulting folders and do any cleanup needed like moving or deleting the BSAs and moving the archived files to the top level working folder.

The intent is to minimize user steps as much as possible. The batch files will run properly with all BSAs or any subset.

 

 

I also have a concept for doing the STEP files, including handling the normal maps and handling the different resolution needs of different users, but I feel the vanilla textures version should be done first.

[z929669]

For vanilla texture installation my concept is the following (some details are left out):

• Copy all relevant vanilla BSAs into a working folder (vanilla textures, 3 HRDLC (if being used), and any DLC BSAs).
• Use BSAopt (preferred) or DDSopt  to extract all into the Vanilla Extracted directory (single process step).
• Run the revised batch file to repair the HRDLC (if HRDLC is present)
• Run a new batch file (using XXcopy) moves .png files to "Vanilla Optimized" and removes all non-texture data
• Use DDSopt to create optimizes the texture files putting them in "Vanilla Optimized" (single process step, no fiddling with filters)
• Run a new batch file (using 7z.exe with the proper command line flags) to archive the resulting folders and do any cleanup needed like moving or deleting the BSAs and moving the archived files to the top level working folder.

The intent is to minimize user steps as much as possible. The batch files will run properly with all BSAs or any subset.

 

 

I also have a concept for doing the STEP files, including handling the normal maps and handling the different resolution needs of different users, but I feel the vanilla textures version should be done first.

Are these BAT files ready, or are you proposing at this point? It should be possible to create a single BAT to handle all of this ...?

[phazer11]

What I did with the .PNG's takes longer but did seem to help a bit and not decrease the quality noticeably. I took them into GIMP after making mirrors of the texture directory's structure for the extracted vanilla textures and the DLC bsa's (textures/interface/books folder in the respective folders) and used the RIOT plugin to optimize them into .PNG while keeping transparency using batch mode.

 

While some oh them were minutely larger after optimization some were majorly smaller (between 50-75%) and I did side by side compares and couldn't really tell a difference. They aren't used all the time so I doubt it makes much difference but I figre every little bit helps. If anyone wants them send a holler and I'll upload them to drop box so people can try them out. Overall the book and other .png's don't take up that much space but...

 

Edit: Also is anyone else experiencing slow speed? It took me a long time to post this message and I know it's not on my end. Are the server's finally being migrated?

[Kelmych]

Are these BAT files ready' date=' or are you proposing at this point? It should be possible to create a single BAT to handle all of this ...?[/quote']

I wanted to show what I'm planning to get any comments you and others might have. I hope to finish the HRDLC repair batch today. I haven't found any suitable command line programs that can extract files from a BSA into a specified folder; DDSopt/BSAopt can compress but not extract. Many of the options we use in DDSopt in optimizing (e.g., the options in the constraint tab; these are the ones that need to be changed at runtime if it is run in a batch file) don't seem to be available in the command line version, and the command line version doesn't use the options that are set in the GUI version.

[EssArrBee]

On the Constraints tab if the boxes are unchecked for that certain type of texture, say model space normals, then will DDSopt just skip that type of texture? I ask because going through each folder to uncheck the normal maps is a bit of a pain, so just unchecking everything except the tangent space normals and running it again would be preferred.

[z929669]

On the Constraints tab if the boxes are unchecked for that certain type of texture' date=' say model space normals, then will DDSopt just skip that type of texture? I ask because going through each folder to uncheck the normal maps is a bit of a pain, so just unchecking everything except the tangent space normals and running it again would be preferred.[/quote']

The check boxes in "Constraints" are the fine-tuning versions of Apply > "Re-comress and re-mip all textures" (note that if you uncheck this setting, then those boxes are grayed out). If this remains unchecked for MSN, then DDSopt will not re-compress or re-mip those textures (*_msn.dds); however, it WILL compress (and mip) any uncompressed/unmipped textures of that type (AFAIK).

 

no need to individually tick/untick any of the textures in the browser pane. Just select the highest directory containing all directories of interest and apply a filter (either one of the pre-configured or a custom one) ... be sure to apply the filter after selecting/typing it in, and do the same to remove it.

 

Most normals in the game will be TSN, but there are others. Use this key to determine what is what:

 

 

        in a "menu"-folder: User-interface element, alpha is element-mask

        in a "landscapelod"- or "terrain"-folder: World-space normal-map for terrain, no alpha

        with a "_n"-suffix: Tangent-space normal-map, alpha is specularity

        with a "_msn"-suffix: Model-space normal-map, no alpha

        with a "_g"-, "_glow"- or "_emit"-suffix: Glow-map, alpha is a mask

        with a "_hh"-suffix: Gloss-map for hair, no alpha

        with a "_hl"-suffix: Detail-map for hair, alpha is opacity

        with a "_m"-suffix: Reflectivity-map for light-sources, no alpha

        with a "_em"- or "_envmap"-suffix: Reflectivity-map for environment-maps, no alpha

        with a "_e"-suffix: Environment-map (some are planar, some are cube-maps), no alpha

        with a "_b"- or "_bl"-suffix: Backlight-map, no alpha

        with a "_s"-suffix: Specularity-map for skins, no alpha

        with a "_sk"-suffix: Tone-map for skins, no alpha

        with a "_p"-suffix: Parallax-map, no alpha

        with a "_d"-suffix: Diffuse-map, alpha is opacity

        with a "_h"-suffix: Haze-map, alpha is unknown

 

 

[Kelmych]

The DDSopt guide now has an updated version of the HRDLC repair procedure using improved batch files that don't use drive substitution. The revised procedure should be much more robust.

[hellanios]

I'm planning to do a new clean install of Skyrim. Apart from the optimized textures[vanilla textures, HRDLC textures, Dawnguard and Dragonborn textures-with selected Maximum Resolution Limit Parameters 2048x2048 (with 1024x1024 maximum resolution for optional separate normal maps)] from the guide I want to use the Bethesda Hi-Res DLC Optimized mod(Hybrid + Vanilla normal maps). Can someone provide the order in which I should install each file?

 

Note : THe Unofficial High Resolution Patch --COMPATIBLE VERSION-- seems to be out of date since UHRP is in version 1.1.2.

[TechAngel85]

So if you followed the recommendation and optimized the normals for performance the installation order for optimized texture and normal archives should be like this I'm assuming:

• Skyrim - Textures (mine is named Vanilla Optimized)
• Skyrim - Textures Normals (mine is named Vanilla Optimized Normals)
• Dawnguard Textures
• Dawnguard Optimized Normals
• Hearthfire Textures
• Hearthfire Optimized Normals
• Dragonborn Textures
• Dragonborn Optimized Normals
• HRDLC 1 Textures
• HRDLC 1 Optimized Normals
• HRDLC 2 Textures
• HRDLC 2 Optimized Normals
• HRDLC 3 Textures
• HRDLC 3 Optimized Normals

This is the general order that I proposed. No one said it was wrong so I'm guessing it's the right order.

[hellanios]

Aha, but what about the Bethesda Hi-Res DLC Optimized mod? I can't use both the optimized textures and the mod?

[TechAngel85]

Sure you can. Just install it in the order that it is found in the mod tables. I install the above before any STEP mods are installed.

[z929669]

The DDSopt guide now has an updated version of the HRDLC repair procedure using improved batch files that don't use drive substitution. The revised procedure should be much more robust.

Thanks Kelmych. I'll check it out.

[rootsrat]

Optimising textures is something I never really understood - talking about mipmaps, normal maps and all the DDSOpt settings malarchy... I would really appreciate if someone explained to me what the settings in DDSOpt should be to optimise, say, Skyrim Flora Overhaul to 1K with half normals (as that what the common pratice seems to be).

 

I plan to optimise the main exterior texture replacers (namely Skyrim HD and SFO + Automatic Variants), but I am not sure how to do it, and reading through 101 pages of this thread isn't something I would be looking forward to. :)

 

Can anyone help?

[Aiyen]

You can just poke me on Steam and I will try to help out... a bit easier then creating a wall of text here!

[Kelmych]

Optimising textures is something I never really understood - talking about mipmaps, normal maps and all the DDSOpt settings malarchy... I would really appreciate if someone explained to me what the settings in DDSOpt should be to optimise, say, Skyrim Flora Overhaul to 1K with half normals (as that what the common pratice seems to be).

 

I plan to optimise the main exterior texture replacers (namely Skyrim HD and SFO + Automatic Variants), but I am not sure how to do it, and reading through 101 pages of this thread isn't something I would be looking forward to. :)

 

Can anyone help?

Unfortunately there haven't been a lot of comparison tests reported, especially with changing formats and resolutions for normal maps, to allow making good recommendations for anything other than architectural textures and some landscape textures for large objects like mountains. You could read through the entire 100+ pages of this thread and find relatively little in definitive test results and only a few useful anecdotal findings. It's particularly hard to make any recommendations for SFO since the author has already done some optimization, and many of the textures are rectangular (which complicates optimization). As I mentioned in a recent post, I'm working on some new processing steps, initially for the vanilla textures, to greatly simplify the optimization process. The problem I have isn't on creating some batch processing steps to eliminate a number of the current tedious steps - that's fairly straightforward - but on coming up with some default parameter recommendations especially for how to handle normal maps (e.g., does it actually make sense to use additional compression/lower resolution for normal maps with the vanilla textures since so few are architectural or landscape related; and is the answer different for a system running extreme STEP vs. a lower end system).