GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Kelmych]

I searched for "bsa" as a keyword, too short. Tried a couple phrases, no joy.

 

I, apparently, successfully optimized my vanilla, HRDLC, UHRP, and some loose file mod textures, according to the Wiki Guide. My resulting textures directory was 4.6GB. Game looked fine, Vurt's stress test crashed once (on shoreline west of Winterhold) out of several trials, and I was enjoying some better performance. Couldn't leave well-enough alone.

 

I have a small ramdisk (3GB) and even my optimized texture directory won't fit. However, (and I know this isn't popular around STEP), bsa compression drastically reduces texture storage requirements. I put my textures directory into 4 (a result of testing) bsa files with compression using Archive, registered them in Skyrim.ini, moved the Skyrim-Textures, 3 HRDLC, and the UHRP bsa's and esp's to storage, and renamed my textures directory to disable it entirely. The 4 bsa's (2.1GB total) are now in ramdisk using symlinks. Everything is fine so far; Vurt's test won't crash at all (its amazingly smooth, as well), and that worries me.

 

There are comments in the Wiki Guide concerning optimized textures packed into a bsa causing CTD's. Are there known locations (or known textures) which cause the CTD's? I can't find any problem areas outdoors and I'm getting sea-sick flying around at x1000 trying to find them.

 

I've had CTD problems at Fort Dawnguardwith the Dawnguard BSA when I recompressed all the resources back into a BSA, but I haven't tried using compressing textures back into BSAs and then registering them in the .ini files. The CTDs with DDSopt/BSAopt compressed BSAs are rare, but the problem is that if you get a CTD you don't know whether it is due to Skyrim mishandling a BSA you created or some other reason. Initially people thought the problem was with the level of compression that DDSopt/BSAopt uses which is higher than Bethesda typically uses, but some users have reported CTDs in Skyrim even when lower compression levels were used. I don't know the details of user problems with using textures in BSA files created by BSAopt, DDSopt, or other BSA creators.

 

I consider my current texture load a "default" setup, so the bsa's aren't that much of an issue for me. I'll put further mod textures in the directory tree, after I get my default game tested. I hate random crashes and am looking to eliminate as many potential sources as possible. Even "good guesses" as to why DDSOpt-imized textures in a bsa may cause crashes are more than welcome. Thx.

[Kelmych]

Some recommendations for SSD use during optimization were added to the DDSopt guide, and additional notes were added on use of the directory tree figure to ensure that files are in the right place for the batch files (this is one of the main causes of problems with the batch file processing). Some older figures were replaced with ones that better match the text, including the figures with recommended constraints settings when optimizing uncompressed textures.

[Searcher7]

 

 

(edited for brevity)

 

There are comments in the Wiki Guide concerning optimized textures packed into a bsa causing CTD's. Are there known locations (or known textures) which cause the CTD's? I can't find any problem areas outdoors and I'm getting sea-sick flying around at x1000 trying to find them.

 

I've had CTD problems at Fort Dawnguardwith the Dawnguard BSA when I recompressed all the resources back into a BSA, but I haven't tried using compressing textures back into BSAs and then registering them in the .ini files. The CTDs with DDSopt/BSAopt compressed BSAs are rare, but the problem is that if you get a CTD you don't know whether it is due to Skyrim mishandling a BSA you created or some other reason. Initially people thought the problem was with the level of compression that DDSopt/BSAopt uses which is higher than Bethesda typically uses, but some users have reported CTDs in Skyrim even when lower compression levels were used. I don't know the details of user problems with using textures in BSA files created by BSAopt, DDSopt, or other BSA creators.

 

 

Ahhh, misunderstood the context of the Wiki warnings; I created new BSA files, did not repack.

 

Wasn't sure of the exact load precedence of Skyrim.ini registered bsa's, so I created "dummy" esp's and moved my bsa loading to just below the default esm's. Still testing, but no repeatable CTD's yet. (Actually, none at all.) EDIT: Will thrash out Dawnguard HQ and see what happens.

 

Thanks for the reply.

[jakeslim]

While I, like many others (I would imagine), are extremely grateful for the time and effort put in by people to create the DDSOpt guide here @STEP, may I respectfully point out that it could do with some serious improvements in terms of organisation and clarity.

 

Having just followed it for the first time on one small mod as a test run, I'm surprised at how basic the actual steps I had to take were compared to the incredibly cluttered, complex, and confusing nature of the guide. If anyone would like me to provide specific input on ways to improve these things I'd be very willing to help, with the caveat that as you have gathered I know nothing about the technical aspects of DDSOpt. As a bare minimum I think there needs to be a simple step-by-step guide with literally NOTHING else interfering, and all extra info be provided on another tab or somesuch.

 

Please don't let this come across as bad-natured or ungrateful - it is not meant that way - I just really think that many people will either not use this guide, or use it incorrectly, due to the issues I describe, and it's a shame to let the good work put into it go to waste.

[Kelmych]

We always make use of comments on how to improve the guide. The guide has evolved based on comments and inputs from a lot of users. A lot of the material in the guide was originally shorter, but was expanded because of user comments on problems they had when performing optimization. If there are some useful ways to provide a shorter approach which provides the same graphic quality and that users find easier to follow than the main guide sections, the Quickstart section would be a good place for such changes. The intent of the quickstart section of the guide is to provide a brief overview and a simplified path through the optimization process. Currently there isn't an especially short path in this section.

 

With most mods, using DDSopt on an individual mod is often fairly straightforward. The approach in the guide tries to deals with how to handle a hundred or so mods some of which aren't necessarily easy to optimize. Originally the guide had users make use of the filtering capabilities built into DDSopt. The guide was noticeably shorter, but the steps were tedious and prone to error and most users didn't successfully complete the optimization. In addition, the steps didn't take care of some the quality problems that optimization was introducing in a few of the textures.

 

The optimization steps in the guide are currently aimed at getting as much texture size reduction as possible without noticeably change in graphic quality. If your goal for using DDSopt is different (e.g., reduce the size of textures for VRAM use reduction with an emphasis on simplicity of steps vs. best possible graphic quality) then it's certainly possible to have a simpler set of steps toward this goal.

[cybexgames]

@Kelmych

 

I just use DDSopt to optimize the Skyrim textures and found that using the Vanilla Uncompressed Textures settings, I am getting the characters bodies not being displayed correctly, what I mean that is that if I remove the armor the body the blacked out.

 

Can you post the Constraints settings in clear text as I am using DDSopt on a notebook and find reading the settings from the website hard.

 

I kept a copy original settings and they worked.

 

Not sure if I am doing something wrong...

 

Regards

Steve

[Kelmych]

The description of the constraints settings in the guide has been expanded to describe the recommended default uncompressed settings which use the lossless uncompressed parameters except for tangent space normal maps which use lossy settings.

 

The most recent posts in this thread are also discussing the male body texture problem.

 

What settings did you use when optimizing the vanilla textures? What mod management program are you using? Are you using all the STEP mods including RaceMenu, and what other mods are you using that affect male bodies?

 

I loaded a game with the vanilla textures and no mods (e.g., RaceMenu) that affect the male body. I had no problems with the display of the male body with either the 1Kx1K uncompressed malebody textures produced by DDSopt when optimizing HRDLC3 or the original 2Kx2K malebody textures from HRDLC3. The HRDLC3 textures overwrite those from Skyrim-textures (STD) so the game will use the ones from HRDLC3 when they are installed.

[cybexgames]

Hi Kelmych,

 

Using the settings from the STEP DDSopt wiki for Vanilla Uncompressed Textures, on the vanilla textures only. Using Mod Organizer and using no other mods except the Unofficial Skyrim Patches.

 

Will try to optimise again.... using the latest DDSopt.ini

 

[uPDATE]

Just used DDSopt to optimize just the Vanilla Uncompressed Textures for HRDLC3, using the Wiki settings and still got the same result. All the files in this folder seem to be *_msn.dds. I have no idea of why.

 

[uPDATE 2]

Change the Model-space normal-maps to R5G6B5 (2:3, lossy), instead of R8G8B8 (1:1, lossless) and the problem is no more.

[Kelmych]

What value did you use for the Constraint tab resolution limit parameter when you did the optimization? The textures in Vanilla Uncompressed\HRDLC3\textures\actors\character are all model space normal maps.

[cybexgames]

1024x1024 for both Uncompressed and Compressed

[Kelmych]

I ran a test with the optimized vanilla textures and a portion of the STEP mods and and I haven't found a problem yet, but I'll keep adding mods and seeing if there are problems.

[Octopuss]

While I, like many others (I would imagine), are extremely grateful for the time and effort put in by people to create the DDSOpt guide here @STEP, may I respectfully point out that it could do with some serious improvements in terms of organisation and clarity.

 

Having just followed it for the first time on one small mod as a test run, I'm surprised at how basic the actual steps I had to take were compared to the incredibly cluttered, complex, and confusing nature of the guide. If anyone would like me to provide specific input on ways to improve these things I'd be very willing to help, with the caveat that as you have gathered I know nothing about the technical aspects of DDSOpt. As a bare minimum I think there needs to be a simple step-by-step guide with literally NOTHING else interfering, and all extra info be provided on another tab or somesuch.

 

Please don't let this come across as bad-natured or ungrateful - it is not meant that way - I just really think that many people will either not use this guide, or use it incorrectly, due to the issues I describe, and it's a shame to let the good work put into it go to waste.

 

I tried to follow the guide and gave up after about an hour of staring at it. For a completely new user like myself, it's pure hell. I got as far as extracting some of the .bsa files in the suggested folder structure, but unfortunately the guide is so horribly overcomplicated I really have no idea what to do (like, what do I extract into the STD folder? where do I extract the non-DLC files?).

At first it seemed like a guide for optimizing the hi-res textures because there were some errors in them, but then it progressed into extracting absolutely everything, and then started about compressing textures in general which looks like lowering the quality for the sake of performance (I don't need that, my PC is powerful enough).

 

Now, I might either sound like an ass or an idiot (or both), but I completely fail to understand what is it about (I am sure it's very useful and even more if one isn't just a normal player but rather a modder).

Honestly, the guide could use radical simplifying process.

[Nearox]

Well if you're completey new I'd advise not to do something rather complicated as DDSopt the vanilla textures. Depending on your system &resolution, ddsopting may or may not be necessary. You can alternatively also use the 1024 Full version (which is actually the 'light' version) of the HRDLC Optimized mod mentioned in STEP. That should already save you quite some VRAM. Secondly, you can use the 2 mods called Bethesda Performance Textures (check nexus) to further reduce vram load.

 

That said, the instructions of SRLE use a different methodology and are slightly more concise than the step ddsopt guide. They might be easier to understand for you. https://wiki.step-project.com/User:Neovalen/Skyrim_Revisited_-_Legendary_Edition#Optimizing_The_Bethesda_Textures

[Kelmych]

While I, like many others (I would imagine), are extremely grateful for the time and effort put in by people to create the DDSOpt guide here @STEP, may I respectfully point out that it could do with some serious improvements in terms of organisation and clarity.

 

Having just followed it for the first time on one small mod as a test run, I'm surprised at how basic the actual steps I had to take were compared to the incredibly cluttered, complex, and confusing nature of the guide. If anyone would like me to provide specific input on ways to improve these things I'd be very willing to help, with the caveat that as you have gathered I know nothing about the technical aspects of DDSOpt. As a bare minimum I think there needs to be a simple step-by-step guide with literally NOTHING else interfering, and all extra info be provided on another tab or somesuch.

 

Please don't let this come across as bad-natured or ungrateful - it is not meant that way - I just really think that many people will either not use this guide, or use it incorrectly, due to the issues I describe, and it's a shame to let the good work put into it go to waste.

I tried to follow the guide and gave up after about an hour of staring at it. For a completely new user like myself, it's pure hell. I got as far as extracting some of the .bsa files in the suggested folder structure, but unfortunately the guide is so horribly overcomplicated I really have no idea what to do (like, what do I extract into the STD folder? where do I extract the non-DLC files?).

At first it seemed like a guide for optimizing the hi-res textures because there were some errors in them, but then it progressed into extracting absolutely everything, and then started about compressing textures in general which looks like lowering the quality for the sake of performance (I don't need that, my PC is powerful enough).

 

Now, I might either sound like an ass or an idiot (or both), but I completely fail to understand what is it about (I am sure it's very useful and even more if one isn't just a normal player but rather a modder).

Honestly, the guide could use radical simplifying process.

Some reorganization and editing was done today to make the guide a little easier to use, so any comments on portions that seemed particularly confusing would be useful. The first question I ask someone who wants to use a guide like this is "What do you want to accomplish". The guide. like most of the guides I've read or helped build, doesn't start with a map that says which parts to use depending on the answer to the first question. User comments have asked for a wide variety of information and the way this is accommodated is to add material so we don't have to frequently answer the same questions.

 

If you have some concrete suggestions (especially ones that are fairly specific) or examples please post them so we can make progress on improving the guide.

[Octopuss]

I don't think I am in a position to make any suggestions when I don't even understand what half of the stuff means :)

 

I am not even sure what I want to accomplish, but the main guide says "consider texture optimization", so I thought sure, why not, the possibility of better performance and cleaner data is always good. Unfortunately the way the guide is written (at least as of yesterday afternoon). It's very unclear what is its purpose. There seems to be several things mixed together in very confusing way.

 

Anyway, from random dude who tries to understand the whole thing's point of view, I would suggest:

 

1)

Go through all the links and make sure they are not dead or partially broken. Lots of them point to nonexistant parts of pages. Few examples:

Consider texture optimization link from the main guide is broken in that it opens the initial DDSOpt guide.

DDSopt instructions in the "repair vanilla HRDLC textures" part.

There are LOTS of these across the entire Wiki, and it makes already confusing pages even worse, because key informations are essentially missing.

 

2)

Simplify the guide. A lot. There are several parts that look almost identical to each other and seem to overlap.

I would suggest to add some sort of content index to the top of the page and to unify the format of individual sections, which despite being closely related to the main theme (texture optimization in general) talk about different things. I can imagine very brief explanation of the problem and goal, potential prerequisities, and clearly marked steps.

The guide often seems to be going into unnecessary depths when in fact it could be few relatively simple sentences (maybe I am exaggerating here a little). There is nothing wrong with explaining stuff in detail, but do not bloat the main steps leading to the goal (careful font formatting, bold and italics is really helpful here)

 

3)

Confusing instructions and/or inconsistencies.

I thought ok, I don't need to mess with everything, but if you say the hi-res textures have problems, I can optimize at least those. But that part of the guide suddenly starts talking about extracting everything without explaining why, and suddenly points me to somewhere at bottom of the page. IMO, this is not good, and if you insist, make a link that will take me to the exact spot. Repair the Vanilla HRDLC Textures section seems to abruptly end without any clear indication whether additional steps should be taken or what's supposed to happen, or whether I am supposed to continue to the next section, which looks to be unrelated (if I ended up on the page only with optimizing the hi-res textures in mind).

 

 

lol, I actually produced a wall of text without knowing what was I talking about. I think I would make a good manager.