GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[Spock]

Hmm... strange. I executed the batch file again with the exact same mods and this time no error at all. I did delete the old log though, is the logging with the batch cumulative? Maybe that explains it.

 

The compress mods batch hat some problems naming the mods, for example: Serious HD Retexture RIFTEN 1024px-2146.7z_opt.7z

It seems like some files with "_" or "-" symbols have problems with naming.

[Kelmych]

I tried the archiving batch file on that file and others with dashes and spaces and underlines and there were no problems renaming. I'll add quotes around the file names in the batch file in case they are needed on some systems.

[Spock]

Hm... that is strange. Many of my optimized archives have weird file naming. Both the incidents make me think there was some kind of bug going on on my system. Maybe I have to go through the process again.

[Kelmych]

Renaming uses the rename command. You can use it in a command windows to see what it does to mods.

---

I also notice that the mod example you showed wasn't renamed correctly at all. You showed Serious HD Retexture RIFTEN 1024px-2146.7z_opt.7z but it should have been Serious HD Retexture RIFTEN 1024px-2146_opt.7z . The batch file uses the command

rename *.7z *_opt.7z

which would not have "7z" twice in the name

 

I'm also assuming you are using the batch file from the most recent set of batch files. The rename command is next command after the echo off and line that does the archiving.

[mikegray]

If the mods are in folders in the "Mod Optimized" folder, the "5_Compress_Mods_7z.bat" batch file will automatically archive the individual mods and rename them by adding "_opt" to the name (the DDSopt default renaming approach, by the way). You then have individual archives for each mod which are easy to use when installing. This approach also reduces issues in maintenance. When you want to use a new version of a mod you only need to replace a single file, and this doesn't affect any of the other mods already in MO. You'll find that over a few months mod maintenance (optimizing new updates and replacing old versions) will take more time than the original optimization and installation effort, so whatever approach you use should be designed to minimize maintenance.

Ah. The Mod Optimized folder is what I had missed. Next time I do this, knowing that will definitely simplify the job.

 

OTOH, for current testing purposes, I'm happy with what I've got: Every mod I ddsopt'd now has two folders in the Mod Organizer mods directory: The original version is followed by a second folder (exact same name + "OPT") directly under it in the load order. I can activate or deactivate the OPT folder at will in MO's left-hand pane, which is helpful right now. (For instance, I noticed that my ddsopting Warburg's maps to 1024/512 was a bad idea. So i just unchecked the OPT folder.) The only hassle was getting all the OPT folders moved to the right position in the load order - but now that they're in, they OUGHT to stay in play, even when I copy in new version of the ddsopt'd material.

[Spock]

Strange, I redid the whole progress and got misnamed files again. The folders in the mod optimized folder are named correctly. Dunno why this happens. I did rename a large bunch of mods at the same time maybe that's too much for the batch?

[Kelmych]

I tested the batch file with a large set of mods. Here is the version that adds quotes around the files for the renaming step. Let me know if it doesn't work and I'll do more searching with google to see if there are other reports with similar problems with the rename command. You can also try the following in a command window in the directory that contains the existing misnamed files.

 

rename "*.7z_opt.7z" "*_opt.7z"

 

 

This should fix any misnamed files. You can try it with or without the quotes; I put the quotes there in case they were needed on your system.

[Spock]

It's not only that, the batch sometimes also cuts off names. Like Falskaar V1_1_2-37994-1-1-2 (folder name) -> Falskaar V1_1_opt.7z. This is just an example, it did it with several more mods. I just hope all the files are there, but they probably are.

 

The quotes didn't change anything. This doesn't seem to be a random error because I get the same file names each time.

 

I'm on Win7 x64, with a OCZ Vertex 3 Max Iops in raid if that helps. I'm not actually using it as raid partition but my bios is in raid mode to backup my storage hdd (I don't work there though).

[Kelmych]

If the path name gets too long there could be a problem, but this is unusual since it would only happen with extremely long path names. What is the working directory name that is printed on the screen at the beginning of the batch file?

 

If you can, try running the batch file with the rename command removed (put "rem" without the quotes at the beginning of the line that starts with ren) and see if it is still a problem. There will not be an _opt in the name, but see if the folder name is correct.

 

I did some searching but couldn't find information on problems like the one you are having.

[mikegray]

I'm a little confused on one point - but I'd better back up first to explain what I've done so far:

 

- I'm working through a new set of mods I've downloaded from Nexus - most of them are STEP mods that give NPCs a helping hand in the beauty department.

 

- I've run "4_Mod_Sorting&Pre-optimization-beta.bat," which has separated the various textures into their respective folders (exterior, normal maps, mixed uncompressed - etc.

 

- I'm now ready to run ddsopt.

 

- I've setup ddsopt according to the tutorial here: https://wiki.step-project.com/Guide:DDSopt#tab=Using_DDSopt.

 

THE QUESTION: Having done this can I now run ddsopt on every single mod, with those exact same settings - except for my own personal choices regarding resolution limits - or do I need to change some of the other constraints for each folder?

 

I'm particularly interested in this question since most of these mods relate to things like skin, eyes, etc.

[Kelmych]

The changes recently in that batch file have been to improve how it handles skin. eyes, etc. Almost all of these textures should now be moved by "4_Mod_Sorting&Pre-optimization-beta.bat" to the Mod Mixed Uncompressed Textures folder so they can be optimized separately from the rest of the textures. The recommended constraints tab parameters in the Optimizing the STEP mods section of the guide should be fine, but for the Mod Mixed Uncompressed Textures folder I'd suggest having the model space normal map compression parameter set to lossless. There will be some compressed textures in this folder, but when DDSopt is optimizing the uncompressed textures using these parameters it will still automatically handle the compressed textures properly (it won't convert them to uncompressed).

 

Note that the part of the guide you reference is just for initial setup of DDSopt. The constraints tab parameters for use in optimizing mods are covered in the Optimizing the STEP mods section of the guide. There are different parameter sets for each of the 3-5 folders created by the batch file; of course, only a few of the parameters are actually changed when optimizing a different folder. 

[mothergoose729]

If I wanted to compress and resize only the color maps in a directory, but I wanted to leave normal maps/ect untouched, what options would I select in the DDSopt menu? If its not to much trouble, what would be the command line argument to do the same thing? Thank you.

[phazer11]

Well for the most part the normals end in either _msn or _n you can use the wildcard filter in DDSOpt to find all of those files with _n etc. then click the uncheck all button once the search results come up.

[Searcher7]

I searched for "bsa" as a keyword, too short. Tried a couple phrases, no joy.

 

I, apparently, successfully optimized my vanilla, HRDLC, UHRP, and some loose file mod textures, according to the Wiki Guide. My resulting textures directory was 4.6GB. Game looked fine, Vurt's stress test crashed once (on shoreline west of Winterhold) out of several trials, and I was enjoying some better performance. Couldn't leave well-enough alone.

 

I have a small ramdisk (3GB) and even my optimized texture directory won't fit. However, (and I know this isn't popular around STEP), bsa compression drastically reduces texture storage requirements. I put my textures directory into 4 (a result of testing) bsa files with compression using Archive, registered them in Skyrim.ini, moved the Skyrim-Textures, 3 HRDLC, and the UHRP bsa's and esp's to storage, and renamed my textures directory to disable it entirely. The 4 bsa's (2.1GB total) are now in ramdisk using symlinks. Everything is fine so far; Vurt's test won't crash at all (its amazingly smooth, as well), and that worries me.

 

There are comments in the Wiki Guide concerning optimized textures packed into a bsa causing CTD's. Are there known locations (or known textures) which cause the CTD's? I can't find any problem areas outdoors and I'm getting sea-sick flying around at x1000 trying to find them.

 

I consider my current texture load a "default" setup, so the bsa's aren't that much of an issue for me. I'll put further mod textures in the directory tree, after I get my default game tested. I hate random crashes and am looking to eliminate as many potential sources as possible. Even "good guesses" as to why DDSOpt-imized textures in a bsa may cause crashes are more than welcome. Thx.

[Kelmych]

Well for the most part the normals end in either _msn or _n you can use the wildcard filter in DDSOpt to find all of those files with _n etc. then click the uncheck all button once the search results come up.

There is also an example of selecting the tangent space normal maps and optimizing them in the Appendix of the DDSopt guide. You can use a variant of this for what you want.