GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[phazer11]

Nope still no luck. Maybe give me a directory and a texture name to see if it's there and if it is troubleshoot it from there?

[Kelmych]

Look at the 1_HRDLC_Clean.bat file as a text file. Each line has the name of a texture or directory that is deleted.

[z929669]

The command line version of 7zip isn't as fast as the GUI version' date=' at least for me, even through it is also set for non-solid LZMA2 with multiple cores.[/quote']

testing everything now:

 

The instructions could be a bit more explicit in detailing that the user is to create "Vanilla Extracted" + all subs and then place all of the BAT files along side "Vanilla Textures". The directory housing "Vanilla Textures", this text file and all of the BAT files is henceforth referred to as "The Working Directory".

 

I had set things up wrong, because the instructions seemed to be telling me to place the BAT files into "Vanilla Extracted".

 

1_HRDLC_Clean-START.bat - worked without issue as expected ... 1-2 s

- NOTE: This produced a ton of errors at first, because I was missing some of the files it had expected to find (I had removed them previously and should have started by extracting all BSAs anew ... explicitly stating the need to have extracted the BSAs entirely and without edit just prior to running the BAT (and ensuring all prerequisites are met by using a checklist or better: add script to check if all of the expected folders are present and spit out results to a log. Add a pause after so that the user can be prompted to check the log prior to executing.

 

2_Pre-optimization.bat - worked without issue. Took less than 4 minutes on my SSD

- NOTE: see suggestions in the previous note. Also, add a pause at the end of this BAT so the user sees to check the log. There should be 14 blank lines in the log corresponding to each ">>log.txt"

- NOTE2: Also, it would be good to add rd /s /q  commands to delete the so-called temporary directories ("Vanilla Textures", "Vanilla Normal Maps", and "Vanilla Uncompressed Normal Maps") ... can you explain why these should be deleted? I should read the DDSopt Guide I suppose, but would rather get it from you if possible :P

 

Running the last pieces later ...

[Kelmych]

The command line version of 7zip isn't as fast as the GUI version' date=' at least for me' date=' even through it is also set for non-solid LZMA2 with multiple cores.[/quote'']

testing everything now:

 

The instructions could be a bit more explicit in detailing that the user is to create "Vanilla Extracted" + all subs and then place all of the BAT files along side "Vanilla TexturesExtracted". The directory housing "Vanilla TexturesExtracted", this text file and all of the BAT files is henceforth referred to as "The Working Directory".

"Vanilla Textures" is indeed in the working directory, but it isn't created until after the HRDLC repair is completed.

 

I had set things up wrong, because the instructions seemed to be telling me to place the BAT files into "Vanilla Extracted".

 

1_HRDLC_Clean-START.bat - worked without issue as expected ... 1-2 s

- NOTE: This produced a ton of errors at first, because I was missing some of the files it had expected to find (I had removed them previously and should have started by extracting all BSAs anew ... explicitly stating the need to have extracted the BSAs entirely and without edit just prior to running the BAT (and ensuring all prerequisites are met by using a checklist or better: add script to check if all of the expected folders are present and spit out results to a log. Add a pause after so that the user can be prompted to check the log prior to executing.

 

2_Pre-optimization.bat - worked without issue. Took less than 4 minutes on my SSD

- NOTE: see suggestions in the previous note. Also, add a pause at the end of this BAT so the user sees to check the log. There should be 14 blank lines in the log corresponding to each ">>log.txt"

- NOTE2: Also, it would be good to add rd /s /q  commands to delete the so-called temporary directories ("Vanilla Textures", "Vanilla Normal Maps", and "Vanilla Uncompressed Normal Maps") ... can you explain why these should be deleted? I should read the DDSopt Guide I suppose, but would rather get it from you if possible :P

 

Running the last pieces later ...

I'll change and expand the text in the Quickstart and DDSopt Optimization tabs to make it clearer that the batch files are in the Working Directory. I'll add some steps in the batch file to check for some of the correct folders, writing the results to the command window and a pause after these is printed.

 

I haven't personally deleted the temporary directories yet, but once the user has verified that the optimized textures are working properly in game these three directories can safely be removed. They contain extracted textures, and the Vanilla Extracted directory already contains un-archived versions of all these textures.

[Kelmych]

Nope still no luck. Maybe give me a directory and a texture name to see if it's there and if it is troubleshoot it from there?

I added more tests to the batch file to make sure it is running in the correct place and that the directory structure is what it expects. Try the new version of the the batch files. I also edited the guide discussion to make try to make the process clearer.

[phazer11]

I wasn't thinking clearly the other night and it of course occurred to me to look at the batch files in Notepad++ once I got to thinking of it (instead of trying to run it and read as the batch file executed too fast to read lmao) . It seems like it got rid of the extraneous textures though I'm not sure, I looked for a few and couldn't find them but can't say for sure. I'll try the new ones. as well.

[z929669]

I grabbed the latest BAT and began reading the Q-S guide ... got caught up revising that section to be a bit less wordy and found that all of my time had gone away ... TBC.

[phazer11]

Any updates? I just got around to trying out the pre-optimization and everything ran smoothly. I'm in Optimize the Textures portion of the guide and, for clarity I think you should specify which set of settings should be applied for the vanilla files for STEP Core. I'll assume the "Higher Quality, Ordinary textures" for now though I think STEP Core is generally 1k. I know this is more detailed lower in the steps liek step 3 but most people will be confused if you simply say "ensure that the application is configured according to the preceding screenshots" If I hadn't already looked throught he entire guide and known what I was doing I would have likely sat there trying to figure out which of the screenshots you were telling me to follow. Also you should probably add that people should archive there extracted vanilla files and save them somewhere (I know I'll be mentioning it in the video) and personally I would have the batch files move the files into the new directories instead of copying them which I don't know about you takes up a ton of space for me and I'm doing this on a SSD.

[Kelmych]

Any updates?

None yet - waiting for more users to try them. I'll add an additional small test in the HRDLC fix to try to notice potential problems. Also, I'll add a new batch file to delete the temporary folders (probably excluding Vanilla Extracted) since they take up a lot of space, and some guide text to go along with this.

I just got around to trying out the pre-optimization and everything ran smoothly. I'm in Optimize the Textures portion of the guide and, for clarity I think you should specify which set of settings should be applied for the vanilla files for STEP Core. I'll assume the "Higher Quality, Ordinary textures" for now though I think STEP Core is generally 1k.

We've been using "vanilla" to denote only the files supplied by Bethesda; I assume you mean the mod files in STEP:core. The STEP guide mentions which resolution version of a mod to use. It doesn't discuss optimization and tacitly assumes that all textures are processed the same way, and for this assumption the settings you mention are appropriate (along with unchecking the option to raise foliage-map opacity for each mip level ). I've been planning a set of batch files for optimizing STEP mods that, like the new vanilla texture processing, handle normal maps separately from color maps. These will use different settings for normal maps than color maps.

 

I know this is more detailed lower in the steps liek step 3 but most people will be confused if you simply say "ensure that the application is configured according to the preceding screenshots" If I hadn't already looked throught he entire guide and known what I was doing I would have likely sat there trying to figure out which of the screenshots you were telling me to follow.

I'll change this.

Also you should probably add that people should archive there extracted vanilla files and save them somewhere (I know I'll be mentioning it in the video) and personally I would have the batch files move the files into the new directories instead of copying them which I don't know about you takes up a ton of space for me and I'm doing this on a SSD.

A batch file to move a directory to a new directory based on user input is problematic because of all the potential problems with the user input. After running the new batch file that deletes everything except the batch files and the Vanilla Extracted directory a user can with a single GUI step move (or delete) the Working Directory or just move (or delete) the Vanilla Extracted folder.

[phazer11]

Any updates?

None yet - waiting for more users to try them. I'll add an additional small test in the HRDLC fix to try to notice potential problems. Also, I'll add a new batch file to delete the temporary folders (probably excluding Vanilla Extracted) since they take up a lot of space, and some guide text to go along with this.

I just got around to trying out the pre-optimization and everything ran smoothly. I'm in Optimize the Textures portion of the guide and, for clarity I think you should specify which set of settings should be applied for the vanilla files for STEP Core. I'll assume the "Higher Quality, Ordinary textures" for now though I think STEP Core is generally 1k.

We've been using "vanilla" to denote only the files supplied by Bethesda; I assume you mean the mod files in STEP:core. The STEP guide mentions which resolution version of a mod to use. It doesn't discuss optimization and tacitly assumes that all textures are processed the same way, and for this assumption the settings you mention are appropriate (along with unchecking the option to raise foliage-map opacity for each mip level ). I've been planning a set of batch files for optimizing STEP mods that, like the new vanilla texture processing, handle normal maps separately from color maps. These will use different settings for normal maps than color maps.

 

 

No I'm only talking about the vanilla textures. The guide doesn't specify what general STEP Core resolution is. So I was just wanting to clear it up for STEP users that the baseline is say... 1k for baseline for all textures to give them a reference point so they can choose to resize the vanilla textures if they think they'll need it. I'll probably be able to clear up most of this in the video I'm just making suggestions. I could also use your and Zed's input on the thread I made in the Tester section.

[phazer11]

Also to make it clear you should edit the 7z.exe bit to say that 7z.exe should be placed in the Vanilla Optimized Folder and not the main working directory.

[Kelmych]

Oops. Since I'm making some other small changes I'll fix the batch file for 7z-in-folder so it expects 7z.exe in the working directory. I'll upload a new version of several of the batch files with some small changes. There was also some editing of the guide to simplify the discussion of the process.

[phazer11]

Alright I'll take a look at it tomorrow with a fresh set of eyes mine are pretty much shot I got about an hour of sleep.

[z929669]

I will continue working through the guide instructions as well. I am the one that consolidated and avoided using redundant instructions, so I'll figure out a way to be more explicit and succinct at the same time.

[phazer11]

Well I don't have a problem with getting rid of some of the redundancy but I feel the succinctness would be better placed if it were moved around. I know the instructions and descriptions are there but they are in the section after the one I am talking about.