GUIDE [WIP] DDSopt & Texture Overhauls

[z929669]

Discussion thread:
DDSopt Guide by STEP
Wiki Link

---

 

GET DDSopt:
Github Pre-release versions
Official Nexus versions (select pre-release update 4)

[z929669]

The 64-bit exe sucks memory at a constant rate starting at just over 2 gigs at the start of processing. Once memory max is reached (for me, that's 8 gig), the process dies with a Windows memory error. Also, the processor does not get utilized at 100%, but rather fluctuates between approx 90-100%.

 

I'll test 32-bit next.

---

EDIT:Decided to restart and try the 64-bit once more to be certain I had clean memory. Looks like Windows is limiting DDSopt 64 to 7.5 Gb Phys mem on my system for now. Running 15 min, says 5:45:00 left, but that should decrease.

---

EDIT: Confirmed that 64-bit depletes RAM even after fresh startup. Happens at about 30 minutes into the run.

---

EDIT: 32-bit version crashes at roughly the same point after two attempts. Roughly 5 minutes into processing Skyrim - Textures.

[frihyland]

ahh ok the memory leak is probably what killed me than, must be very constant to happen at exactly the same point each time.

[z929669]

The 32-bit crash may not be related to memory, but I can't say for sure. It actually crashes and shuts down DDSopt. The 64-bit problem is memory related and results in a Windows warning and process stop (but not a crash... DDSopt window is still open).

 

I must say that I really like the new "Apply" menu and options, so configuration options look much better now. One other problem is that I cannot figure out how to get the image compare working. An added option to explicitly compare any two images or directories on demand would be very nice (e.g., see Compressonator image and entire-directory compare options ... both very handy).

[Ethatron]

So, this version is about 500-600% faster at DDS-processing than an already faster version (from before) than 0.7. And the quality should be up a bit as well.

[z929669]

grabbing it NOW :P

---

UPDATE: Memory leak is gone for 64-bit; consumed ram holding steady at just under 3 gig; textures processing noticeably faster; projected completion time down from about 13:00:00 initially (Pre1-2) to 2:45:00 initially with Pre3.

 

Previous versions would always drastically over-estimate processing time initially and gradually increase accuracy as the true endpoint approached... wonder if the same is true now or if the projection is simply more accurate? If the latter, then processing time will likely be about the same (Skyrim - Textures, HighResTexturePack01, HighResTexturePack02 [all loose files to loose files]). If not ....

---

... NOT. Finished processing these in less than 30 min!! (used to take 1.5 hr) will test later tonight

Final report "D:\_BACKUP\Skyrim Mods\DDSopt\0-out\Vanilla":
 processed files: 21233
 modified textures: 21093
 skipped textures: 10
 fixed textures: 0
 fixed files: 0
 broken files: 0 (0 without fixed ones)
 planar (1x1) textures: 3324
 changed texture formats: 3566
 i/o delta: -187975647 bytes
 tex delta: -170422552 bytes

---

... And the result of the above pass:

 

 

using default options without added alpha/constraint options. Order for both like sets below is:

 

 

DDSopt 7.3 > vanilla > DDSopt Pre3:

 

 

 

I like the new result better than both of the previous. I see better contrast and more detail, especially in the rocks. foliage looks great. I want to now compare the foliage options when ticked in the behave menu (will this then activate foliage as identified in the ini?).

[Ethatron]

Don't compare with the II, that is a literaly broken version. Every normalmap of II is kaputt fe.The foilage-algo is now almost exactly what they did, it's about this operation:

mip-alpha = alpha * (1 + level)

So level 1 is twice as bright, level 2 three times etc. No guarantee it works on the other maps, you have to look for some of the aspen.

Just use the diff-tool so see how staggering sometimes the improvement on the mip-maps is. I identified they used NVTT (it's marked in the DDS-header), but the quality is at times abysmal. Just look the pine texture at lower miplevels. DDSopt correctly maintains the brightness of the green at the tips of the branches, NVTT just makes it a dark green piece of mud.

Or some of the hrotgar-normal maps, NVTT has them really blocky on the brick-joints, the partial derivatives of DDSopt preserve such an amount of detail there, it's awesome.

[z929669]

Nice work Ethatron

 

I'll run some more tests using the added functionality and post the compares. I still cannot figure out how to activate the compare to get the diff output...

 

Lastly, my previous post was a compare of the last stable release of DDSopt (7.3), vanilla and this latest pre-release (III). IOW, I did not even include results from previous two DDSopt pre-releases (first pre-release had the NVTT "mud" branch tips, and the second would not even complete an optimization pass).

 

Will report back shortly with more compares.

 

Looking forward to this next final release

[z929669]

.. and the latest compares...

 

Vanilla > DDSopt preIII (default Behave) > DDsoptIII (alpha/constraint Behave):

 

Final report "D:\_BACKUP\Skyrim Mods\DDSopt\0-out\Vanilla":
 processed files: 21233
 modified textures: 21093
 skipped textures: 10
 fixed textures: 0
 fixed files: 0
 broken files: 0 (0 without fixed ones)
 planar (1x1) textures: 3324
 changed texture formats: 3566
 i/o delta: -187975647 bytes
 tex delta: -170422552 bytes

I think that the latest PreIII is a real winner; however, selecting:

Raise foliage-map ... &

Contrast-correct mip-map ... behave options

 

... had no effect at all in terms of I/O or visual aesthetic. Any reason why this might be the case?

 

All-in-all. Love it for speed and result, regardless

[stoppingby4now]

Great compares, detail really comes out with those. I notice select trees in the background as better defined, snow on rocks having more clarity, and rock crevices/edges become much more evident. The difference between PreIII and III is so subtle and only in very specific areas. Upon further inspection, I actually think the III images have some things with better detail. The ledge of upper ground on the left hand side is smoother, and more detail. The log on the ground slight to the right of center on the other side of the river also has better definition of moss.

[Ethatron]

Raise foliage-map ... &Contrast-correct mip-map ... behave options

... had no effect at all in terms of I/O or visual aesthetic. Any reason why this might be the case?

 

They are mutual exclusive for foilage. It'd give strange results if we'd mix them.

[frihyland]

Updated to use the new headertabs.

[Ethatron]

Can you give me green light to publish it? Please be a little bit thorough in testing and torturing, little checklist maybe, I think it shouldn't have hidden big issues though. Thanks for the comitment BTW. :^)

[z929669]

Will do.

Is there any way to spit out a timestamp in the log?? The only way I know how to currently estimate processing time is by watching the HH:MM:SS just before it completes (I would rather be doing something else!).

 

So far, I have run the new version over vanilla textures and hi res packs 3x each. Always processes very fast and looks good. What I have not done extensively is play the game and assess the textures of many in-game environments.

---

EDIT: all of the following are yet ADDITIONAL runs. So Other than the but I mention in the third post after this one, I think the Pre is definitely ready to be promoted to an official beta ....

[z929669]

DDSopt PreIII RUN:

 

Skyrim - Textures.bsa --> Skyrim - Textures (opt).bsa

 

Run time - 00:20:50 (no errors)

 

[log attached]

Skyrim - Textures (opt).7z

[z929669]

DDSopt PreIII RUN II:

 

Skyrim - Textures --> Skyrim - Textures (opt2)

 

 

 

Run Time (roughly the same as above) - no errors

[log attached]

Skyrim - Textures (opt2).7z