Ethatron

Yeah, I detected on a mod some tint-maps, and because skin-tones are sometimes severly changed by the colour-precision offered by DXT-compression, I thought it should be adressable if someone wants to. They also are not like regular colour images which will be shown on your monitor, but they are basically data for a blending-algorithm which must not be considered as-if-to-be-displayed-on-your-monitor, like all the other colour textures. So while I already had to classify them I just added them to the constraints. Hm, I just exposed the strength "knobs" of the algorithms to the GUI. How much contrast is applied, how much gamma-corretion is applied, how much brighter (thicker) the foilage gets etc. They are all ad-hoc values, there is no scientific basis to it, trial'n'error.I added the information which strength will be used of what to the preview-info, in that info-field below. So one basically can become aware when what applies. In pre5 I think I added the new alpha-channel compressor, which should have half the error-rate. I also changed some thinks to be Skyrim-specific, which differ from Oblivion - like in Oblivion the used shader get's switched if you use DXT1 or DXT3, that is not the case in Skyrim; so in Skyrim it doesn't matter if a map is DXT1 or DXT5 unlike in Oblivion. This leads to more smaller textures, at least on my run. I recompressed all of the texture I use, and it looks fine so far. I'm still after the exact file which causes the screen's blood-splatter to go black (it's probably fxbloodflare.dds, and the reason is the 1x1 planar conversion), but haven't had the time to fire Skyrim up. The Soulgems Differ should work in theory, but the setup is such a PITA without NMM (and even with) that I haven't checked it out. You may just try and report back.

"Any volunteers?" Sure, just pass me the uncompressed maps, that's all what's needed.

I checked Crimson Tide out, in general it's okay, just the screen-splash is black, I'm not sure which that is (maybe screenbloodalpha01opt.dds).

:D Best of luck to us for this then.

Sure, no problem. I just wish there would be a standard etikette to approach authors just to make the mod a bit better. Sometimes people are seemingly resistant to see that one only has the best intentions. I don't assume someone does little errors intentionally, and teamwork is also a cool thing if you can trust [the intentions]. :)

"Vurt's Skyrim Flora Overhaul", well ... he destroyed a few of his textures with DXT3 compression: - architecture\whiterun\wrtreealivebark01_n.dds (should be DXT5) - architecture\whiterun\wrtreedeadbark01_n.dds (could be DXT5, should check PSNR in the compressonator) - landscape\grass\forestgrass01_n.dds (could be DXT5, should check PSNR in the compressonator) - landscape\grass\forestgrass02_n.dds (should be DXT5) - landscape\trees\treeaspenbarkcomp_N.dds (should be DXT5) - landscape\trees\treeaspenbranchcomp_n.dds (could be DXT5, should check PSNR in the compressonator) - landscape\trees\treepineforestbarkcomp.dds (should be DXT1, is DXT3 black) - landscape\trees\treepineforestbarkcomp_n.dds (could be DXT5, should check PSNR in the compressonator) - landscape\trees\treepineforestbarkcompsnow_N.dds (should be DXT5) - ... and so on, now I'm tired, actually all normal-map alpha channels should be revisited and probably almost all have higher quality as DXT5 I'm not sure if the bark-shader employs transparency, if yes, then all barks have black/transparent speckles. If someones convinces Vurt to give out the uncompressed maps, we can make a proper compressed version for him (using the format with the least error).

"Expanded Towns and Cities" has a broken BSA, the length-field of each file in the BSA is by 1 too short. Looking at the memory and the DDS-files it appears the size is wrong and no data missing, a DDS is alway a multiple of 4, and the common length is 349653, while a valid DDS file is 349652. Just wanted to share that, maybe someone is a fan of the mod and approaches the author kindly, telling that the used BSA-utility is borken.

SSE3: https://en.wikipedia.org/wiki/SSE3

I'll upload a new preview later (like right now). Most of the work I've done lately has more to do with DX11 compression (top quality of all available libraries), but I could cross-port a few of the things to the old DX9 compression. - alpha-channel compression should be about 50% nearer to lossless (I managed to reduce the compression error/loss by half), just matters for uncompressed-to-compressed conversions - colour compression should be a tiny bit more exact, but I don't think it's percievable - I think it should be a bit faster now as well My checklist: "Soul Gems Differ" passes now with compressed non-pow2 textures, -42.685.640 bytes"Better Dwemer Ruins" passes, but I think some normal maps are broken (black holes, rims), uh and "dwemerobservatoryskyrimmapglow.dds" shows what piece of junk the compressor did (green speckles, but different ones in each mip-level), -212.230.504 bytes"Better Dwemer Ruins - Bonus Clutter" passes, but I think some normal maps are broken (black holes, rims), -2.796.256 bytes"101 Bugs - HD Insect Retextures" passes, non-pow2 and no mipmaps in the original, lot's of strips but also lots of adding mipmap (every file), +612.216 bytes"W.A.T.E.R." passes, I changed the ini to "effects\fxwatertileflat_n.dds=RAW", that file needs every bit of precision it can get, -11.709.161 bytes"Better Beast Races" passes, I recommend compressing to R565 the difference is unpercievable, I have to check that in-game though, -67.458.380 bytes"Book of Silence" passes, though I want to investigate this pack a bit, the textures are the best I've seen (from compression PoV), though often very softened out (check out the alpha-channel of "armor\elven\f\cuirass_n.dds", at 1024x1024, the source is very flattened out, I almost think DXT5 would have been a better option anyway), I'm not sure but I may suggest to calculate all normals-maps without the steep-option, or to do a longer and detailed comparison, -12.757.888 bytes"Silverlight Armor", same observation, very nice texture, too much smooth in the mipmaps"Cloaks of Skyrim", passed, some texture are gamma-correct mipmapped, wow, others are not compressed at all, hmm, -154.839.011 bytes"Climates Of Tamriel", passed, basically no changes"Dev Aveza", passed, -2.490.648 bytes"Expanded Towns and Cities" passed, I added a tint-type under the constraints tab and prevented the gamma-correction to apply to face/tint-maps, -45.830.152 bytes"Expanded Winterhold Ruins" passes, -928.768 bytes"HQ Food Ingredients" passes, -19.923.736 bytes"HQ Towns and Villages" passes, -17.520.480 bytes"That's Ice" passes, but I recommend to compress to R565 here, I guess the DXT is too blocky, have to check that ingame, -1.835.007 bytes"Realistic Smoke and Embers", this already has R565 normal maps, no need to re-compress"Ruins Clutter Improvement HD" passes, interestingly there is one R444A4 normal-map in it, seems someone is thinking about precisions after all, still, -8.388.962 bytes"The Eyes of Beauty" passes, is criminal, not a single file with mipmaps, +20.362.668 bytes"Maty's Mithril" passes, -560.435 bytes"Real Wildlife" passes, I have the impression some maps are actually DDSopted, must be an old version as the files are not watermarked, -18.153.188 bytes"Morrowind Armor" passes, but it has probably the rarest of all rarest occurences, there is a black & white subsurface map on a dagger which gets converted to DXT1, basically killing all the texture, I added it to the .ini and I also extended the DXT1-aproved code to back off when there is alpha==black (fully transparent) but having full colour, -52.724.921 bytes"Project Parallax" passes, though I added parallax-maps as extra type to the constraints tab and prevented the gamma-correction to apply to parallax-maps, if you want really great quality they should be in L8 format, maybe someone convinces the author to release the "Lossless L8"-Kit of it, none of the textures has mip-maps, is that intentional?, +8.937.448 bytes "Terrain Bump" passes, these maps of course shouldn't be DXTed, defeats the purpose, but they are so noisy that you can use R444A4 and R565 on all of them without problem, -42.292.548 bytes"SkyTEST - Realistic Animals and Predators" passes, -20.385.406 bytes "Insanity's Texture Pack" passes, -35.652.176 bytes

Just a question, do you have all the texture uncompressed? I'd like to get a look onto big amounts of uncompressed normal-maps for example. DDSopt is suppose to do the compression from uncompressed to DDS as well, and I'd anyway like to verify the quality of the compressor in general. Any chance to have a peek?

Thanks. Do you know what the guy's using to create the DDSs?

Link?

C'mon, "look at every texture that exist" is not a good response. :P

Someone compiles me a list of mods to look at? (besides Soul Gems Differ) And the idea with the "ini" was that you can start managing, hacking, exchanging, sharing, organizing the unsolvable special cases without the supervisor. It's not the supervisor's favorite officially approved list. ;^) The special alpha-mode is only on for files marked "foilage" in the ini, it isn't applied by default.

Can you give me green light to publish it? Please be a little bit thorough in testing and torturing, little checklist maybe, I think it shouldn't have hidden big issues though. Thanks for the comitment BTW. :^)

They are mutual exclusive for foilage. It'd give strange results if we'd mix them.

Don't compare with the II, that is a literaly broken version. Every normalmap of II is kaputt fe.The foilage-algo is now almost exactly what they did, it's about this operation: mip-alpha = alpha * (1 + level)So level 1 is twice as bright, level 2 three times etc. No guarantee it works on the other maps, you have to look for some of the aspen.Just use the diff-tool so see how staggering sometimes the improvement on the mip-maps is. I identified they used NVTT (it's marked in the DDS-header), but the quality is at times abysmal. Just look the pine texture at lower miplevels. DDSopt correctly maintains the brightness of the green at the tips of the branches, NVTT just makes it a dark green piece of mud. Or some of the hrotgar-normal maps, NVTT has them really blocky on the brick-joints, the partial derivatives of DDSopt preserve such an amount of detail there, it's awesome.

So, this version is about 500-600% faster at DDS-processing than an already faster version (from before) than 0.7. And the quality should be up a bit as well.

I uploaded a "repaired" version now.

Looks kaputt to me. Sure, the foilage-files make sense.

LOL, yeah. Though after I managed to do the 40 minute run I thought to take it out again. :^D

options: No changes here, only better names and positions for quick comprehension. :) purple: Right, purple are files which could be processed (converted or optimized ...) but won't because of some setting (ini, or menu). italics: Is for indicating compressed files in the input (compressed in the BSA or NTFS compression). raise alpha opacity: This corresponds to a brightness adjustment to the alpha-channel when they are merged for the lower mipmap-levels from the higher resolutions. It doesn't appect level 0, just lower ones, and successively stronger. It's a balanced value, so it won't go really strong. It's tied to foilage, which can be marked as such in the ini. contrast-correction: This corresponds to a contrast adjustment to the alpha-channel the same way as above, just that here dark-grey gets blacker and light-grey gets whiter. This helps a lot with the inevitable blurriness of the alpha-mask when you go down the mipmap levels, it works for any texture with opacity in the alpha-channel, and won't be applied to those which have other stuff in the alpha-channel. don't eliminate: I wouldn't use it, but if someone wants to go purist, there is the option. :) difference previews: Lots of them, nice ones, I like the angular ones (color hue and normal rotation) the most and those show the "bad" loss in DXT-compression, magnitudes (of slope or brightness) are not that "bad" losses, and for partial derivative normal-maps it's a concious adjustment (>60° slope will artefact the lighting to some degree - in plain words >60° makes normal-mapping look unrealistic fakery instead of quite cool fakery). save as: Manually saving a texture to it's planned location (the default one) with automatically activate the difference view. ratio: In the drop-down you can see now the ratio of "current format" vs. "choosen format", so you know if you go smaller, or stay the same, or grow in size. All the fixes I made for next version BSAopt are in as well of course.

I'm a bit late don't worry, I'm just having it running a last time. It appears I can transcode Skyrim - Textures in about 40m (single pass bsa-to-bsa: decompress + optimize + recompress). Size is down to 1.292.054.283 from 1.329.559.634 (old version). Okay, then here is the testversion: https://github.com/Ethatron/ddsopt/downloads Assume all files are broken. :^P

Yes, that's correct. The artist was either manually filling in the mipmaps, so he made every downscaled version twiddling the contrast knob until the results were as he wanted; or he has a program which has a global knob to successively raise contrast and he choosed some value. As I put such a global "knob" - well it's two checkboxes, who's combination leads to more or less the same, I didn't want to expose any numbers (or sliders) as I anyway couldn't exactly reproduce the strong density present in that texture - into DDSopt, we may experiment with any foilage texture in the same manner. I think the artists may not have been the same (it's also visible in the quality of other textures, you almost can say which textures are made by which staff) so the Aspen haven't got it because there was no communication/synchronization, or he didn't think the Aspen was worth it, which implies he sat a few hours on the Pines and didn't want to repeat it for all the other foilage textures. Alpha-to-coverage is an engine feature, not a shader-"option". AFAIR only AMD has it as DX9 extension available, and it is mainly to support transparency AA - if implemented you get the half-transparent gradients back again - without you get the obvious on/off steps. That has to do with the simplification of the render-pipeline for speed, and error-robustness. That a texture has alpha-gradients of a specific kind is only to decide when on and when off is choosen (halftoning). I'll put a version-to-test-for-error on Github later, so you can play with that and please report any other errors, as I changed a hell of stuff in the compressor for speed.

There is no correlation with the name for Bethesda artists, see "treeaspenbranchcomp.dds", it's not specially made. But one can always make it better then Bethesda artists. ;) As far as I can see, the pine is the only texture made like that.