Optimizing Texture Alpha Threshold

[Mousetick]

Recap of essential info:

• DynDOLOD 3a148 / Resources SE 3a45
• Vanilla tree meshes with Pine Branches Redone texture replacer
• Level0 at LOD Level 4
• DynDOLOD Resources SE's 3D tree LOD models for vanilla trees
• Edited to add UseMipMaps to crown nodes
• In the process of adjusting NiAlphaProperty Threshold to make LODs thinner

Now that the atlas textures are using the full texture mipmaps, I'm running into transparency/loss of details issues at smaller resolutions.

Look at the trees near the house in the center of the pictures below. Notice there is no difference between 112 and 127, and at 128 the LOD is very very close to the full model, which is fantastic.

LOD Alpha Threshold 112 > 127 > 128 > Full model

However notice the trees further in the distance to the right of the house have been severely "shaved" at Alpha threshold 128. It gets worse with smaller mipmaps, looking at the trees far away at the foot of the mountains:

It would seem that the full texture's mipmaps were not created with alpha-to-coverage, or they are, but with some kind of cutoff point for alpha around 127.

Based on the following shots, does it look to you like the mipmaps have alpha-to-coverage?

I see on Microsoft's Texconv page that the -keepcoverage option takes an "alpha test reference value". If the mipmaps of the full texture were generated with an "alpha test reference value" of 0.5, could this explain the observed behavior, i.e. no change between 112 and 127, and then "shaving" at 128?

What value does DynDOLOD use when it generates the atlas mipmaps (i.e. when not using UseMipMaps)?

Thanks in advance for your input.

[Mousetick]

On 9/27/2023 at 8:52 AM, sheson said:

If threshold < 128 alpha *= 128 / threshold
If threshold > 128 alpha ^= 1 / (128 / threshold * 2 - 1)

Is this the code verbatim or is there more to it? The formulae can overflow or produce out-of-range values.
For example, with threshold=27 and alpha=255, assuming alpha is an integer, the result is 1 or 257.

Edited September 29, 2023 by Mousetick

[sheson]

1 hour ago, Mousetick said:

Is this the code verbatim or is there more to it? The formulae can overflow or produce out-of-range values.
For example, with threshold=27 and alpha=255, assuming alpha is an integer, the result is 1 or 257.

Channels are clamped to 0 - 255.

https://github.com/ElminsterAU/imaginglib/blob/953f27413dc7e994b2ca07d05712458b2b87d19e/Source/ImagingUtility.pas#L834