[WiP] Community shader PBR support

[Exist]

Hi, writing about support for the new Community Shaders PBR assets. Hopefully not much would be needed from your side. I will mention some facts I believe are causing issues with LODs. Because the PBR textures are incompatible with vanilla, we use unique paths (textures/pbr/ prefix) to avoid unintended meshes from using one of them and looking wrong. Meshes that use PBR can be distinguished by the Unused01 property in Shader Flags 2. PBR albedo is brighter than vanilla diffuse and is saved in dds with the sRGB flag, which converts it to linear on the gpu. Now the problems:

1. Some object LODs seem to be using regular textures, for example mountains. Because LODs won't use the PBR shader, using unmodified PBR textures makes it look wrong. This is our issue and I think authors can just generate lower res textures in the vanilla paths that will only be used for these LODs. I don't know how many LODs actually use regular textures but hope not many.
2. TexGen seems to ignore PBR textures. I suppose TexGen always uses vanilla paths and ignores Texture Set records, otherwise I don't know why this would happen. In this case it would be nice to take texture set paths into account, or check if a PBR path exists before vanilla one. For landscapes we could also have authors generate low res versions at vanilla paths, but it's not viable to do this for everything.
3. DynDOLOD uses PBR textures, but probably because of the sRGB flag (which converts them to linear) the LODs are too dark. This happens to tree LODs, objects such as farmhouses and nordic ruins seem good, but not sure. This would probably happen in the textures in the previous point too if they weren't ignored. From our experiments vanilla diffuse is roughly halfway between usual PBR albedo brightness in linear and sRGB. So it would be best to do something like apply power of 1/1.5 to make the brightness consistent with vanilla. Ideally it could be multiplied with ambient occlusion and adjusted based on metalness which are in another texture, but that's not needed.

Hopefully I got the reasons for why it doesn't work right, I will include some comparison screenshots and links to PBR assets so you can try it yourself. Thank you for taking the time to read this and I hope this can be supported. Oh and feel free to ping me or others on CS or Cathedral discord, I might forget to check back here for some time.

https://imgsli.com/Mjg5ODQ5
https://imgsli.com/Mjg5ODUy
https://imgsli.com/Mjg5ODU0
https://imgsli.com/Mjg5ODU1

More info about our PBR implementation: https://github.com/doodlum/skyrim-community-shaders/wiki/True-PBR

Some assets to try:

Trees - https://mega.nz/file/dqdzGLwY#jBJFwPIWv7iX314UEKZnHCRvlITLbXI7saLY-4oV6Nw

Farmhouses - https://mega.nz/file/duUxDKiA#mnzDzjBFXEIDq-MFoucOHRJT55wOmbUNHOLZ5JxG3lY

Lots of other stuff - https://onedrive.live.com/?authkey=!AB5oYH6iqIMDkHM&id=C23FB01B59FA671C!2427&cid=C23FB01B59FA671C

[had]

Any suggestion why for texture on fort only partially match?

Idk if it cased by pbr, but i use pbr for forts. This one.

Logs:

https://drive.google.com/file/d/17xl8iRifcsSRFB1aOVW3S66nkf7PllRu/view?usp=sharing

https://drive.google.com/file/d/1QHu0d7d4ibKGLWm86nX4xcjNxO2Vx5Uy/view?usp=sharing

[sheson]

11 hours ago, had said:

Any suggestion why for texture on fort only partially match?

Idk if it cased by pbr, but i use pbr for forts. This one.

Logs:

https://drive.google.com/file/d/17xl8iRifcsSRFB1aOVW3S66nkf7PllRu/view?usp=sharing

https://drive.google.com/file/d/1QHu0d7d4ibKGLWm86nX4xcjNxO2Vx5Uy/view?usp=sharing

See https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs which TexGen and DynDOLOD debug log to also upload when making posts
Also upload ..\DynDOLOD\Logs\DynDOLOD_SSE_Object_Report.txt

Use tll in console to test if something is LOD or still a full model as explained in https://dyndolod.info/Official-DynDOLOD-Support-Forum#Rudimentary-Troubleshooting

Make a useful screenshot of the full model with more informative console and a close up of the LOD model as explained in https://dyndolod.info/Official-DynDOLOD-Support-Forum#In-Game-Screenshots

[had]

https://imgur.com/a/9M2xqIX

1, 2 Before/after tll.

3, 4 Ids, but cant select mismatching part with console while it still lod.

Seems like mismatching part is lod, while matching is full model.

Logs:

https://drive.google.com/file/d/12sUPutJDYMtPVOlJYa-GbsPRW22Z5p_V/view?usp=sharing

https://drive.google.com/file/d/17xl8iRifcsSRFB1aOVW3S66nkf7PllRu/view?usp=sharing

https://drive.google.com/file/d/1QHu0d7d4ibKGLWm86nX4xcjNxO2Vx5Uy/view?usp=sharing

https://drive.google.com/file/d/1Yiy0FfdPFtEMWYDGvAsPPJqbUouMX517/view?usp=sharing

https://drive.google.com/file/d/1kdI07EZ3MB4n_BjxdjTuhtrwW_w5lHw9/view?usp=sharing

[sheson]

22 hours ago, had said:

https://imgur.com/a/9M2xqIX

1, 2 Before/after tll.

3, 4 Ids, but cant select mismatching part with console while it still lod.

Seems like mismatching part is lod, while matching is full model.

Logs:

https://drive.google.com/file/d/12sUPutJDYMtPVOlJYa-GbsPRW22Z5p_V/view?usp=sharing

https://drive.google.com/file/d/17xl8iRifcsSRFB1aOVW3S66nkf7PllRu/view?usp=sharing

https://drive.google.com/file/d/1QHu0d7d4ibKGLWm86nX4xcjNxO2Vx5Uy/view?usp=sharing

https://drive.google.com/file/d/1Yiy0FfdPFtEMWYDGvAsPPJqbUouMX517/view?usp=sharing

https://drive.google.com/file/d/1kdI07EZ3MB4n_BjxdjTuhtrwW_w5lHw9/view?usp=sharing

The LOD seems to use the to linear converted PBR texture.

https://dyndolod.info/Mods/Community-Shaders#PBR

TexGen and DynDOLOD will both automatically create linear texture versions of the LOD textures for object and tree LOD in case the full model uses PBR textures. The gamma for the conversion can be adjusted by changing sRGBGamma=2.2 in ..\DynDOLOD\Edit Scripts\DynDOLOD\[TexGen|DynDOLOD]_[GAME MODE].INI. Use lower values to darken the LOD textures derived form the PBR textures.

Try maybe with 1.6 or so.

There is no feedback yet which gamma value works best to closely match the actual way the PBR shader works.

[had]

Do i need to redo both or only texgen?

[sheson]

38 minutes ago, had said:

Do i need to redo both or only texgen?

For testing just TexGen and then Rebuild Atlas in DynDOLOD expert mode.

https://dyndolod.info/Updating#New-or-Updated-Mods-or-Plugins
In case a new mod only updates textures or tree full models (and grass full models in case grass LOD is used) relevant to LOD, remove old TexGen output, then use TexGen to generate new output and install it as usual. Start DynDOLOD in expert mode, select all or only the affected worldspaces and click the Rebuild Atlas button to update the atlas textures. Then install the DynDOLOD output, overwriting the existing DynDOLOD output.

https://dyndolod.info/Help/Expert-Mode#Rebuild-Atlas

Once you found the best value, also update DynDOLOD INI and then generate from scratch.

[had]

Seems gamma correction 1.5 makes fort textures match, but i found rocks have some too white blobs or something

For example

https://imgur.com/a/iajGIfb

Why should this happen?

[sheson]

9 hours ago, had said:

Seems gamma correction 1.5 makes fort textures match, but i found rocks have some too white blobs or something

For example

https://imgur.com/a/iajGIfb

Why should this happen?

See https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs which TexGen and DynDOLOD debug log of the last generation to upload when making posts. Also upload ..\DynDOLOD\Logs\DynDOLOD_SSE_Object_Report.txt

Make useful screenshots of the full model(s) with more informative console as explained in https://dyndolod.info/Official-DynDOLOD-Support-Forum#In-Game-Screenshots. Would be helpful to also click to show the list of used textures with in more informative console.

[had]

Logs

https://drive.google.com/drive/folders/1_f0GfksavhGXlrbCBtHrmvXs724j8AOK?usp=drive_link

Screens

https://imgur.com/a/uBMowMw

Selected object, disabled, turned out where is plain white lod under it.

On last screen example with another rock. Real version is grey, lod much whiter. I changed gamma setting in dyndolod and redid everything from scratch, rock still very white, while fort is ok.

[sheson]

52 minutes ago, had said:

Logs

https://drive.google.com/drive/folders/1_f0GfksavhGXlrbCBtHrmvXs724j8AOK?usp=drive_link

Screens

https://imgur.com/a/uBMowMw

Selected object, disabled, turned out where is plain white lod under it.

On last screen example with another rock. Real version is grey, lod much whiter. I changed gamma setting in dyndolod and redid everything from scratch, rock still very white, while fort is ok.

Unfortunately the uploaded DynDOLOD log and debug log do not show the last LOD generation. They only show the tool being started and closed. Upload the last successful LOD generation session instead.
https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs
While the new log is appended to older logs, the debug log is replaced every time. Keep a copy somewhere else for future references if required.

You did not upload the requested DynDOLOD_SSE_Object_Report.txt but logs\DynDOLOD_SSE_Object_LOD.txt instead.

Curisouly, the More Informative Console window top left always shows the rock IsEnabled: Yes?!

Mountain and rock LOD in the first LOD level typically use the full mountain texture, which in case it is sRGB is converted by Texconv with a fixed gamma that can not be changed yet. The DynDOLOD log should have several lines that look like
Creating ..\DynDOLOD\DynDOLOD_Output\textures\pbr\*\*linear.dds
For example:
Creating ..\DynDOLOD\DynDOLOD_Output\textures\pbr\landscape\mountains\mountainslab01linear.dds
Creating ..\DynDOLOD\DynDOLOD_Output\textures\pbr\landscape\mountains\mountainslab02linear.dds

Find and compare those two textures to their source they are derived from ..\textures\pbr\landscape\mountains\mountainslab01.dds and mountainslab21.dds in an appropriate image/texture viewer that they look the same.  In that case test if darkening the *linear.dds textures manually with an appropriate image program has the desired effect.

LOD and full model are not supposed to show at the same time. See https://dyndolod.info/FAQ#Object-LOD-shows-in-active-exterior-cells:
Can be caused by [MapMenu] uLockedObjectMapLOD=32 in Skyrim.ini (or SkyrimCustom.INI or a mod INI), especially if there are no object LOD level 32 meshes (which is the vanilla/default). See Trees on the Map for further explanations. Also see Maps And Map Mods and Seasons if relevant.
A Clear Map of Skyrim is installed, make sure to really follow its instructions.

[had]

4 hours ago, sheson said:

You did not upload the requested DynDOLOD_SSE_Object_Report.txt but logs\DynDOLOD_SSE_Object_LOD.txt instead.

Redid output

https://drive.google.com/drive/folders/1_f0GfksavhGXlrbCBtHrmvXs724j8AOK?usp=drive_link

4 hours ago, sheson said:

Find and compare those two textures to their source

Yep, they looks same.

4 hours ago, sheson said:

A Clear Map of Skyrim is installed, make sure to really follow its instructions.

It is installed.

Idk, not much instructions to follow, i clicked rules as instructed.

6 hours ago, sheson said:

Curisouly, the More Informative Console window top left always shows the rock IsEnabled: Yes?!

It is just not updated in real time.

More screenshots

https://imgur.com/a/XRtd2td

Another thing i noticed: bug doesnt happen then i fastravel into location. But if i save game, relaunch game and load it is where.

Load game, bug present, fastravel out, fastravel in, bug not present.

Should it happen because mesh partially inside of loaded cell?

Color on rock changed, i added -30 brightness in paint net, but it doesnt match in different direction. I guess i need to change something else.

I guess for xlodgen gamma hardcoded too.

[sheson]

10 hours ago, had said:

Redid output

https://drive.google.com/drive/folders/1_f0GfksavhGXlrbCBtHrmvXs724j8AOK?usp=drive_link

Yep, they looks same.

It is installed.

Idk, not much instructions to follow, i clicked rules as instructed.

It is just not updated in real time.

More screenshots

https://imgur.com/a/XRtd2td

Another thing i noticed: bug doesnt happen then i fastravel into location. But if i save game, relaunch game and load it is where.

Load game, bug present, fastravel out, fastravel in, bug not present.

Should it happen because mesh partially inside of loaded cell?

Color on rock changed, i added -30 brightness in paint net, but it doesnt match in different direction. I guess i need to change something else.

I guess for xlodgen gamma hardcoded too.

Now that you know which texture to change, load the original PBR texture into an image program and change its gamma (levels).
That should have a bit of a different result compared to just reducing the brightness.
Future DynDOLOD version will use TexGen instead of Texconv to create the *linear.dds version.

Read the FAQ answer in my earlier post and this thread https://stepmodifications.org/forum/topic/20455-lod-and-full-models-overlapping/
The ACMOS instruction include to change the Level32 setting in the DynDOLOD_SSE.INI to generate object LOD level 32 files.

 

[sheson]

17 hours ago, had said:

Redid output

https://drive.google.com/drive/folders/1_f0GfksavhGXlrbCBtHrmvXs724j8AOK?usp=drive_link

Yep, they looks same.

It is installed.

Idk, not much instructions to follow, i clicked rules as instructed.

It is just not updated in real time.

More screenshots

https://imgur.com/a/XRtd2td

Another thing i noticed: bug doesnt happen then i fastravel into location. But if i save game, relaunch game and load it is where.

Load game, bug present, fastravel out, fastravel in, bug not present.

Should it happen because mesh partially inside of loaded cell?

Color on rock changed, i added -30 brightness in paint net, but it doesnt match in different direction. I guess i need to change something else.

I guess for xlodgen gamma hardcoded too.

Download this future version of DynDOLOD that uses TexGen routines/shader to create the linear RGB texture instead of Texconv with the hardcoded DirectX math routines https://mega.nz/file/9A5iWTrB#_Ond0nAwN0XvZnvPPsS9wd7ODlLhgZPV8A_HKZkPoqE

Set sRGBGamma in DynDOLOD_SSE.ini to the same value as you did in TexGen_SSE.ini. then generate LOD again.

If it complains about Texconvx64.exe download this version https://mega.nz/file/QV4w1YRY#vHCnSwg69AOLZOB1BpErWV2Tki7wDaq_UH7xDM4WfIY and replace it in the Edit Scripts folder.

Eventually there should be a simple and poor man's version of a PBR shader added to TexGen that creates a simple baked linear diffuse that hopefully matches a bit better what happens in the game with CS than just converting sRGB textures to linear RGB.

[had]

3 hours ago, sheson said:

The ACMOS instruction include to change the Level32 setting in the DynDOLOD_SSE.INI to generate object LOD level 32 files.

I did this too.

As far as i can see this is only one particular rock, maybe because it start in loaded cell.

This is my result with manually tweaked texture. Kind of close.

35 minutes ago, sheson said:

Download this future version of DynDOLOD that uses TexGen routines/shader to create the linear RGB texture instead of Texconv with the hardcoded DirectX math routines https://mega.nz/file/9A5iWTrB#_Ond0nAwN0XvZnvPPsS9wd7ODlLhgZPV8A_HKZkPoqE

Getting error with it.

Logs

https://drive.google.com/drive/folders/1dhOy27g6_7LGhjyjEiLlxDMOayyKJCZI?usp=sharing

[sheson]

1 hour ago, had said:

I did this too.

As far as i can see this is only one particular rock, maybe because it start in loaded cell.

This is my result with manually tweaked texture. Kind of close.

Getting error with it.

Logs

https://drive.google.com/drive/folders/1dhOy27g6_7LGhjyjEiLlxDMOayyKJCZI?usp=sharing

You do not have this error for the same load order with DynDOLOD 3 Alpha 182 release?

What is the last plugin to overwrite the worldspace arnimaDUPLICATE003 with form ID 15000D62 with 15 being arnima.esm

What is the last plugin to overwrite the water record 15035260 in arnima.esm?