DynDOLOD 3.00 Alpha 180

[sheson]

1 hour ago, Modslave said:

 

Apologies for the delay.  I'd already started a fresh texgen/dyndolod output after trying to eliminate potential issues and even downgrading the Dyndolod script package to the last version and noticed the log requirements after that. I just had to sit through a 2 hour  Dyndolod output to ensure I have the logs required. One hour and 44 minutes of that literally being generating the Tamriel LODGen.

Here are the relevant logs I believe.

https://drive.google.com/file/d/1j2puHpv6JfE2sszQkgj_TQQXpOqbWeRj/view?usp=drive_link

The debug log alone is 185mb something is very not right here.

The largest asset folder in the output is the \DynDOLOD_Output\Meshes\Terrain\Tamriel\Objects folder, which is coming in at a whopping 27.6 GB uncompressed.

Here is the Tamriel ModelsUsed.

https://pastebin.com/Sa5REP8n

The strange part of this is I haven't added anything that should be doing this, I've actually been removing things that add objects. And its not helping.

My Texgen output.....by comparison, is literally less than 130 MB.

The size of the debug log is irrelevant.

From DynDOLOD_SSE_log.txt (which shows the top 10 from DynDOLOD_SSE_ModelsUsed_Tamriel.txt)
Meshes\dyndolod\lod\trees\landscape\treepineforest05_6396fa85passthru_lod.nif used 3770 times, ~ 2647.68 MB
Meshes\dyndolod\lod\trees\landscape\treepineforest03_adaf1a95passthru_lod.nif used 3462 times, ~ 2130.19 MB
Meshes\dyndolod\lod\trees\landscape\treepineforest02_d20290f1passthru_lod.nif used 2217 times, ~ 1557.00 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow05_1e583551passthru_lod.nif used 2846 times, ~ 487.41 MB
Meshes\dyndolod\lod\trees\landscape\treeaspen03_038d0d94passthru_lod.nif used 1396 times, ~ 440.27 MB
Meshes\dyndolod\lod\trees\landscape\treeaspen01_26002b66passthru_lod.nif used 1053 times, ~ 332.10 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow02_abdadffbpassthru_lod.nif used 2121 times, ~ 299.71 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow03_8606344epassthru_lod.nif used 2624 times, ~ 254.38 MB
Meshes\dyndolod\lod\trees\landscape\treeaspen04_78718205passthru_lod.nif used 747 times, ~ 235.58 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow04_3101733epassthru_lod.nif used 2311 times, ~ 230.90 MB
Just these 10 not really optimized 3D tree "LOD" models occupy 8615.22 MB = 8.4GB of the object LOD for Tamriel
Just as example, the vanilla 3D tree LOD model for the vanilla tree treepineforest05 is ~8kB, while the 3D tree "LOD" model above is  ~720kb. Almost a 100 times more. If you can not find better optimized 3D tree LOD models, consider using Billboard4 (for some) instead.

Also check the DynDOLOD_SSE_log.txt for messages similar to the one below which lists that the full model is used for LOD
<Warning: LOD model meshes\lod\windhelmsurjamte\wheeltowerbridge_lod.nif has same CRC32 2B781726 as full model Meshes\surwindhelmcustommeshes\architecture\wheeltowerbridge.nif WindhelmSSE.esp WHgate2WHNew [STAT:311BE6AA]>
Each one by itself does not seem much with ~1 or 2 MB. However, actual properly made LOD models are often much less than 100kB.

Grass LOD is being generated. Lowering the grass LOD density will help to reduce BTO files size.

Object LOD Level 32 for the map is being generated.
Some specific rules define to use full models for object LOD Level 32.
The last / rules defines to use LOD models meant for object LOD level 4 for the LOD Level 32.
Typically the map uses object LOD 16 that is much less detailed with LOD models made for LOD level 16 and none for smaller objects.
Note the total size of all object LOD Level 32 files. Stick with the default object LOD 16 for the map instead if necessary.

[bread-dreams]

12 hours ago, sheson said:

If the CTD also happens with a LOD patch that is either using DynDOLOD DLL SSE and Scripts or PapyrusUtil as you seem to report, also upload a crash log of that. Not need for the DynDOLOD log or debug log for that generation.

Sorry, forgot to mention that the CTDs do not seem to happen with the non-NG DLL, at least as far as I could see, but on the other hand loading times when entering/exiting places seemed a tiny bit longer than usual which was actually annoying though it totally could be some sort of weird momentary issue or placebo or whatever, I should try again later.

12 hours ago, sheson said:

To make sure, nothing overwrites textures\clutter\ShrineofAzura.dds or textures\clutter\ShrineofAzura_n.dds?

Well I'm using Cleaned Skyrim SE Textures which does override the vanilla ShrineofAzura(_n).dds but no other mod overrides it including DynDOLOD.

12 hours ago, sheson said:

Unrelated, how are the better terrain LOD Meshes without the vanilla bugs and with almost perfect coast lines a problem? What is the issue with better terrain LOD textures that match full terrain in color and tone? What alternative are you using that was not made with xLODGen? I suggest to read the first post, https://dyndolod.info/Help/xLODGen and/or to follow a modding guide.

Well to be completely fair, when I did generate with xLODGen I used fairly low quality settings because my computer isn't that powerful so I was worried about performance, but I totally could try again with higher quality settings. One thing I did notice was that the snow colour was way too dark in the world map (honestly looked like it was missing nearly? not sure), whilst in vanilla it's you know, snow coloured, off white. I wasn't feeling like figuring out how to fix that, because when I searched for similar issues it required messing with the brightness or gamma settings or whatever and that's kind of annoying especially when generating takes a long time.

And i'm not using any alternative really, I'm literally just using whatever comes vanilla. So basically I generated billboards with TexGen and then LOD with DynDOLOD, without running xLODGen at all or installing any other sort of terrain textures or lod or whatever beforehand, which seems to work fine for me, I'm not very demanding regards graphics and I'm only using DynDOLOD because it's a neat tool and absolutely improves LOD, but xLODGen's textures didn't do it for me.

Thanks for the help again, your tools are wonderful.

[sheson]

41 minutes ago, bread-dreams said:

Sorry, forgot to mention that the CTDs do not seem to happen with the non-NG DLL, at least as far as I could see, but on the other hand loading times when entering/exiting places seemed a tiny bit longer than usual which was actually annoying though it totally could be some sort of weird momentary issue or placebo or whatever, I should try again later.

Well I'm using Cleaned Skyrim SE Textures which does override the vanilla ShrineofAzura(_n).dds but no other mod overrides it including DynDOLOD.

Well to be completely fair, when I did generate with xLODGen I used fairly low quality settings because my computer isn't that powerful so I was worried about performance, but I totally could try again with higher quality settings. One thing I did notice was that the snow colour was way too dark in the world map (honestly looked like it was missing nearly? not sure), whilst in vanilla it's you know, snow coloured, off white. I wasn't feeling like figuring out how to fix that, because when I searched for similar issues it required messing with the brightness or gamma settings or whatever and that's kind of annoying especially when generating takes a long time.

And i'm not using any alternative really, I'm literally just using whatever comes vanilla. So basically I generated billboards with TexGen and then LOD with DynDOLOD, without running xLODGen at all or installing any other sort of terrain textures or lod or whatever beforehand, which seems to work fine for me, I'm not very demanding regards graphics and I'm only using DynDOLOD because it's a neat tool and absolutely improves LOD, but xLODGen's textures didn't do it for me.

Thanks for the help again, your tools are wonderful.

Can you confirm the CTD still happens if the statue uses the vanilla textures?

Curious, since the vanilla terrain LOD textures basically never match the full landscape textures and snow in particular is just always off. If contrast, gamma and brightness are left alone, then the combined terrain LOD textures generated by xLODGen  have exactly the color and brightness as the full landscape textures. The settings can not make a single texture darker or brighter, it always affects the end result equally. In addition to the noise.dds texture, the map has specific INI settings / overlay textures which are applied in the game onto terrain LOD textures and thus can be changed at any time https://dyndolod.info/Mods/Maps-And-Map-Mods#Snow. If generating all terrain LOD textures takes a long time, then for testing, generate a specific chunk only until satisfied. You may want to check out A Clear Map of Skyrim and Other Worlds.
Instead of vanilla terrain LOD textures, you could use the terrain LDO textures from Terrain LOD Redone. You will have to get rid its terrain LOD meshes though or use xLODGen to generate terrain LOD meshes only in order to overwrite them.

[Modslave]

3 hours ago, sheson said:

The size of the debug log is irrelevant.

From DynDOLOD_SSE_log.txt (which shows the top 10 from DynDOLOD_SSE_ModelsUsed_Tamriel.txt)
Meshes\dyndolod\lod\trees\landscape\treepineforest05_6396fa85passthru_lod.nif used 3770 times, ~ 2647.68 MB
Meshes\dyndolod\lod\trees\landscape\treepineforest03_adaf1a95passthru_lod.nif used 3462 times, ~ 2130.19 MB
Meshes\dyndolod\lod\trees\landscape\treepineforest02_d20290f1passthru_lod.nif used 2217 times, ~ 1557.00 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow05_1e583551passthru_lod.nif used 2846 times, ~ 487.41 MB
Meshes\dyndolod\lod\trees\landscape\treeaspen03_038d0d94passthru_lod.nif used 1396 times, ~ 440.27 MB
Meshes\dyndolod\lod\trees\landscape\treeaspen01_26002b66passthru_lod.nif used 1053 times, ~ 332.10 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow02_abdadffbpassthru_lod.nif used 2121 times, ~ 299.71 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow03_8606344epassthru_lod.nif used 2624 times, ~ 254.38 MB
Meshes\dyndolod\lod\trees\landscape\treeaspen04_78718205passthru_lod.nif used 747 times, ~ 235.58 MB
Meshes\dyndolod\lod\trees\landscape\treepineforestsnow04_3101733epassthru_lod.nif used 2311 times, ~ 230.90 MB
Just these 10 not really optimized 3D tree "LOD" models occupy 8615.22 MB = 8.4GB of the object LOD for Tamriel
Just as example, the vanilla 3D tree LOD model for the vanilla tree treepineforest05 is ~8kB, while the 3D tree "LOD" model above is  ~720kb. Almost a 100 times more. If you can not find better optimized 3D tree LOD models, consider using Billboard4 (for some) instead.

Also check the DynDOLOD_SSE_log.txt for messages similar to the one below which lists that the full model is used for LOD
<Warning: LOD model meshes\lod\windhelmsurjamte\wheeltowerbridge_lod.nif has same CRC32 2B781726 as full model Meshes\surwindhelmcustommeshes\architecture\wheeltowerbridge.nif WindhelmSSE.esp WHgate2WHNew [STAT:311BE6AA]>
Each one by itself does not seem much with ~1 or 2 MB. However, actual properly made LOD models are often much less than 100kB.

Grass LOD is being generated. Lowering the grass LOD density will help to reduce BTO files size.

Object LOD Level 32 for the map is being generated.
Some specific rules define to use full models for object LOD Level 32.
The last / rules defines to use LOD models meant for object LOD level 4 for the LOD Level 32.
Typically the map uses object LOD 16 that is much less detailed with LOD models made for LOD level 16 and none for smaller objects.
Note the total size of all object LOD Level 32 files. Stick with the default object LOD 16 for the map instead if necessary.

Hrrrm well this is all very curious because the tree and grass models I'm using are the same ones I've been using since 2022. There was an update for them recently which I did update to, but they've never done this before....the passthru lods there seem to be being generated as a reference object from nifs but with much larger file sizes because Veydogolt is no where near that size for the passthru lods's in its own files.

The only level32 considerations are for ACMOS. Billboard 6 for Tree and Level0 for /

 

[sheson]

31 minutes ago, Modslave said:

Hrrrm well this is all very curious because the tree and grass models I'm using are the same ones I've been using since 2022. There was an update for them recently which I did update to, but they've never done this before....the passthru lods there seem to be being generated as a reference object from nifs but with much larger file sizes because Veydogolt is no where near that size for the passthru lods's in its own files.

The only level32 considerations are for ACMOS. Billboard 6 for Tree and Level0 for /

The screenshot shows that the 3D tree "LOD" file treepineforest05_6396fa85passthru_lod.nif has a file size of 720kB.
The log you posted shows the tree occurs 3770 times in the object LOD of Tamriel. That means the LOD for these 3770 trees accounts for almost 2.6GB file size as reported by the logs. 

ACMOS contains more than the two mesh rules you mention. It also sets a good number of full models being used for LOD Level 32.

[Modslave]

Okay well this is all far more arcane than I've ever had to deal with with this system, I didn't just download a collection yesterday and have no idea what I'm doing I've been using Dyndolod with these resources now for several years.

While I appreciate you taking the time to answer me you're not really telling me anything you're just telling me something that used to work doesn't work like it did literally 2 days ago and to RTFM.

So I guess I'll just uninstall everything and start over cause this is getting me no where.

EDIT to add: I just tried a Dyndolod session backing down Vedogoit Trees from 10.1 to 7.5 and got similar long period LODGen and high file sizes under Alpha 169.

I have now backed down Dyndolod itself to Alpha 167 which was the last working version I had that generated outputs in about 20 minutes and am trying a new output now, will report back once I see what the results are.

Edited April 10 by Modslave

[sheson]

10 hours ago, Modslave said:

Okay well this is all far more arcane than I've ever had to deal with with this system, I didn't just download a collection yesterday and have no idea what I'm doing I've been using Dyndolod with these resources now for several years.

While I appreciate you taking the time to answer me you're not really telling me anything you're just telling me something that used to work doesn't work like it did literally 2 days ago and to RTFM.

So I guess I'll just uninstall everything and start over cause this is getting me no where.

EDIT to add: I just tried a Dyndolod session backing down Vedogoit Trees from 10.1 to 7.5 and got similar long period LODGen and high file sizes under Alpha 169.

I have now backed down Dyndolod itself to Alpha 167 which was the last working version I had that generated outputs in about 20 minutes and am trying a new output now, will report back once I see what the results are.

I looked at the logs that were provided and explained several factual reasons that are causing large object LOD files. I did not mention or made comparisons to older generations, since no other logs were provided. I did not tell you to RTFM in the last two posts here or here. That was only in the first post to refer to the explanations which log files to upload and to the already existing FAQ answers for this issue.

Let me summarize the reasons for large object LOD files I already wrote about in the past posts contrary to your claim I did not tell anything:
Ultra tree LOD with way too large 3D tree "LOD" from third party mods cause huge object LOD files requiring at least a dozen GB. 
Mods use or set full models to be used for object LOD.
Grass LOD is being generated.
LOD Level 32 is being generated.

[sheson]

23 hours ago, bread-dreams said:

Sorry, forgot to mention that the CTDs do not seem to happen with the non-NG DLL, at least as far as I could see, but on the other hand loading times when entering/exiting places seemed a tiny bit longer than usual which was actually annoying though it totally could be some sort of weird momentary issue or placebo or whatever, I should try again later.

Get DynDOLOD DLL NG Alpha 18. It hopefully fixes the CTD.

[bread-dreams]

31 minutes ago, sheson said:

Get DynDOLOD DLL NG Alpha 18. It hopefully fixes the CTD.

Hehe this is good timing, I was just testing what you suggested ("Can you confirm the CTD still happens if the statue uses the vanilla textures?"). I hid the modded textures and after a few hours of testing, I could not get a single crash. I unhid them and after a bit of playing I got a crash (leaving Frostflow Lighthouse southeast of Dawnstar) so they do seem to have been the problem somehow? Had a look at the modded textures and nothing seems obviously broken or something about them so I have really no idea why they would be causing crashes. It might be a fluke.

But now I'll try the updated DLL, with the modded textures enabled. I might take a while but I'll report back about whether or not the CTDs happen (hopefully they don't obvs). Would love to know what you changed too. Also below is what I had written before you posted about the new DLL, about xLODGen.

23 hours ago, sheson said:

Curious, since the vanilla terrain LOD textures basically never match the full landscape textures and snow in particular is just always off.

The main issue for me is the world map, which ends up looking fairly ugly to me. Here's a comparison, without xlodgen on the left and with xlodgen on the right.

  

Also, for example, around Morthal, the swamp ends up looking broken.

Is this just because of low quality settings?

23 hours ago, sheson said:

If generating all terrain LOD textures takes a long time, then for testing, generate a specific chunk only until satisfied. You may want to check out A Clear Map of Skyrim and Other Worlds.
Instead of vanilla terrain LOD textures, you could use the terrain LDO textures from Terrain LOD Redone. You will have to get rid its terrain LOD meshes though or use xLODGen to generate terrain LOD meshes only in order to overwrite them.

Cheers, I'll check these out when I can, and I think I must've messed up some settings when using xLODGen but I didn't realise you could generate specific chunks so that will definitely help, thank you.

[sheson]

10 hours ago, bread-dreams said:

Hehe this is good timing, I was just testing what you suggested ("Can you confirm the CTD still happens if the statue uses the vanilla textures?"). I hid the modded textures and after a few hours of testing, I could not get a single crash. I unhid them and after a bit of playing I got a crash (leaving Frostflow Lighthouse southeast of Dawnstar) so they do seem to have been the problem somehow? Had a look at the modded textures and nothing seems obviously broken or something about them so I have really no idea why they would be causing crashes. It might be a fluke.

But now I'll try the updated DLL, with the modded textures enabled. I might take a while but I'll report back about whether or not the CTDs happen (hopefully they don't obvs). Would love to know what you changed too. Also below is what I had written before you posted about the new DLL, about xLODGen.

The main issue for me is the world map, which ends up looking fairly ugly to me. Here's a comparison, without xlodgen on the left and with xlodgen on the right.

  

Also, for example, around Morthal, the swamp ends up looking broken.

Is this just because of low quality settings?

Cheers, I'll check these out when I can, and I think I must've messed up some settings when using xLODGen but I didn't realise you could generate specific chunks so that will definitely help, thank you.

From the changelog: DynDOLOD DLL NG and Scripts - do not reset currently disabled persistent large references

To improve the coastlines of terrain LOD meshes on the map generated by xLODGen, set the Optimized Unseen dropdown to 550 with a quality lower than 10 (0 for best) for LOD level 32 as explained on the first post and https://dyndolod.info/Help/xLODGen#Generating-Terrain-LOD-Meshes

[bread-dreams]

12 hours ago, sheson said:

To improve the coastlines of terrain LOD meshes on the map generated by xLODGen, set the Optimized Unseen dropdown to 550 with a quality lower than 10 (0 for best) for LOD level 32 as explained on the first post and https://dyndolod.info/Help/xLODGen#Generating-Terrain-LOD-Meshes

Just tested that, that solved it. I was just using too low a quality setting before. The dark snow "issue" persists, though increasing brightness and contrast and reducing the gamma made it much better, though now grass on the world map looks very yellowish… I'll just keep fiddling with the settings until I find something that works for me.

Edited April 12 by bread-dreams

[sheson]

29 minutes ago, bread-dreams said:

Just tested that, that solved it. I was just using too low a quality setting before. The dark snow "issue" persists, though increasing brightness and contrast and reducing the gamma made it much better, though now grass on the world map looks very yellowish… I'll just keep fiddling with the settings until I find something that works for me.

As I wrote before, the terrain LOD textures for all LOD levels match the full textures in the active cells 1:1 if the brightness, contrast, gamma settings are left alone. The brightness, contrast, gamma affect the end result equally. If you want only a specific texture or two to be different, then you would need to change/replace the full landscape texture temporality. In any case, the differences are because of post processing effects, overlay textures and INI settings applied in the game.

[bread-dreams]

8 minutes ago, sheson said:

As I wrote before, the terrain LOD textures for all LOD levels match the full textures in the active cells 1:1 if the brightness, contrast, gamma settings are left alone. The brightness, contrast, gamma affect the end result equally. If you want only a specific texture or two to be different, then you would need to change/replace the full landscape texture temporality. In any case, the differences are because of post processing effects, overlay textures and INI settings applied in the game.

Well, for curiosity's sake, why are the raw dds files for vanilla and xLODGen so different then? Were the vanilla textures not meant to match active cells at all? I generated a LOD32 texture with the same 256x256 size as vanilla without changing brightness/contrast/gamma and comparing with vanilla, it's much darker (attached below). If I strongly prefer the vanilla LOD32 texture could I simply remove xLODGen's generated LOD32 textures and use vanilla while still using the rest of LOD16/8/4 from xLODGen? Thanks for all the help btw.

[z929669]

2 hours ago, bread-dreams said:

Well, for curiosity's sake, why are the raw dds files for vanilla and xLODGen so different then? Were the vanilla textures not meant to match active cells at all? I generated a LOD32 texture with the same 256x256 size as vanilla without changing brightness/contrast/gamma and comparing with vanilla, it's much darker (attached below). If I strongly prefer the vanilla LOD32 texture could I simply remove xLODGen's generated LOD32 textures and use vanilla while still using the rest of LOD16/8/4 from xLODGen? Thanks for all the help btw.

All I see here is a brightness difference. Are you certain that you didn't change LOD32 gamma?

[sheson]

9 hours ago, bread-dreams said:

Well, for curiosity's sake, why are the raw dds files for vanilla and xLODGen so different then? Were the vanilla textures not meant to match active cells at all? I generated a LOD32 texture with the same 256x256 size as vanilla without changing brightness/contrast/gamma and comparing with vanilla, it's much darker (attached below). If I strongly prefer the vanilla LOD32 texture could I simply remove xLODGen's generated LOD32 textures and use vanilla while still using the rest of LOD16/8/4 from xLODGen? Thanks for all the help btw.

You would have to ask Bethesda why they made terrain LOD textures that do not match the actual full textures and do not match the terrain in the active cells or why they created shaders and post processing effects that make the difference worse or the map have a greenish etc.

Terrain LOD meshes and textures can be added or removed any time. They have standard file names and thus are interchangeable. Note that terrain LOD Level 32 is used in the game in far distance and the map and a brightness difference will be visible unless you increase the terrain LOD multiplier INI setting at the cost of performance.

https://dyndolod.info/Help/Mod-Configuration-Menu#Settings, https://dyndolod.info/Help/Terrain-LOD-and-Water-LOD, https://dyndolod.info/Help/xLODGen#Terrain-LOD-Textures-Brightness, https://dyndolod.info/Mods/Maps-And-Map-Mods