DynDOLOD 3.00 Alpha 182

[sheson]

6 hours ago, asebw said:

My bad for missing that part. Compressed debug uploaded.

There is a icewall01lod.dds and icewall01lod_n.dds file in the output folder. Don't currently have a proper viewer program for the files installed atm, so unsure of their state.

There is a process for Taking your first born, Texture conversion command-line tool (texconvx64.exe), and a console window host running in task manager. However, once texgen hits that icewall01lod file, the cpu usage drops to 0%, but retains hold of a fair bit of memory.

In the years I have been using TexGenDynDOLOD, UAC and anti-virus have never given me an issue. I did do a check earlier while troubleshooting for completeness, and unless there's something hidden/obscure I missed, there doesn't appear to be anything blocking either texgen or textconvx64.exe.

TexGen_SSE_Debug_log.7z 597.64 kB · 0 downloads

So TexGen is waiting for Texconvx64.exe to finish.

Antivir never changing in all those years? New graphics driver?

Check if the folder C:\Users\Alexi-PC\AppData\Local\Temp\TexGen_SSE\ still exists from former sessions. Delete it.
Then run TexGen again and check if there are any files left in the folder when TexGen reaches the end and waits for Texconvx64.exe to finish.

See what happens when you end Texconvx64.exe process in task manager.

Use https://github.com/Microsoft/DirectXTex/wiki/Texdiag, a sister program of texconv, to determine the compression format of textures to see which ones are converted to BC7 and which one are still uncompressed.

[sheson]

3 hours ago, GuyWithOneEye said:

OK, so first of all here's the debug log (it seems awfully big but it starts with the latest session so I don't know if I can shorten it) + a couple more screenshots looking at a how a specific tree changes with each season. - https://ufile.io/lxu7y6z6

You can see how everything is totally normal in AUTUMN, but SPRING and SUMMER have a normal looking pine (it should be a dead pine) and WINTER has some type of aspen (should be a snowy dead pine).

And I checked all of the trees that I first linked, I compared the .dds files with the .nif files and they all seem to be matching, as they should right? I'm not seeing anything wrong there.

The only one that is different is TreeDeadShrubSnow pictured in 4b of my first post, it says this in the tree report:

TreeDeadShrubSnow [TREE:000A731D] Meshes\landscape\plants\deadshrub01snow.nif [CRC32:C45663F1] using textures\plants\deadshrub01snow.dds, textures\plants\deadshrub01_n.dds
    Vanilla tree, Billboard DDS/TXT not found textures\terrain\lodgen\skyrim.esm\deadshrub01snow_000a731d.dds, 3D LOD model not found deadshrub01snow_C45663F1

So no billboard found there, and I was only able to find that .nif in XEdit, not in MO2. I think it's in a .bsa in the Skyrim folder? It also looks slightly different than what I'm seeing in game but not much, maybe it's just the picture. Not sure what to make of that.

I mean there must be something massively messed up with the season swapping, again I have no idea how that all works but it seems like the billboards are all generated fine and matching with the meshes/textures but when it comes to the swapping everything during the season change it's grabbing completely unrelated billboards, but not in AUTUMN.

Doublecheck that none of the files from DynDOLOD output are being overwritten by other mods or the MO2 Overwrite folder. The tree LOD being stuck in active cells typically only happens if tree LOD meshes data does not match the current order references/plugins. You are sure the load order did not change after generating LOD?

Regarding TreeDeadShrubSnow, there should quite a few shrubs, bushes etc. that do not have any billboards and consequently no LOD.

[GuyWithOneEye]

Ok, hold on. I decided to look around in Xdit, opening files from Skyrim - Meshes1.bsa. I've attached pictures from NifSkope when using the asset viewer thing from XEdit (I'm running it as an .exe from MO2 just fyi) to open pine variants of "lod_flat" files which appear to be vanilla tree lods. They all messed up and misaligned and...

 

treepineforest01 is a straight up aspen tree.

02 has some aspen assets.

03 and 04 have parts of snowy variant assets.

treepineforestdeadsnow02 appears to visually empty but still has that 3d box around it. Many of the lod_flats are empty like that.

treepineforestdeadsnow04 is a mess.

They still don't really seem to correspond to, in terms of file names and what they look like in game, what I see in game. treepineforestdeadsnow02 has a short aspen lod in game, but it looks like an empty tall pine tree here.

Ok so then I used Bethesda Archive Extractor to open Skyrim - Meshes1.bsa directly NOT through XEdit, and looked at all of those lod_flat files, they all look like the normal vanilla billboards.

So looking at them though XEdit they're messed up, but not when looking at them directly at the .nifs extracted from the .bsa. But even if those lods are completely messed up like this, why are the vanilla lods not getting overwritten by DynDOLOD? And how does this relate to the season switching, and everything looking normal in Autumn? How is it that I only see the messed up vanilla lods when looking through XEdit?

 

And regarding your last comment, I've tried this probably a dozen times. I have been very careful to not touch anything, I would drop the output into MO2, install, activate, and load the game up. I also have tried a couple times using LOOT afterwards just to test that.

When I double click DynDOLOD_Output it shows no conflicts, and there's no lightning bolt icon either. I assume that means it's good? There is some stuff in the overwrite folder but that's fine right if it shows no files being overwritten?

[sheson]

8 minutes ago, GuyWithOneEye said:

Ok, hold on. I decided to look around in Xdit, opening files from Skyrim - Meshes1.bsa. I've attached pictures from NifSkope when using the asset viewer thing from XEdit (I'm running it as an .exe from MO2 just fyi) to open pine variants of "lod_flat" files which appear to be vanilla tree lods. They all messed up and misaligned and...

 

treepineforest01 is a straight up aspen tree.

02 has some aspen assets.

03 and 04 have parts of snowy variant assets.

treepineforestdeadsnow02 appears to visually empty but still has that 3d box around it. Many of the lod_flats are empty like that.

treepineforestdeadsnow04 is a mess.

They still don't really seem to correspond to, in terms of file names and what they look like in game, what I see in game. treepineforestdeadsnow02 has a short aspen lod in game, but it looks like an empty tall pine tree here.

Ok so then I used Bethesda Archive Extractor to open Skyrim - Meshes1.bsa directly NOT through XEdit, and looked at all of those lod_flat files, they all look like the normal vanilla billboards.

So looking at them though XEdit they're messed up, but not when looking at them directly at the .nifs extracted from the .bsa. But even if those lods are completely messed up like this, why are the vanilla lods not getting overwritten by DynDOLOD? And how does this relate to the season switching, and everything looking normal in Autumn? How is it that I only see the messed up vanilla lods when looking through XEdit?

 

And regarding your last comment, I've tried this probably a dozen times. I have been very careful to not touch anything, I would drop the output into MO2, install, activate, and load the game up. I also have tried a couple times using LOOT afterwards just to test that.

When I double click DynDOLOD_Output it shows no conflicts, and there's no lightning bolt icon either. I assume that means it's good? There is some stuff in the overwrite folder but that's fine right if it shows no files being overwritten?

lod_flat.nif files in the data folder are used by CK for tree LOD generation and have nothing to do with trees in the game, tree LOD generation with xLODGen/DynDOLOD or seasons.

It is unclear what "some stuff" means. Files in the MO2 Overwrite folder should be moved to their respective mod folders.
Use MO2 right window data tab to check which mods files come from and that they are all from the same LOD generation session. Enable the columns Name, Mod, Date modified for example.
Meshes\Terrain\Tamriel\Trees\*.* all files.
Textures\Terrain\Tamriel\Trees\TamrielTreeLOD.dds
Textures\Terrain\Tamriel\Trees\TamrielTreeLOD.WIN.dds
Textures\Terrain\Tamriel\Trees\TamrielTreeLOD.SPR.dds
Textures\Terrain\Tamriel\Trees\TamrielTreeLOD.SUM.dds
Textures\Terrain\Tamriel\Trees\TamrielTreeLOD.AUT.dds

[Oozaru85]

10 hours ago, sheson said:

HLT = Happy Little Trees. You honestly see a difference between using billboards4 and 3D tree LOD in LOD level 8/16? Seems like a waste of resources and FPS.

You may have issues because OS, OS settings, drivers other background processes etc. changed.

When was the last time you generated occlusion. What version?

Is memory also going to 99% with DynDOLOD and OcclusionMaxThreadsObjectLOD=1?

1. I was going to try lower settings there next, with lod8=billboard4 and lod16=billboard4 or billboard1, but I'm not really into wating 3 hours again for occlusion data to be generated just for this. Unless I can rerun the lod generation and don't have to re-generate occlusion for that small change of tree lod quality?

 

2. Maybe, I was going to order a new SSD and make a clean new install of Win10 anyway. Hopefully this is going to fix it. But wouldn't the OS slow down all of Dyndolod, also the lod generation and not just occlusion? 

 

3. The last time when it worked fast and flawlessly was few weeks ago. I was still on Dyndolod 3 alpha 106. BUT it was also on this version few days ago when this issue first happened, thats the reason why I updated to alpha118. But this didn't fix it sadly.

 

4. Yes, been running it on OcclusionMaxThreadsObjectLOD=1 all the time now, and nothing changed.

Edited March 5, 2023 by Oozaru85

[GuyWithOneEye]

Ok if I understood correctly, I found those files and in Meshes\Terrain\Tamriel\Trees\ I see a ton of files. They were all created the same date and at most a difference of 1 minute, so I think they're from the same session.

 

And in the MO2 overwrite folder there's five folders, one of them IS a Seasons folder with the MainFormSwap_WIN.ini, and there's a SKSE folder with po3_Seasons of Skyrim.ini in it. I'm guessing I should move all of those then.

I definitely never put anything there on purpose, and I have read elsewhere that the MainFormSwap file from Seasons of Skyrim generates in Overwrite. I'm curious, why would that stuff generate there if you're just supposed to move it?

Do you think those stray .inis would be enough to cause these issues?

 

Yeah I think that did something possibly. Still some issues, but I think I'm gonna go back an older save before I started messing with all this, make a new clean save and try another run here.

Thank you very much Sheson.

[sheson]

2 hours ago, Oozaru85 said:

1. I was going to try lower settings there next, with lod8=billboard4 and lod16=billboard4 or billboard1, but I'm not really into wating 3 hours again for occlusion data to be generated just for this. Unless I can rerun the lod generation and don't have to re-generate occlusion for that small change of tree lod quality?

 

2. Maybe, I was going to order a new SSD and make a clean new install of Win10 anyway. Hopefully this is going to fix it. But wouldn't the OS slow down all of Dyndolod, also the lod generation and not just occlusion? 

 

3. The last time when it worked fast and flawlessly was few weeks ago. I was still on Dyndolod 3 alpha 106. BUT it was also on this version few days ago when this issue first happened, thats the reason why I updated to alpha118. But this didn't fix it sadly.

 

4. Yes, been running it on OcclusionMaxThreadsObjectLOD=1 all the time now, and nothing changed.

You do not need to update Occlusion.esp in this case. Hide it. Generate LOD. Unhide it.

When the program reads large BTOs, especially in parallel, it needs a lot of memory for that particular process. OcclusionMaxThreadsObjectLOD=1 should limit it, though. You might want to try with OcclusionMaxThreadsObjectLOD=0 but it may end up being slower overall.
Let the OS handle virtual memory settings. Close unneeded (background) processes.

[sheson]

2 hours ago, GuyWithOneEye said:

Ok if I understood correctly, I found those files and in Meshes\Terrain\Tamriel\Trees\ I see a ton of files. They were all created the same date and at most a difference of 1 minute, so I think they're from the same session.

 

And in the MO2 overwrite folder there's five folders, one of them IS a Seasons folder with the MainFormSwap_WIN.ini, and there's a SKSE folder with po3_Seasons of Skyrim.ini in it. I'm guessing I should move all of those then.

I definitely never put anything there on purpose, and I have read elsewhere that the MainFormSwap file from Seasons of Skyrim generates in Overwrite. I'm curious, why would that stuff generate there if you're just supposed to move it?

Do you think those stray .inis would be enough to cause these issues?

 

Yeah I think that did something possibly. Still some issues, but I think I'm gonna go back an older save before I started messing with all this, make a new clean save and try another run here.

Thank you very much Sheson.

And all the files in Meshes\Terrain\Tamriel\Trees\ and all the files in Textures\Terrain\Tamriel\Trees\ have about the same timestamp and they are all from the mod, the DynDOLOD Output that was generated for that load order?

If the swap INIs (or their overwrite order) changed after LOD generation, then LOD needs to be updated as well.

MO2 puts new files written to the game data folder into its Overwrite folder. Existing files are being replaced in their respective mod folder. Refer to the manual or wiki of MO2  to learn how it works.

[GuyWithOneEye]

Ok I think I figured out where the inis should go, so there were 3 concerning files in overwrite. In the data tab of MO2, I've moved the files such that I'm seeing MainFormSwap_WIN.ini and Serialization.ini (both were in a folder called Seasons in the overwrite) are now in the "seasons" folder and it says for mod "Seasons of Skyrim SKSE" and po3_SeasonsOfSkyrim.ini (which was in SKSE>Plugins in the overwrite) is now in SKSE > Plugins and it also says for mod Seasons of Skyrim SKSE. Does this sound correct? And I'll need to rerun the generation again now right?

[Oozaru85]

4 hours ago, sheson said:

You do not need to update Occlusion.esp in this case. Hide it. Generate LOD. Unhide it.

When the program reads large BTOs, especially in parallel, it needs a lot of memory for that particular process. OcclusionMaxThreadsObjectLOD=1 should limit it, though. You might want to try with OcclusionMaxThreadsObjectLOD=0 but it may end up being slower overall.
Let the OS handle virtual memory settings. Close unneeded (background) processes.

Omg, yes! Changing my virtual memory settings actually fixed it! Works smoothly and fast again, now. Thanks!

[sheson]

2 hours ago, GuyWithOneEye said:

Ok I think I figured out where the inis should go, so there were 3 concerning files in overwrite. In the data tab of MO2, I've moved the files such that I'm seeing MainFormSwap_WIN.ini and Serialization.ini (both were in a folder called Seasons in the overwrite) are now in the "seasons" folder and it says for mod "Seasons of Skyrim SKSE" and po3_SeasonsOfSkyrim.ini (which was in SKSE>Plugins in the overwrite) is now in SKSE > Plugins and it also says for mod Seasons of Skyrim SKSE. Does this sound correct? And I'll need to rerun the generation again now right?

MainFormSwap_WIN.ini and Serialization.ini are typically generated by po3_SeasonsOfSkyrim.dll, so it makes sense to keep them with that mod.

[z929669]

In LODGen_SSE_<worldspace>_log.txt, I see:

Output: C:\Modding\Tools\DynDOLOD\DynDOLOD_Output\Meshes\Terrain\Tamriel\Objects\
Using UV Atlas: C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_ObjectAtlasMap_Tamriel.txt
Using Flat Textures: C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_FlatTextures.txt
Using Alt Textures: C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_AltTextures_Tamriel.txt
Generating object LOD meshes for worldspace Tamriel
Reading C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_Terrain_Tamriel.bin
Threads: 32
Concs: 8

Isn't the last line supposed to read "Cores" rather than "Concs"? I'm doing some experiments with LODGenThreadSplit settings and this threw me off a bit.

[sheson]

33 minutes ago, z929669 said:

In LODGen_SSE_<worldspace>_log.txt, I see:

Output: C:\Modding\Tools\DynDOLOD\DynDOLOD_Output\Meshes\Terrain\Tamriel\Objects\
Using UV Atlas: C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_ObjectAtlasMap_Tamriel.txt
Using Flat Textures: C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_FlatTextures.txt
Using Alt Textures: C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_AltTextures_Tamriel.txt
Generating object LOD meshes for worldspace Tamriel
Reading C:\Modding\Tools\DynDOLOD\Edit Scripts\Export\LODGen_SSE_Terrain_Tamriel.bin
Threads: 32
Concs: 8

Isn't the last line supposed to read "Cores" rather than "Concs"? I'm doing some experiments with LODGenThreadSplit settings and this threw me off a bit.

It is a short for "Concurrents" and used together with the threads value to set the initial and max count of a SemaphoreSlimDynamic.

[asebw]

13 hours ago, sheson said:

So TexGen is waiting for Texconvx64.exe to finish.

Antivir never changing in all those years? New graphics driver?

Check if the folder C:\Users\Alexi-PC\AppData\Local\Temp\TexGen_SSE\ still exists from former sessions. Delete it.
Then run TexGen again and check if there are any files left in the folder when TexGen reaches the end and waits for Texconvx64.exe to finish.

See what happens when you end Texconvx64.exe process in task manager.

Use https://github.com/Microsoft/DirectXTex/wiki/Texdiag, a sister program of texconv, to determine the compression format of textures to see which ones are converted to BC7 and which one are still uncompressed.

The plot thickens...

Other than updates, anti virus has not changed. I did upgrade from a geforce 1070 to a 3060 ti. Did not blow away the old graphics drivers though, as they autimatically updated to the 3060.

the TexGen_SSE folder deletes itself whenever texgen is shut down. I took a peek in the file when texgen gets stuck, I checked the file generated in TexGen_SSE, copied its contents, and then killed texconvx64.exe. Texgen generated the following error message. Both file contents and error message as is below.

Spoiler

#"D:\game installs\skyrim mod storage\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -y -sepalpha -aw 256 -w 212 -h 256 -m 1 -f BC7_UNORM -bc q -o "D:\Dyndolod output\textures\terrain\lodgen\nature of the wild lands.esp"
D:\Dyndolod output\textures\terrain\lodgen\nature of the wild lands.esp\fagustortuosagremsheim_dead01_004643ad_2_n.dds
D:\Dyndolod output\textures\terrain\lodgen\nature of the wild lands.esp\fagustortuosagremsheim_dead01_004643ad_2.dds

 

EDIT: it appears I did not save the correct error message to go with this file. Apolgies.

 

nature of the wild lands also has 3D lods designed for DynDOLOD 3. I reinstalled the mod without the LODs, and still got the same issue, but with different errors:

Spoiler

#"D:\game installs\skyrim mod storage\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -y -sepalpha -aw 256 -w 144 -h 256 -m 1 -f BC7_UNORM -bc q -o "D:\Dyndolod output\textures\terrain\lodgen\s3dlandscapes nextgenerationforests.esp"
D:\Dyndolod output\textures\terrain\lodgen\s3dlandscapes nextgenerationforests.esp\s3dlandscapes_treesnowqueen02_0011c4b2_2_n.dds
D:\Dyndolod output\textures\terrain\lodgen\s3dlandscapes nextgenerationforests.esp\s3dlandscapes_treesnowqueen02_0011c4b2_2.dds
D:\Dyndolod output\textures\terrain\lodgen\s3dlandscapes nextgenerationforests.esp\s3dlandscapes_treesilkyoak02_0020ffce_1.dds
D:\Dyndolod output\textures\terrain\lodgen\s3dlandscapes nextgenerationforests.esp\s3dlandscapes_treesilkyoak02_0020ffce_1_n.dds

[Main Instruction]
Texconv error 1.

[Content]
"D:\game installs\skyrim mod storage\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -y -sepalpha -aw 256 -w 144 -h 256 -m 1 -f BC7_UNORM -bc q -o "D:\Dyndolod output\textures\terrain\lodgen\s3dlandscapes nextgenerationforests.esp" -flist "C:\Users\Alexi-PC\AppData\Local\Temp\TexGen_SSE\Texconv_143_4_C007E2D58C0041F4946B60A659E83397.txt"

 

 

I did a check of the dds file texgen keeps hanging on, and the info and analyze are as follows:

Spoiler

texdiag info icewall01lod.dds

icewall01lod.dds
        width = 256
       height = 128
        depth = 1
    mipLevels = 1
    arraySize = 1
       format = R8G8B8A8_UNORM
    dimension = 2D
   alpha mode = Opaque
       images = 1
   pixel size = 128 (KB)

texdiag analyze icewall01lod.dds

icewall01lod.dds
Results by item (  1) and mip (  1)

          Minimum - (0.227451 0.317647 0.352941 1.000000)
          Average - (0.429984 0.558623 0.610880 1.000000)
          Maximum - (0.686275 0.835294 0.874510 1.000000)
         Variance - (167.980606 201.697525 206.240845 0.000000)
          Std Dev - (12.960733 14.202025 14.361088 0.000000)
        Luminance - 0.794902 (maximum)

 

 

Edited March 5, 2023 by asebw

[RainingTacco]

The evlas discussion and LOD's not getting proper shadowing with EVLAS and CG suffering from the same in the lod distance is nothing new, i've written about it months ago, CABAL written about it on enb dev forum also many months ago. I guess many people don't mind it.