DynDOLOD 3.00 Alpha 169

[Derek317]

Background:
So I had an issue generating a dyndolod output. Once dyndolod finished its LODGENx64 program and start 'updating the height map from object lod meshes'. It will consume all of my ram and my OS will not operate well, for example : input delay, not responding to what I am doing( of course since all my ram was consumed),etc. Later on , the Windows system will be BSOD with an error message: memory_management(happened mostly) or critical process died(only happened once), or if I am lucky enough the program(dyndolodx64) will shut down automatically and back to normal. In order to resolve this problem, I find out that I have to disable ISLC(Intelligent standby list cleaner v1.0.2.9) before I run the dyndolodx64 program. 

My Problem:
Even after I have disabled ISLC, dyndolod still consumes all my ram during the 'updating the height map from object lod meshes' process. And I wonder why the program consumes such an amount of ram and nearly causes BSOD. I've run the win10 diagnostic tool to diagnose my ram and the result is my rams have no problem. 

My PC Spec:
Ryzen 7 5800X w/ PBO
RTX 4070
G*skill 16GB RAM *2
1 SATA3 SSD(BX500)
1 M.2 KINGSTON A2000 
PSU 650W

Dyndolod log: https://ufile.io/1v5bu1dx

 

[sheson]

43 minutes ago, Derek317 said:

Background:
So I had an issue generating a dyndolod output. Once dyndolod finished its LODGENx64 program and start 'updating the height map from object lod meshes'. It will consume all of my ram and my OS will not operate well, for example : input delay, not responding to what I am doing( of course since all my ram was consumed),etc. Later on , the Windows system will be BSOD with an error message: memory_management(happened mostly) or critical process died(only happened once), or if I am lucky enough the program(dyndolodx64) will shut down automatically and back to normal. In order to resolve this problem, I find out that I have to disable ISLC(Intelligent standby list cleaner v1.0.2.9) before I run the dyndolodx64 program. 

My Problem:
Even after I have disabled ISLC, dyndolod still consumes all my ram during the 'updating the height map from object lod meshes' process. And I wonder why the program consumes such an amount of ram and nearly causes BSOD. I've run the win10 diagnostic tool to diagnose my ram and the result is my rams have no problem. 

My PC Spec:
Ryzen 7 5800X w/ PBO
RTX 4070
G*skill 16GB RAM *2
1 SATA3 SSD(BX500)
1 M.2 KINGSTON A2000 
PSU 650W

Dyndolod log: https://ufile.io/1v5bu1dx

 

Moved to the DynDOLOD 3 Alpha thread. Read the first post which debug log to also upload. https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs
It seems to be missing from the archive.

The DynDOLOD log that was uploaded shows the generation completed successfully.

See answers for https://dyndolod.info/FAQ#High-memory-usage-Out-of-memory and see https://dyndolod.info/Help/Occlusion-Data#Out-of-Memory-while-Generating-Occlusion-Data, which explains why reading several object LOD BTO at the same time might use a lot of memory and how to limit the max number of threads to limit memory usage.

BSOD is a hardware, BIOS settings, OS, drivers etc. problem. The OS is what controls how much memory a program is allowed to use. Typically leave virtual memory settings in Windows the default, so it is automatic.

The screenshot seems to show about 6GB RAM usage which is less than half of the 16GB total available. The peak working set column might have recorded more.
Do not take pictures of a monitor, make a proper screenshot instead. https://www.cnet.com/tech/services-and-software/how-to-easily-take-screenshots-in-windows-10-and-11

[FinalCraneFall]

On 8/1/2023 at 1:20 AM, sheson said:

The bridge in second screenshot seems to consist of several different objects. There are hundreds of objects in each cell that can be clicked with console open that will show a form ID and lots of additional information with more informative console.

If you can not click any objects to get their form ID then troubleshoot and fix that problem first.

In order to be able to troubleshoot problems with specific mod(s), records or models they need to be known.

Provide the cell coordinates.

Also upload ..\DynDOLOD\Edit Scripts\Export\LODGen_SSE_Export_Tamriel.txt

Did you generate any terrain LOD meshes with xLODGen (what settings for LOD Level 32?) or are using a terrain LOD mod?

It is a big bridge with only clutters on it. The clutters can be selected but are not related to the bridge's body. 

Yes I have xLOD.
 
https://ufile.io/diit2e03

[Derek317]

Thank you for your reply. 

Concerning the debug log , unfortunately, I can't find any file that is called debug log in my dyndolod file. I used wiztree to search my drive, and there is no file that's called DynDolod_SSE_Debug_log.txt . 

As I have mentioned before, during the "updating the height map from object lod meshes" process my system is not responsive, so that I can't have a proper screenshot for you. Besides that, I actually have 32GB ram in my system. And you are right, it records more when it is in peak working set , I remember 27GB is the peak.

And I know that the occulsion data and reading several objects LOD simultaneously use lots of memory. However,  I used dyndolod a year ago with similar mods and exactly the same setting, which didn't consume such an amount of my ram and at least my OS is responsive instead of a frozen screen. I am just wondering if is there anything that I do wrong this time which cause such consequences. 

Here is my modlist: https://pastebin.com/BgvWc4dr

[sheson]

1 hour ago, FinalCraneFall said:

It is a big bridge with only clutters on it. The clutters can be selected but are not related to the bridge's body. 

Yes I have xLOD.
 
https://ufile.io/diit2e03

Post screenshots of the objects that can be selected. Preferably the ones that have LOD.
Report the cell coordinates for the bridges .

Upload ..\xLODGen\SEELODGen_log.txt from the terrain LOD generation.

[sheson]

1 hour ago, Derek317 said:

Thank you for your reply. 

Concerning the debug log , unfortunately, I can't find any file that is called debug log in my dyndolod file. I used wiztree to search my drive, and there is no file that's called DynDolod_SSE_Debug_log.txt . 

As I have mentioned before, during the "updating the height map from object lod meshes" process my system is not responsive, so that I can't have a proper screenshot for you. Besides that, I actually have 32GB ram in my system. And you are right, it records more when it is in peak working set , I remember 27GB is the peak.

And I know that the occulsion data and reading several objects LOD simultaneously use lots of memory. However,  I used dyndolod a year ago with similar mods and exactly the same setting, which didn't consume such an amount of my ram and at least my OS is responsive instead of a frozen screen. I am just wondering if is there anything that I do wrong this time which cause such consequences. 

Here is my modlist: https://pastebin.com/BgvWc4dr

The debug log is saved after the the normal log into the same log folder. Simply letting the tool shutdown (exit, only one lick on X etc.) on its own should let it save the debug log.

Nothing really changed in the principle code of occlusion generation reading BTO files in xLODGen terrain LOD beta in a long time.
BTO files are read in parallel loop. The OS decides if it gives requested memory to programs. The OS decides how many threads to run on which core.
If reading BTOs is consuming too much memory, just lower the max number of threads as suggested or simply endure the 1 minute that peak seems to take for things complete successfully.

See https://dyndolod.info/Changelog to learn what changed.
The order in which BTO files are read in parallel is not guaranteed to be the same each session.
The reason something is different is probably because of larger BTO files. For example, Worldspace Transition Tweaks adds Solstheim to Tamriel, way too complex 3D tree LOD assets, full models used for LOD etc.

https://dyndolod.info/Messages/Textures-Do-Not-Match
While they may not be the reason, those message indicate that outdated LOD assets are used. Assets from DynDOLOD Resources or the MM DynDOLOD V3.0 LOD Pack are being overwritten by another mod. This can affect visuals negatively.

[vektor9999]

Hey, there

I have a z-fighting issue using dyndolod (Alpha 128) and seasons of skyrim.

Seems that the lod transition to full model has issue (only during winter)

It's really noticable when moving.

If i save and reload, problem is solved until i come back then later ofc, it does not happen for all the same trees models, i can have a same tree model that loads correctly just next to this one.

Here is my log (DynDOLOD_SSE_Debug_log)

https://paste.ee/p/RbypC

I have no bugreport.txt file

I'm thanking you in advance for your help

[sheson]

14 minutes ago, vektor9999 said:

Hey, there

I have a z-fighting issue using dyndolod (Alpha 128) and seasons of skyrim.

Seems that the lod transition to full model has issue (only during winter)

It's really noticable when moving.

If i save and reload, problem is solved until i come back then later ofc, it does not happen for all the same trees models, i can have a same tree model that loads correctly just next to this one.

Here is my log (DynDOLOD_SSE_Debug_log)

https://paste.ee/p/RbypC

I have no bugreport.txt file

I'm thanking you in advance for your help

Unfortunately the debug log only shows DynDOLOD being started and closed. The debug log gets replaced with every session.
Also upload the normal log as explained in the first post and https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs

See https://dyndolod.info/Official-DynDOLOD-Support-Forum#In-Game-Screenshots how to make screenshot with additional information.
Use "tll" in console to see if tree/object LOD is involved as explained.
After clicking an object with console open, type "disable" in the console to see if there is a duplicate model remaining.

See answers for https://dyndolod.info/FAQ#Object-LOD-shows-in-active-exterior-cells

[vektor9999]

33 minutes ago, sheson said:

Unfortunately the debug log only shows DynDOLOD being started and closed. The debug log gets replaced with every session.
Also upload the normal log as explained in the first post and https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs

See https://dyndolod.info/Official-DynDOLOD-Support-Forum#In-Game-Screenshots how to make screenshot with additional information.
Use tll to see if tree/object LOD is involved as explained.
After clicking an object in console, use type "disable" in the console to see there is a duplicate model remaining.

See answers for https://dyndolod.info/FAQ#Object-LOD-shows-in-active-exterior-cells

Thanks for your quick answer,

here is my DynDOLOD_SSE_log.txt

https://paste.ee/p/V6Jwe

I have no other debug log than the one i sent you

Here is a small video to illustrate

When i click tll, the tree is loaded correctly

when i console click on the object and type disable only this remain

Edited August 4, 2023 by vektor9999

[sheson]

21 minutes ago, vektor9999 said:

Thanks for your quick answer,

here is my DynDOLOD_SSE_log.txt

https://paste.ee/p/V6Jwe

I have no other debug log than the one i sent you

When i click tll, the tree is loaded correctly

when i console click on the object and type disable only this remain

You would need to do a new generation and then make sure that the debug is not being replaced later. Keep in mind for the next time.

This looks like stuck object LOD. See answers for https://dyndolod.info/FAQ#Object-LOD-shows-in-active-exterior-cells

I am assuming ultra LOD was generated and probably object LOD Level 32 is used for the map but the object LOD Level 32 files do not exist for that season.

[vektor9999]

1 hour ago, sheson said:

You would need to do a new generation and then make sure that the debug is not being replaced later. Keep in mind for the next time.

This looks like stuck object LOD. See answers for https://dyndolod.info/FAQ#Object-LOD-shows-in-active-exterior-cells

I am assuming ultra LOD was generated and probably object LOD Level 32 is used for the map but the object LOD Level 32 files do not exist for that season.

Again thanks for your quick answer,

tho following the documentation you posted my map menu is this

[MapMenu]
uLockedObjectMapLOD=16
uLockedTerrainLOD=32

i have trees on my map and both ultra and dynamic lod are checked and i also installed the mod LOD Unloading Bug Fix which does not fix the issue as well

I have no more clue sadly

Edited August 4, 2023 by vektor9999

[sheson]

9 minutes ago, vektor9999 said:

Again thanks for your quick answer,

tho following the documentation you posted my map menu is this

[MapMenu]
uLockedObjectMapLOD=16
uLockedTerrainLOD=32

i have trees on my map and both ultra and dynamic lod are checked and i also installed the mod LOD Unloading Bug Fix which does not fix the issue as well

I have no more clue sadly

The dynamic LOD already addresses LOD not unloading after fast travel, so the LOD not unloading bug fix is not needed anyways.

Make sure to check the INIs that are actually used in the game, in the MO2 profile folder, use MO2 menu Tools / Tool Plugins / INI Editor.
Also check SkyrimCustom.INI and every INI found in data folder with the game plugins.

If the INIs are not default, make a backup and test with default INIs.

[vektor9999]

4 minutes ago, sheson said:

The dynamic LOD already addresses LOD not unloading after fast travel, so the LOD not unloading bug fix is not needed anyways.

Make sure to check the INIs that are actually used in the game, in the MO2 profile folder, use MO2 menu Tools / Tool Plugins / INI Editor.
Also check SkyrimCustom.INI and every INI found in data folder with the game plugins.

If the INIs are not default, make a backup and test with default INIs.

The posted settings are from my mo2 ini files, i checked using the ini editor.

I also checked that i don't have an overrided value elsewhere for maps but i don't

I'm gonna regen again and come back to you with more info.

Thanks a lot Sheson

Edited August 4, 2023 by vektor9999

[megablackk1d]

All these years I have been avoiding dyndolod because it looked way to hard to install, but I finally decide to try it. No matter what I do the process hangs and gets stuck close to 3 minutes in. This is the most frustrating mod I have ever had to deal with. Here is the log:

 

[vektor9999]

2 hours ago, sheson said:

The dynamic LOD already addresses LOD not unloading after fast travel, so the LOD not unloading bug fix is not needed anyways.

Make sure to check the INIs that are actually used in the game, in the MO2 profile folder, use MO2 menu Tools / Tool Plugins / INI Editor.
Also check SkyrimCustom.INI and every INI found in data folder with the game plugins.

If the INIs are not default, make a backup and test with default INIs.

OK i'm back again,

I double checked map settings in skyrim.ini and also in the custom.ini and the values are correct, so i don't use uLockedObjectMapLOD=32.

I deleted all logs before regen.

I regened lod with ultra, dynamic lod and seasons checked

I only generated lods for tamriel without grass cache so it was faster

DynDOLOD_SSE_log ==>  https://paste.ee/p/8iKr4

DynDOLOD_SSE_Debug_log (more than 300MB so i uploaded the file on my private cdn, fast download no worry) ==> https://nolvus-storage.b-cdn.net/DynDOLOD_SSE_Debug_log.txt

Compressed file here if you have issue ==> https://nolvus-storage.b-cdn.net/DynDOLOD_SSE_Debug_log.7z

 

Thanks a lot for your help

Edited August 4, 2023 by vektor9999