DynDOLOD 3.00 Alpha 182

[sheson]

12 minutes ago, Thaumatarge said:

Here are all logs that are in the logs folder.

Logs.rar 656.66 kB · 0 downloads

I would need the debug log from a session that had the problem.

[Thaumatarge]

11 minutes ago, sheson said:

I would need the debug log from a session that had the problem.

Right, and since it has happened to me every time I tried to generate it, I can just try generating stuff again to get a new log with the same problem.

[Thaumatarge]

https://ufile.io/d4pp7pb8 Here you go, logs from a session that once again almost killed my computer. Here is also a picture of when I decided to manually stop all the Tamriel LODGENs because they crashed explorer.exe and didn't let it restart because there was no memory free to actually do anything. I could run it again and post a picture of the Task Mangager, but I am honestly starting to lose hope of being able to explain my problem.

[sheson]

35 minutes ago, Thaumatarge said:

https://ufile.io/d4pp7pb8 Here you go, logs from a session that once again almost killed my computer. Here is also a picture of when I decided to manually stop all the Tamriel LODGENs because they crashed explorer.exe and didn't let it restart because there was no memory free to actually do anything. I could run it again and post a picture of the Task Mangager, but I am honestly starting to lose hope of being able to explain my problem.

Your first post was about an "OpenGL: Out of Memory". LODGen does not use the GPU and can not report OpenGL errors. I had to request uploading of logs 3 times.

Now that I have a debug log from DynDOLOD all I can see that in the last session there is no OpenGL error and DynDOLOD ran without error to the part where the LODGen processes for Tamriel were ended by the user. It seems the "problem" is that DynDOLOD starts 5  (and then later some more) separate LODGen processes to generate season LODs for Tamriel.

If the PC/OS can not handle the CPU and memory being used by several different programs at the same time, set the number of virtual CPU cores (12 in this case I believe) for LODGenThreadSplit in ..\DynDOLOD\Edit Scripts\DynDOLOD\DynDOLOD_SSE.ini That should reduce the number of threads each LODGen creates which should reduce memory requirements.

Let the OS handle virtual memory settings automatically.

Use print screen key to make screenshots.

[cajeffreys]

Alright, taking a few steps back to get some clarification since I'm going through this mostly for Season's of Skyrim.  xLODGen: should I have generated default files in addition to spring, summer, autumn and winter? If I do need this can I just run xLODGen for it and add the output to my existing output?  xLODGen: Do I need to rerun xLODGen if I change a city texture pack or does that make any difference?  TextGen: Should the files output from xLODGen be activated when running it?

I'm going to leave the textures I copied out where they are while running TextGen from MO2, maybe that will solve my issue.

Edited March 7, 2022 by cajeffreys
more questions

[Thaumatarge]

10 minutes ago, sheson said:

Your first post was about an "OpenGL: Out of Memory". LODGen does not use the GPU and can not report OpenGL errors. I had to request uploading of logs 3 times.

Now that I have a debug log from DynDOLOD all I can see that in the last session there is no OpenGL error and DynDOLOD ran without error to the part where the LODGen processes for Tamriel were ended by the user. It seems the "problem" is that DynDOLOD starts 5  (and then later some more) separate LODGen processes to generate season LODs for Tamriel.

If the PC/OS can not handle the CPU, GPU, memory being used by several different programs at the same time, set the number of virtual CPU cores (12 in this I believe) for LODGenThreadSplit in ..\DynDOLOD\Edit Scripts\DynDOLOD\DynDOLOD_SSE.ini

Let the OS handle virtual memory settings automatically.

Use print screen key to make screenshots.

I will try that. I couldn't use printscreen because I would have no way to actually save a screenshot of it because, again, the processes ate up all of the RAM and it crashed most of my programs, including explorer.exe

The OpenGL: Out Of Memory happened once and I lost the log where it happened, but every time I tried to generate stuff it just froze up my computer and I had to either restart it or manually shut everything down.

[sheson]

32 minutes ago, cajeffreys said:

Alright, taking a few steps back to get some clarification since I'm going through this mostly for Season's of Skyrim.  xLODGen: should I have generated default files in addition to spring, summer, autumn and winter? If I do need this can I just run xLODGen for it and add the output to my existing output?  xLODGen: Do I need to rerun xLODGen if I change a city texture pack or does that make any difference?  TextGen: Should the files output from xLODGen be activated when running it?

I'm going to leave the textures I copied out where they are while running TextGen from MO2, maybe that will solve my issue.

Terrain LOD texture generation uses landscape full textures. The only connection between terrain LOD and DynDOLOD is when LODGen is generating object LOD meshes it uses the terrain LOD meshes to optimize the object LOD meshes. See https://dyndolod.info/Generation-Instructions#Prerequisites

TexGen generates object LOD textures and tree/grass LOD billboards which are used by DynDOLOD for object LOD (grass LOD) and tree LOD generation. https://dyndolod.info/Help/TexGen

Which seasons require terrain LOD generated for depends on the available swap data and po3_SeasonsOfSkyrim.ini settings.
https://stepmodifications.org/forum/topic/16730-xlodgen-with-seasons-of-skyrim-tots/

[jkp993]

Hi sheson,

I'm trying to run dyndolod but my RAM (16GB) and disk (1TB SSD) are at 100%. Dyndolod log when it was running showed this error:

<Warning: Error when loading "meshes\terrain\Tamriel\objects\Tamriel.4.-20.-20.bto": Error reading NIF block 83 BSSubIndexTriShape: Out of memory>

There was no progress for >20 mins. My PC became very slow, like very slow response of cursor etc.

I ended up terminating through task manager. Please help.

Which logs/files do I attach here?

Edited March 7, 2022 by jkp993

[BrotherShamus]

Is there any way for me to get the older versions of Dyndolod 3 and Dyndolod 3 resources so I can test the issue I'm having?  I'm currently using Resources Alpha 22 and Dyndolod Alpha 77.  I've exhausted every troubleshooting step I can find (re-install, repair ,Net Framework 3.5 and 4.8) and the problem persists.  I was able to successfully generate LOD with the previous versions hours before I downloaded the new versions and just want to make sure I'm not having some strange interaction.  My only other option as far as I can tell is to format my hard drive.

[sheson]

29 minutes ago, BrotherShamus said:

Is there any way for me to get the older versions of Dyndolod 3 and Dyndolod 3 resources so I can test the issue I'm having?  I'm currently using Resources Alpha 22 and Dyndolod Alpha 77.  I've exhausted every troubleshooting step I can find (re-install, repair ,Net Framework 3.5 and 4.8) and the problem persists.  I was able to successfully generate LOD with the previous versions hours before I downloaded the new versions and just want to make sure I'm not having some strange interaction.  My only other option as far as I can tell is to format my hard drive.

Since you report an issue with LODGen.exeLODGenx64.exe, see what happen if you use the version included in the latest xLODGen terrain LOD beta.

It would be better to actually provide information about the error so whatever the problem can be fixed.

46 minutes ago, jkp993 said:

Hi sheson,

I'm trying to run dyndolod but my RAM (16GB) and disk (1TB SSD) are at 100%. Dyndolod log when it was running showed this error:

<Warning: Error when loading "meshes\terrain\Tamriel\objects\Tamriel.4.-20.-20.bto": Error reading NIF block 83 BSSubIndexTriShape: Out of memory>

There was no progress for >20 mins. My PC became very slow, like very slow response of cursor etc.

I ended up terminating through task manager. Please help.

Which logs/files do I attach here?

Read the first post which log, debug log and bugreport.txt if it exists to upload.

Read https://dyndolod.info/Help/Occlusion-Data#Out-of-Memory-while-Generating-Occlusion-Data first.

[Thaumatarge]

 

28 minutes ago, sheson said:

Since you report an issue with LODGen.exeLODGenx64.exe, see what happen if you use the version included in the latest xLODGen terrain LOD beta.

It would be better to actually provide information about the error so whatever the problem can be fixed.

Read the first post which log, debug log and bugreport.txt if it exists to upload.

Read https://dyndolod.info/Help/Occlusion-Data#Out-of-Memory-while-Generating-Occlusion-Data first.

I am having pretty much the same issue as jkp993, this cannot be a random issue I am having, I run the program, it works just fine, then while generating Tamriel LODs it gets stuck, and only a moment ago I noticed my computer COMPLETELY FROZE and I had to forcefully restart it, thus I cannot provide any meaningful logs. 

The fix you have given me made it work slightly smoother, with the LODGEN not jumping around when generating Level 4 lods to generating Level 8 lods randomly (the logs I have already provided show that).

But what I have noticed in my testing is the fact that LODGEN allocates a lot of memory and uses none of it, sometimes having bursts of sudden memory usage but in general not using any memory. I used the Performance Tracker program that's in windows to track this, it's more detailed for this stuff than Task Manager. I would provide a printscreen screenshot, but sadly, printscreen doesn't work when my computer is completely frozen. Under the "Confirmed" memory, which summarizes all the allocated memory, all 5 windows allocate more than 3GB of RAM eventually, with some going as high as 9GB of RAM, but only using around 1 to 100 MB of it in random bursts. The allocated memory always exceeds actual memory available.

[jkp993]

36 minutes ago, sheson said:

Since you report an issue with LODGen.exeLODGenx64.exe, see what happen if you use the version included in the latest xLODGen terrain LOD beta.

It would be better to actually provide information about the error so whatever the problem can be fixed.

Read the first post which log, debug log and bugreport.txt if it exists to upload.

Read https://dyndolod.info/Help/Occlusion-Data#Out-of-Memory-while-Generating-Occlusion-Data first.

Thank you for the reply. The dyndolod logs are from a previous run only. In the ini do I also have to change OcclusionMaxThreadsTerrainLOD to 1? Right now it is -1. I'm confused since the file related to the error is in meshes\terrain\Tamriel\objects

[sheson]

25 minutes ago, Thaumatarge said:

 

I am having pretty much the same issue as jkp993, this cannot be a random issue I am having, I run the program, it works just fine, then while generating Tamriel LODs it gets stuck, and only a moment ago I noticed my computer COMPLETELY FROZE and I had to forcefully restart it, thus I cannot provide any meaningful logs. 

The fix you have given me made it work slightly smoother, with the LODGEN not jumping around when generating Level 4 lods to generating Level 8 lods randomly (the logs I have already provided show that).

But what I have noticed in my testing is the fact that LODGEN allocates a lot of memory and uses none of it, sometimes having bursts of sudden memory usage but in general not using any memory. I used the Performance Tracker program that's in windows to track this, it's more detailed for this stuff than Task Manager. I would provide a printscreen screenshot, but sadly, printscreen doesn't work when my computer is completely frozen. Under the "Confirmed" memory, which summarizes all the allocated memory, all 5 windows allocate more than 3GB of RAM eventually, with some going as high as 9GB of RAM, but only using around 1 to 100 MB of it in random bursts. The allocated memory always exceeds actual memory available.

You do not have pretty much the same issue as jkp993 as is clear by the log you uploaded. Since you kill the LODGen processes, the process never gets to read the object LOD bto meshes for the occlusion generation because it is aborted.

Multi-threaded parallel execution of code has no guaranteed execution order. There is no guarantee which LOD quad/level is generated next.

If you find a c# .net memory allocation / garbage collector not working or whatever, i suggest to make a report to its developer.

If the OS, PC can not handle x64 programs using CPU and memory at the same time, then do not generate all seasons all at the same time. You can temporarily halt a LODGen window execution by marking text in the command prompt window, then once another LODGen finished, hit enter key on that halted command prompt window.

Let the OS handle virtual memory settings.

19 minutes ago, jkp993 said:

Thank you for the reply. The dyndolod logs are from a previous run only. In the ini do I also have to change OcclusionMaxThreadsTerrainLOD to 1? Right now it is -1. I'm confused since the file related to the error is in meshes\terrain\Tamriel\objects

As you can see by the folder name and the *.BTO suffix the file is an object LOD mesh. By the time the process reads them it already read the terrain LOD meshes.

You are generating grass LOD, right?

[Thaumatarge]

10 minutes ago, sheson said:

You do not have pretty much the same issue as jkp993 as is clear by the log you uploaded. Since you kill the LODGen processes, the process never gets to read the object LOD bto meshes for the occlusion generation because it is aborted.

Multi-threaded parallel execution of code has no guaranteed execution order. There is no guarantee which LOD quad/level is generated next.

If you find a c# .net memory allocation / garbage collector not working or whatever, i suggest to make a report do its developer.

If you OS, PC can not handle x64 programs using CPU and memory at the same time, then do not generate all seasons all at the same time. You can temporarily halt a LODgen window execution by marking text in the command prompt window, then once a another LODGen finished, hint enter on that command prompt window.

Let the OS handle virtual memory settings.

I will give that a try, but I have been struggling with this for hours and nothing so far has worked. I cannot provide logs if the program cannot either crash to give an error, or finish to give a finished log, because the computer just freezes.

[jkp993]

12 minutes ago, sheson said:

You do not have pretty much the same issue as jkp993 as is clear by the log you uploaded. Since you kill the LODGen processes, the process never gets to read the object LOD bto meshes for the occlusion generation because it is aborted.

Multi-threaded parallel execution of code has no guaranteed execution order. There is no guarantee which LOD quad/level is generated next.

If you find a c# .net memory allocation / garbage collector not working or whatever, i suggest to make a report do its developer.

If you OS, PC can not handle x64 programs using CPU and memory at the same time, then do not generate all seasons all at the same time. You can temporarily halt a LODgen window execution by marking text in the command prompt window, then once a another LODGen finished, hint enter on that command prompt window.

Let the OS handle virtual memory settings.

As you can see by the folder name and the *.BTO suffix the file is an object LOD mesh. By the time the process reads them it already read the terrain LOD meshes.

Thanks a lot! I will try this and update you.