Skyrim TVDT - Occlusion Culling Data

[z929669]

1 hour ago, EazyH said:

Sorry Sheson.  But thanks, I will try looking it up in nifskope. As far as mentioning memory, does it mean RAM? VRAM? That doesn’t make sense to me either, as I have 6 GB of vram and 16 Gb of Ram and have never had this issue in the past, and I honestly haven’t changed out my load order much

You should check your Windows memory management settings in case you have throttled them by setting pagefile size. Make sure you have a pagefile for Windows defined using your fastest drive with plenty of unused space. You can also check background processes and programs that may be running. If you are generating occlusion with a web browser open with a bunch of tabs and other programs running, you will have less memory available.

Right click taskbar and select Taskmanager to see what background programs may be hogging resources and close them (if you know what they are and they are unneeded). Go to the Performance tab to see available resources.

Right click Windows Start Menu icon, select System > Advanced System Settings:

... check that virtual memory is assigned to your fastest drive and Windows is managing:

[EazyH]

3 hours ago, sheson said:

 

Error when loading  "meshes\terrain\Tamriel\objects\Tamriel.4.-8.20.bto": Error reading NIF block 24 BSSubindex TriShape: Out of Memory

Do not post screenshots of text. Post the entire log as text into spoilers or to pastebin.

If the mentioned NIF is not corrupt and loads without error with NifSkope, then the program needs more memory as the message says.

 

 

 

[z929669]

@EazyH

Please DO NOT paste text from unnecessarily huge log files on these forums. Also please DO NOT double post to get your questions answered, especially when they contain unnecessarily huge log file texts in spoilers. This renders the affected topic unusable. This may be related to your memory management issues. Loading this topic after your posts gave me this:

... which is a ridiculous amount of memory devoted to a single browser tab when viewing this topic. Spoiler has been removed from your post and first double post deleted. It took me 10 minutes cumulative wait time to delete the first large post and load, edit, save the second one.

Use pastebin or another sharing service to convey logs, or truncate your logs to the last meaningful content. It's been stated many times all over these forums that DynDOLOD logs append each run. Nobody wants to look at your last 10+ runs but only the last run that cause issues. Either delete all of your DynDOLOD related logs and regenerate LOD to yield a single run log or move logs to another location before running the DynDOLOD-related process in question OR truncate the log you are posting to the last meaningful run.

once your logs are of a manageable size, then you can simply upload the file here or to pastebin OR then place them into a spoiler. If the log upload or text file is > 1 Mb, then you need to use pastebin.

[EazyH]

25 minutes ago, z929669 said:

@EazyH

Please DO NOT paste text from unnecessarily huge log files on these forums. Also please DO NOT double post to get your questions answered, especially when they contain unnecessarily huge log file texts in spoilers. This renders the affected topic unusable. This may be related to your memory management issues. Loading this topic after your posts gave me this:

... which is a ridiculous amount of memory devoted to a single browser tab when viewing this topic. Spoiler has been removed from your post and first double post deleted. It took me 10 minutes cumulative wait time to delete the first large post and load, edit, save the second one.

Use pastebin or another sharing service to convey logs, or truncate your logs to the last meaningful content. It's been stated many times all over these forums that DynDOLOD logs append each run. Nobody wants to look at your last 10+ runs but only the last run that cause issues. Either delete all of your DynDOLOD related logs and regenerate LOD to yield a single run log or move logs to another location before running the DynDOLOD-related process in question OR truncate the log you are posting to the last meaningful run.

once your logs are of a manageable size, then you can simply upload the file here or to pastebin OR then place them into a spoiler. If the log upload or text file is > 1 Mb, then you need to use pastebin.

I’m sorry it caused issues, but 3 or 4 posts back I was literally told post the entire log in the spoiler or use pastebin. I am trying to learn these tools and I do not know all of what he needs to see when his message says to post the entire log. I couldn’t know of that amendment to rule beforehand. Not only that, I did not realize it even went through twice because on my end I got an error window. I thought there was an issue with the site. This was the only reason I tried again to send a message, not to double post. This was the reason I took the screenshot, because I didn’t think that unnecessarily large file was needed and it’s entirety, but then I was told that was wrong, which I understand now for pastebin next time. I only write this up to defend myself so you’re not thinking I’m some impatient jerk, when I couldn’t know otherwise for some of this

 

thank you, I will check out this information, and I know better for next time, but again, none of that was done purposely or with ill intent.

[EazyH]

3 hours ago, z929669 said:

You should check your Windows memory management settings in case you have throttled them by setting pagefile size. Make sure you have a pagefile for Windows defined using your fastest drive with plenty of unused space. You can also check background processes and programs that may be running. If you are generating occlusion with a web browser open with a bunch of tabs and other programs running, you will have less memory available.

Right click taskbar and select Taskmanager to see what background programs may be hogging resources and close them (if you know what they are and they are unneeded). Go to the Performance tab to see available resources.

Right click Windows Start Menu icon, select System > Advanced System Settings:

... check that virtual memory is assigned to your fastest drive and Windows is managing:

Unfortunately I am still getting an “out of memory“ error. On the previous version, I believe I was using v.77.7, I never came across this error. I am trying again with that previous version to see if the error pops up.

 

edit: still waiting for it to finish, it simply said updating height map and I have not gotten an error yet, but it stopped at 15 seconds while the time hadnelapsed to 20 minutes and 41 seconds, with nothing seemingly happening.

it literally JUST started showing the different cells as it normally would, so it does seem to be working now w/ v77.7. I will pop back up when it finishes to see if I get any similar error. The other occlusion ESP did finish generating, with that one error showing. Had  I decided to use that occlusion esp with that one error, would I have any issues? Not sure if it  remains skipped, or if LODgen ever circles back to it if it manages to access more memory

 

Update: so using version 77.7, it generated without any error indicating a memory issue. When looking into my task manager and everything, all looked the same, nothing out of the ordinary. I may have had tabs running once that were a lot, but not for the most recent runs that gave me an error on version 83, I can’t think of any reason for those errors. So not sure if it’s related to the new version or what. Only strange thing with both versions was again, seemingly nothing happening for the first 20 minutes before generation starts, but I can live with that knowing it’s working behind the scenes. I get the same “delay” when rebuilding atlas in DynDOLOD 

 

Edited July 25, 2021 by EazyH

[sheson]

8 hours ago, EazyH said:

Sorry Sheson.  But thanks, I will try looking it up in nifskope. As far as mentioning memory, does it mean RAM? VRAM? That doesn’t make sense to me either, as I have 6 GB of vram and 16 Gb of Ram and have never had this issue in the past, and I honestly haven’t changed out my load order much

It is means system memory. What matters is how much system memory is available and how much memory the program wants to use and if the OS grants the request.

The OS controls system memory and the OS is what tells the program it can not have any more.

If it is always reporting an out of memory error with the exact same BTO even after restarting the PC or closing other programs, then there is an issue with the file and the xEdit nif reader, even if NifSkope is able to load it.

[EazyH]

1 hour ago, sheson said:

It is means system memory. What matters is how much system memory is available and how much memory the program wants to use and if the OS grants the request.

The OS controls system memory and the OS is what tells the program it can not have any more.

If it is always reporting an out of memory error with the exact same BTO even after restarting the PC or closing other programs, then there is an issue with the file and the xEdit nif reader, even if NifSkope is able to load it.

I am not sure what that issue could have been though. As I said before, switching to version 77.7 generated Occlusion with no such error. The only possible issue I can think of is that my terrain LOD was generated w/ 77.7, before I updated xLODGen. After generating Terrain LOD, I went through TexGen and DynDOLOD, when I was informed there was a newer version of xLODGen. We both figured there would be no issue simply updating and running xLODGen for occlusion. But as discussed, I got the error more than once. Switching to 77.7 I didn’t get the error once and it finished on the first run 

[sheson]

46 minutes ago, EazyH said:

I am not sure what that issue could have been though. As I said before, switching to version 77.7 generated Occlusion with no such error. The only possible issue I can think of is that my terrain LOD was generated w/ 77.7, before I updated xLODGen. After generating Terrain LOD, I went through TexGen and DynDOLOD, when I was informed there was a newer version of xLODGen. We both figured there would be no issue simply updating and running xLODGen for occlusion. But as discussed, I got the error more than once. Switching to 77.7 I didn’t get the error once and it finished on the first run 

Whatever happens with long outdated versions is meaningless.

The issue is that the OS told the program can not have anymore memory when it requests some. Error messages are that plain and straight forward. Obviously available memory and memory requirements are different every time. Unless there is an repeatable issue with a specific file. Unfortunately so far you have only mentioned it happened more than one time, but not if it happens with the same BTO or different BTOs specifically. Also I have not yet seen any log of the last generation with the error message.

If the BTOs contain 3D tree LOD or grass LOD they require much more memory to be loaded.

[EazyH]

6 hours ago, sheson said:

Whatever happens with long outdated versions is meaningless.

The issue is that the OS told the program can not have anymore memory when it requests some. Error messages are that plain and straight forward. Obviously available memory and memory requirements are different every time. Unless there is an repeatable issue with a specific file. Unfortunately so far you have only mentioned it happened more than one time, but not if it happens with the same BTO or different BTOs specifically. Also I have not yet seen any log of the last generation with the error message.

If the BTOs contain 3D tree LOD or grass LOD they require much more memory to be loaded.

My apologies, I do not have access to that log anymore, as I tried again and it was gone w/ a new log. I can say that when it happened again, it was with a different bto. I believe I tried three different times and each time it was different, and only happened in the very beginning before eventually “Catching“ and starting generation. With version 77.7, it took over while to get going in the beginning as well, but once generating begins I didn’t get any of those errors. FWIW, I didn’t notice the issue when using Occlusion directly from DynDOLOD either, tho I don’t believe I generated with ultra at that time for all trees

[Jacques_Strappe]

Sheson, as I understand the first post, we are to run SSELODGEN, then TEXGEN64, then DynDoLOD, then re-run SSELODGEN? Or do we just enable the occlusion.esp as the last step?

[sheson]

6 hours ago, Jacques_Strappe said:

Sheson, as I understand the first post, we are to run SSELODGEN, then TEXGEN64, then DynDoLOD, then re-run SSELODGEN? Or do we just enable the occlusion.esp as the last step?

There doesn't exist an Occlusion plugin until it is created per the fourth step. 

[sa547]

After generating and playtesting, there is a consistent CTD with the occlusion plugin (occlusion.esp) at Tamriel worldspace cell -36,14 (FormID 6FE1), all while trying to make a gamesave.

Used DYNDOLOD 3 as directed, with the occlusion option on. Even tried generating the occlusion plugin with xLODGEN and used it. Same CTD.

The only mod making changes to this cell is 3DNPC, specifically the navmesh.

I may have to test again but without the occlusion plugin, see what I could possibly find.

EDIT: may be a problem on 3DNPC's part. CTD happens with a specific quest.

Crash_2021_9_15_16-12-14.txt

Edited September 15, 2021 by sa547

[sheson]

1 hour ago, sa547 said:

After generating and playtesting, there is a consistent CTD with the occlusion plugin (occlusion.esp) at Tamriel worldspace cell -36,14 (FormID 6FE1), all while trying to make a gamesave.

Used DYNDOLOD 3 as directed, with the occlusion option on. Even tried generating the occlusion plugin with xLODGEN and used it. Same CTD.

The only mod making changes to this cell is 3DNPC, specifically the navmesh.

I may have to test again but without the occlusion plugin, see what I could possibly find.

EDIT: may be a problem on 3DNPC's part. CTD happens with a specific quest.

Crash_2021_9_15_16-12-14.txt 53.58 kB · 1 download

If you believe this has anything to do with Occlusion.esp, edit the plugin in xEdit and remove the TVDT data from that cell, replace it with the vanilla data or remove the entire CELL overwrite record from Occlusion.esp to test if any of that changes anything.

Also from the DynDOLOD FAQ:

If there are problems saving in Skyrim Special Edition, install SSE Engine Fixes 4.8 or newer and set SaveGameMaxSize = true in the EngineFixes.toml. Alternatively add/set [SaveGame] uiCompression=1 in Skyrim.INI to change from the default 2 for LZ4 to 1 for zlib compression. 0 means no compression like games are saved in Skyrim.

[sa547]

Ok, found that removing the TVDT in question from occlusion.esp resolved the savegame problem.

[sheson]

19 minutes ago, sa547 said:

Ok, found that removing the TVDT in question from occlusion.esp resolved the savegame problem.

Remove all other records but that one cell from the Occlusion.esp. Test if the problem still exists then.

In case it does, upload the file somewhere for review.

Also check the xLODGen log if there were any error or warning messages while generating occlusion.esp