DynDOLOD 3.00 Alpha 169

[darthbdaman]

I also got a BSOD using alpha 14. I started the dynamic lod generation (I'm using open cities so object was already done), it froze near the start, and then BSOD Clock Watchdog Timeout. Didn't generate any logs, and worked fine when I ran it again.

[sheson]

I also got a BSOD using alpha 14. I started the dynamic lod generation (I'm using open cities so object was already done), it froze near the start, and then BSOD Clock Watchdog Timeout. Didn't generate any logs, and worked fine when I ran it again.

BSOD are caused by hardware, BIOS settings, driver or OS problems. If a simple program compiled with Delphi could bring the OS down, it would be a series flaw in the OS.

Edited December 27, 2020 by sheson

[DarthVitrial]

There's definitely some sort of memory leak in Alpha 14, I think, because running Alpha 14 caused an "out of memory" crash in a system with 32 gigabytes of ram (and 80 gigabytes of "virtual memory" as well).

bugreport: https://mega.nz/file/M9EGgLoZ#PdG9mw-iYGZtJcwXsKIxW6wp3T-PGjTWfG3dviDLXCY

Log: https://pastebin.com/gymyQtbV

Debug Log: https://mega.nz/file/FkVAFRDS#2_ULrOSbImZPHcH2Ojvnip39_A2rimZVs5Epwuq8ii4

I could be totally wrong but I think both the BSOD and the Out of Memory crash happened when processing the Forgotten Vale region.

Edited December 27, 2020 by DarthVitrial

[sheson]

There's definitely some sort of memory leak in Alpha 14, I think, because running Alpha 14 caused an "out of memory" crash in a system with 32 gigabytes of ram (and 80 gigabytes of "virtual memory" as well).

bugreport: https://mega.nz/file/M9EGgLoZ#PdG9mw-iYGZtJcwXsKIxW6wp3T-PGjTWfG3dviDLXCY

Log: https://pastebin.com/gymyQtbV

Debug Log: https://mega.nz/file/FkVAFRDS#2_ULrOSbImZPHcH2Ojvnip39_A2rimZVs5Epwuq8ii4

 

I could be totally wrong but I think both the BSOD and the Out of Memory crash happened when processing the Forgotten Vale region.

Use task manager to find out which process is actually using up all the memory, DynDOLOD, LODGen or TexConv.

 

The Delphi memory leak tester thingy does not find anything in DynDOLOD. Use sensible (default) resolutions for all LOD textures.

 

I have seen LODGen require 5GB+ per thread. It typically starts one per core. Depends on quality of terrain LOD meshes. Temporarily disable high resolution xLODGen meshes, so it uses the vanilla LOD meshes to see if it makes a difference.

 

Consider generating occlusion separately with xLODGen.

Edited December 27, 2020 by sheson

[DarthVitrial]

Use task manager to find out which process is actually using up all the memory, DynDOLOD, LODGen or TexConv.

LodGen

And yeah, I'm using default settings for quality.

 

I guess I can see 5GB per core, times 24 cores, could be an issue if that's what's happening.

I'll try disabling Occlusion in DyndoLod and doing it separately with xLodGen.

Edited December 27, 2020 by DarthVitrial

[sheson]

LodGen

And yeah, I'm using default settings for quality.

 

I guess I can see 5GB per core, times 24 cores, could be an issue if that's what's happening.

I'll try disabling Occlusion in DyndoLod and doing it separately with xLodGen.

If you like, also test running DynDOLOD with the -memory command line. It will be slower but should consume a bit less memory.

[DarthVitrial]

With Occlusion off, LODGeneratorCMD is still using a total of 24 gigabytes of RAM (and climbing).
I definitely never had this with prior alphas, which is odd.,

it then crashed with a "not enough space on disk error"...I'd had 70 GB free, and after running DyndoLOD it went down to 50mb. (Maybe virtual memory related? If Dyndolod exceeded the amount of RAM and started using the disk?)

Obviously this also never happened in prior alphas, usually it only took like two or three gigs.

EDIT: And yet again it happened when processing Forgotten Vale, so I definitely suspect something's related there.

Do you want the logs again?

Edited December 27, 2020 by DarthVitrial

[sheson]

With Occlusion off, LODGeneratorCMD is still using a total of 24 gigabytes of RAM (and climbing).

I definitely never had this with prior alphas, which is odd.,

 

it then crashed with a "not enough space on disk error"...I'd had 70 GB free, and after running DyndoLOD it went down to 50mb. (Maybe virtual memory related? If Dyndolod exceeded the amount of RAM and started using the disk?)

 

Obviously this also never happened in prior alphas, usually it only took like two or three gigs.

EDIT: And yet again it happened when processing Forgotten Vale, so I definitely suspect something's related there.

Do you want the logs again?

Run the worldspace all by itself.

 

I assume the 24GB is when it generates Tamriel?

 

If the OS tells the program there is not enough space to write files to, then there is either not enough space or the program is being blocked, like anti vir etc.

Edited December 27, 2020 by sheson

[DarthVitrial]

Well, it started with 80GB free and then just running DyndoLod caused the Pagefile to balloon to 75GB and fill my disk. Went back to 80 after a reboot.

I couldn't actually tell which one was the 24, I don't think it was Tamriel. It might have been Beyond Reach, actually. Either Beyond Reach or Falskaar.

Running it on just flamer valley worked, btw.

I'm going to try it on just the beyond reach worldspaces next.

[DarthVitrial]

I redid it with Falmer Valley and Beyond Reach and was able to get a DyndoLod Log of the resulting crash. (<Error: LODGenx64.exe failed to generate object LOD for DLC01FalmerValley. LODGenx64.exe returned E0434352. Check C:\Dyndolod\Logs\LODGen_SSE_DLC01FalmerValley_log.txt>)

 

I also noticed very high memory usage and frequent stutters when LodGen was processing the Arnima worldspaces (ArnimaTown and ArnimaDUPLICATE003 seemed the most significant, and also took well over an hour longer to process than it took to process the Tamriel worldspace, as well as using a lot more RAM than the Tamriel worldspace did.)

 

I've zipped all the logs here.

https://mega.nz/file/NtVQkbDI#-tWUlsWLgdpXV-nOhFmmqm4CcOsTnfP7yfccX54Z6NQ


hopefully this helps!

 

(the logs in there might be for the last 2 or 3 runs, not just the most recent)

Edited December 28, 2020 by DarthVitrial

[kojak747]

Using Alpha 14 and re-running grass cache, texgen etc - these are my only remaining errors. I'm going to have a dig around and try and get to the bottom of it, but if you had any pointers that would be great.

 

[05:07] <Warning: Billboard for model Meshes\spooknik\landscape\trees\treepineforest01_large.nif does not contain texture textures\landscape\trees\treepineforestbranchcomp.dds The Flora of Skyrim Trees SSE.esp treepineforest01_largeDUPLICATE002 [TREE:1E00AA92]>

 

[05:07] <Warning: Billboard for model Meshes\spooknik\landscape\trees\treepineforest01_large.nif does not contain texture textures\landscape\trees\treepineforestbranchcomp_n.dds The Flora of Skyrim Trees SSE.esp treepineforest01_largeDUPLICATE002 [TREE:1E00AA92]>

[05:08] <Warning: Billboard for model Meshes\spooknik\landscape\trees\treepineforest03_large.nif does not contain texture textures\landscape\trees\treepineforestbranchcomp.dds The Flora of Skyrim Trees SSE.esp treepineforest03_largeDUPLICATE004 [TREE:1E00AA98]>

 

[05:08] <Warning: Billboard for model Meshes\spooknik\landscape\trees\treepineforest03_large.nif does not contain texture textures\landscape\trees\treepineforestbranchcomp_n.dds The Flora of Skyrim Trees SSE.esp treepineforest03_largeDUPLICATE004 [TREE:1E00AA98]>

[05:09] <Warning: Billboard for model Meshes\spooknik\landscape\trees\treepineforest02_large.nif does not contain texture textures\landscape\trees\treepineforestbranchcomp.dds The Flora of Skyrim Trees SSE.esp treepineforest02_largeDUPLICATE004 [TREE:1E00AA99]>

[04:51] <Warning: Billboard for model Meshes\landscape\grass\rift_egrass01b.nif has different CRC32 for textures\landscape\grass\rift_egrass01.dds Grasses of Galen.esp RIFT_EGrass01bDUPLICATE001 [GRAS:22000A35]>
[04:51] <Warning: Billboard for model Meshes\landscape\grass\rift_egrass02gb.nif has different CRC32 for textures\landscape\grass\rift_egrass02g.dds Grasses of Galen.esp RIFT_EGrass02GbDUPLICATE001 [GRAS:22000A36]>
[04:53] <Warning: Billboard for model Meshes\landscape\grass\rift_esuoxeyedaisy03.nif has different CRC32 for textures\landscape\grass\rift_esuoxeyedaisy03.dds Grasses of Galen.esp RIFT_ESuOxeyeDaisyDUPLICATE004 [GRAS:22000A38]>
[02:14] <Warning: Billboard for model Meshes\landscape\grass\rift_egrass01b.nif has different CRC32 for textures\landscape\grass\rift_egrass01.dds Grasses of Galen.esp RIFT_EGrass01b [GRAS:220009CF]>
[02:16] <Warning: Billboard for model Meshes\landscape\grass\rift_egrass02gb.nif has different CRC32 for textures\landscape\grass\rift_egrass02g.dds Grasses of Galen.esp RIFT_EGrass02Gb [GRAS:220009D0]>
[02:22] <Warning: Billboard for model Meshes\landscape\grass\rift_esuoxeyedaisy03.nif has different CRC32 for textures\landscape\grass\rift_esuoxeyedaisy03.dds Grasses of Galen.esp RIFT_ESuOxeyeDaisy03 [GRAS:220009D2]>

[sheson]

Using Alpha 14 and re-running grass cache, texgen etc - these are my only remaining errors. I'm going to have a dig around and try and get to the bottom of it, but if you had any pointers that would be great.

 

[05:07]

 

[05:07]

 

[05:08]

 

[05:08]

 

[05:09]

 

[04:51]

[04:51]

[04:53]

[02:14]

[02:16]

[02:22]

Make sure to not have older billboard files from earlier versions. Then double check what the message says is true. The billboard txt contains the list of textures and their CRC32 that were installed at their creation time.

[sheson]

I redid it with Falmer Valley and Beyond Reach and was able to get a DyndoLod Log of the resulting crash. ()

 

I also noticed very high memory usage and frequent stutters when LodGen was processing the Arnima worldspaces (ArnimaTown and ArnimaDUPLICATE003 seemed the most significant, and also took well over an hour longer to process than it took to process the Tamriel worldspace, as well as using a lot more RAM than the Tamriel worldspace did.)

 

I've zipped all the logs here.

https://mega.nz/file/NtVQkbDI#-tWUlsWLgdpXV-nOhFmmqm4CcOsTnfP7yfccX54Z6NQ

 

 

hopefully this helps!

 

(the logs in there might be for the last 2 or 3 runs, not just the most recent)

Can you check what happens if you do these worldspaces without grass=1?

 

If it is the grass LOD generation, could you upload the grass cache for these worldspaces?

[DarthVitrial]

Can you check what happens if you do these worldspaces without grass=1?

 

If it is the grass LOD generation, could you upload the grass cache for these worldspaces?

it  seemed to work without grass, yeah.

 

Forgotten Vale Grass: https://mega.nz/file/0pcSiDLT#X24BiKABGDYgeY_n5qoCKs1TRzRyWxWTl112VJw1tzU

 

The arnima grass zip is 600 mb and my internet has, no exaggeration, a 20kbps upload speed, so...

Edited December 28, 2020 by DarthVitrial

[sheson]

it  seemed to work without grass, yeah.

 

Forgotten Vale Grass: https://mega.nz/file/0pcSiDLT#X24BiKABGDYgeY_n5qoCKs1TRzRyWxWTl112VJw1tzU

 

The arnima grass zip is 600 mb and my internet has, no exaggeration, a 20kbps upload speed, so...

OK. Let's see if I can find out anything with the data for the Forgotton Vale.