DynDOLOD 3.00 Alpha 171

[Dez756]

33 minutes ago, Dez756 said:

Do the same for any of the reported textures that are shown when the currently used NIF is highlighted in the asset browser.

Unlike the Meshes, I don't see a way to save the vanilla textures using the Asset Browser.  Am I missing something?

Edit:  Nevermind!  I think I figured it out.

Edited June 28, 2022 by Dez756

[Mousetick]

10 hours ago, Blackread said:

Anyway, I hope this information can help someone else experiencing the same crashes. If you have any questions I will try and answer them, assuming I get notifications for replies to this post.

No offense, but your analysis doesn't demonstrate or explain anything regarding the root cause of your crashes, and is not readily applicable to other users, because it's completely specific to your load order and it doesn't precisely identify the root cause.

All we learn from your analysis is that there is some interaction/conflict (maybe related to navmesh(es)) between some mods in your load order that causes a crash depending on their relative order.

ESMifying a mod changes its position in the plugin load order, and if that mod uses a BSA, it also changes the overwrite priority of its assets and resources in the load order. This is a critical detail that may be easy to overlook. If this mod conflicts with other mods in the load order, the conflicts are now resolved with a different priority with a different outcome. Unless manual patching is done in xEdit and/or manual extraction of BSAs is done to properly resolve those conflicts, the end results will vary in-game.

Another important detail that may easily be missed, and is a corollary to the previous, is that once the load order is modified, especially in the ESM range, the DynDOLOD output should be considered invalid and unsafe. It needs to be re-generated from scratch. This is crucial to ensure DynDOLOD.esm is properly synced up with the load order. When you ESMify a plugin without re-generating DynDOLOD, the previous DynDOLOD.esm doesn't account for the change. For example, it may overwrite references that should be marked persistent but are not because the outdated DynDOLOD.esm doesn't forward changes from the now ESMified plugin. Non-persistent references defined in an ESM are removed from memory when their parent cell is unloaded, causing unpredictable behavior or crashes when they're accessed.

You mention in your analysis that the crash stopped when you disabled a group of mods, including DynDOLOD Output. You didn't specify whether you had re-generated DynDOLOD after ESMifying and re-sorting your mods.

I've ESMified a number of mods (~20), using the script you referenced, including Moon and Star, without any issues. My current load order contains 668 plugins and I've yet to experience a game-breaking bug or crash that is not due to my own mistakes.

If I may offer some advice to help ensure a stable setup:

• Sort load order with LOOT. Use custom ordering rules only as a last resort.
• Systematically clean plugins with xEdit.
• Systematically check for plugins errors in xEdit.
• Systematically check for plugin conflicts and patch them in xEdit.
• Systematically check for conflicts among BSAs, and between BSAs and loose files. Enable 'Archives Parsing' experimental feature in MO2 if using MO2. Overwritten Papyrus scripts require particular attention.
• Re-generate DynDOLOD plugins whenever the load order changes, especially if it changes in the ESM range.
• Religiously pay attention to warnings in DynDOLOD's Summary Report.
• For extra care, review the warnings in DynDOLOD's log files. Some of them are not reported in the Summary Report but are still noteworthy and may require action.
• Fix as many found issues as possible depending on severity, by yourselves, or seek out solutions and patches elsewhere.
• Re-generate 'dynamic/automatic' patches, such as Synthesis or similar, whenever the load order changes.

Needless to say, this is not a simple matter of ESMifying plugin with an xEdit script. It's a lot of time-consuming work that requires some "expertise". So perhaps the best advice would be not to ESMify mods in the first place unless you understand the implications and are prepared to deal with them.

[Blackread]

2 hours ago, Mousetick said:

No offense, but your analysis doesn't demonstrate or explain anything regarding the root cause of your crashes, and is not readily applicable to other users, because it's completely specific to your load order and it doesn't precisely identify the root cause.

All we learn from your analysis is that there is some interaction/conflict (maybe related to navmesh(es)) between some mods in your load order that causes a crash depending on their relative order.

ESMifying a mod changes its position in the plugin load order, and if that mod uses a BSA, it also changes the overwrite priority of its assets and resources in the load order. This is a critical detail that may be easy to overlook. If this mod conflicts with other mods in the load order, the conflicts are now resolved with a different priority with a different outcome. Unless manual patching is done in xEdit and/or manual extraction of BSAs is done to properly resolve those conflicts, the end results will vary in-game.

Another important detail that may easily missed, and is a corollary of the previous, is that once the load order is modified, especially in the ESM range, the DynDOLOD output should be considered invalid and unsafe. It needs to be re-generated from scratch. This is crucial to ensure DynDOLOD.esm is properly synced up with the load order. When you ESMify a plugin without re-generating DynDOLOD, the previous DynDOLOD.esm doesn't account for the change. For example, it may overwrite references that should be marked persistent but are not because the outdated DynDOLOD.esm doesn't forward changes from the now ESMified plugin. Non-persistent references defined in an ESM are removed from memory when their parent cell is unloaded, causing unpredictable behavior or crashes when they're accessed.

You mention in your analysis that the crash stopped when you disabled a group of mods, including DynDOLOD Output. You didn't specify whether you had re-generated DynDOLOD after ESMifying and re-sorting your mods.

I've ESMified a number of mods (~20), using the script you referenced, including Moon and Star, without any issues. My current load order contains 668 plugins and I've yet to experience a game-breaking bug or crash that is not due to my own mistakes.

If I may offer some advice to help ensure a stable setup:

• Sort load order with LOOT. Use custom ordering rules only as a last resort.
• Systematically clean plugins with xEdit.
• Systematically check for plugins errors in xEdit.
• Systematically check for plugin conflicts and patch them in xEdit.
• Systematically check for conflicts among BSAs, and between BSAs and loose files. Enable 'Archives Parsing' experimental feature in MO2 if using MO2. Overwritten Papyrus scripts require particular attention.
• Re-generate DynDOLOD plugins whenever the load order changes, especially if it changes in the ESM range.
• Religiously pay attention to DynDOLOD's Summary Report and fix as many reported issues as possible.
• For extra care, review the warnings in DynDOLOD's log files. Some of them are not reported in the Summary Report but are still noteworthy and may require action.
• Fix as many found issues as possible depending on severity, by yourselves, or seek out solutions and patches elsewhere.

Needless to say, this is not a simple matter of ESMifying plugin with an xEdit script. It's a lot of time-consuming work that requires some "expertise". So perhaps the best advice would be not to ESMify mods in the first place unless you understand the implications and are prepared to deal with them.

In terms of the records edited by RedBag's Rorikstead, the relative order of the overwrites did not change between being an ESM or not. When ESM flagged the only plugins that were overwriting records from RedBag's Rorikstead were patches for RedBag's Rorikstead (non-ESM), that will overwrite the plugin regardless. In terms of assets, they are all packed in a bsa and all the overwrites are loose files, so no change there either.

The DynDOLOD Output I was testing this with was generated with the ESMified version of the plugin, so if anything, it was the non-ESM plugin that should have made it invalid and unsafe. Of course, removing the ESM flag did change some things in the overwrites, because the DynDOLOD.esm plugin overwrites disabled large references with its activator thingy, and these were reverted when the ESM flag was removed and the plugin moved above DynDOLOD.esm. But if this was the cause of the crash, just removing DynDOLOD output should have resolved it, but it didn't.

But prompted by your suspicion I decided to do some further testing. I placed the plugin as the last ESM plugin. This way, at least to my knowledge, the only thing that changes between adding and removing the ESM flag is the flag itself. With the flag the game crashed, without it didn't.

Next I took the original plugin from RedBag's Rorikstead, with the ITMs and deleted navmeshes intact, and added the ESM flag to that. This produced no crashing. Next I ran the plugin through the QuickAutoCleaner. This removed the ITMs, but left the deleted navmeshes. No crashing. Then I copied over the four edited navmesh records which I had separated into a new plugin here https://www.nexusmods.com/skyrimspecialedition/mods/69952, deleted the Navigation Mesh Info Map and regenerated it in CK. Now the game started to crash again.

Then I once again took the original plugin, cleaned ITMs, applied ESM flag, and this time used the separate plugin to overwrite the deleted navmeshes. These four navmesh records were edited in my LO by only Skyrim.esm, RedBag's Rorikstead.esp and my navmesh undeletion plugin. Now there were no crashes.

Putting all this together I would conclude that you were right in that it was a navmesh related issue. Not in terms of conflicts, but some undetermined latent issue with my edits to the deleted navmeshes that would only manifest if the plugin had ESM flag applied to it.

At this point I have pretty much tested all I can think of myself, and probably have reached the limits of my knowledge as well. I'll attach the broken plugin here in case you or someone else wishes to investigate this further. Thank you for your scepticism. It didn't make a big difference in the stability of my game as I had already resolved this issue in an alternative manner by separating the large ref edits into a plugin which I flagged as a master, but I did gain some more insight into the issue thanks to this more granular testing.

Thank you for the advice in your bullet points, but I am already doing all of those things regularly. But they without a doubt will help others reading this thread.

When it comes to Moon and Star though I can't say for sure what's going on with that one. My last experience with it was a couple years ago, around the time I published my ESMifier script. It very well may be that one was just a load order issue. I certainly wasn't as meticulous and thorough back then. Maybe I will revisit that subject some day again.

RedBag's Rorikstead.esp

Edited June 28, 2022 by Blackread

[sheson]

7 hours ago, Dez756 said:

Will do!  

Just to be sure, should I be running Dyndolod after making the foregoing changes before testing?  So far I haven't been.

No, for now just replace the files in the load order with the vanilla once which we assume should be fine.

[Mousetick]

2 hours ago, Blackread said:

Putting all this together I would conclude that you were right in that it was a navmesh related issue. Not in terms of conflicts, but some undetermined latent issue with my edits to the deleted navmeshes that would only manifest if the plugin had ESM flag applied to it.

Thanks for following up with all the details and clarifications. At this point it's pretty clear the issue has nothing to do with DynDOLOD and is specific to your navmesh patch so I'll keep my comments short:

• Editing navmeshes in CK is way beyond my knowledge so I can't help you there.
• As far as I understand, if the Navigation Mesh Info Map links to door/portal/passage references, they need to be persistent. Not being explicitly defined as persistent in the ESP may not be a problem, but could become one once the plugin is ESMified,  as they would now be truly temporary.

2 hours ago, Blackread said:

It didn't make a big difference in the stability of my game as I had already resolved this issue in an alternative manner by separating the large ref edits into a plugin which I flagged as a master

That's good advice and is actually much easier and safer than ESMifying whole plugins if the goal is to fix a few large references bugs rather than limiting the number of references.

[Blackread]

1 hour ago, Mousetick said:

Thanks for following up with all the details and clarifications. At this point it's pretty clear the issue has nothing to do with DynDOLOD and is specific to your navmesh patch so I'll keep my comments short:

• Editing navmeshes in CK is way beyond my knowledge so I can't help you there.
• As far as I understand, if the Navigation Mesh Info Map links to door/portal/passage references, they need to be persistent. Not being explicitly defined as persistent in the ESP may not be a problem, but could become one once the plugin is ESMified,  as they would now be truly temporary.

That's good advice and is actually much easier and safer than ESMifying whole plugins if the goal is to fix a few large references bugs rather than limiting the number of references.

Yes, I actually never meant to imply that this was a DynDOLOD issue. I mainly made a post here because there was a similar looking crash report that seemed to have been somewhat unresolved.

I can't take credit for the separate large ref plugin trick, as it's actually from the DynDOLOD Large Ref guide. But it is likely the least error prone solution in many cases. I often just ESMify the whole plugin out of laziness. xD

[Dez756]

3 hours ago, sheson said:

No, for now just replace the files in the load order with the vanilla once which we assume should be fine.

I replaced all five texture files shown for the icebergsmall02.nif in Asset Browser with vanilla textures, checking to make sure the path was correct in the MO2 Data tab.  Stil CTD'd at Riften Gate.  See crash log.

As a test, I again disabled the Dyndolod Output and traveled to the same spot.  No CTD.

Crash_2022_6_28_15-38-18.txt

[sheson]

46 minutes ago, Dez756 said:

I replaced all five texture files shown for the icebergsmall02.nif in Asset Browser with vanilla textures, checking to make sure the path was correct in the MO2 Data tab.  Stil CTD'd at Riften Gate.  See crash log.

As a test, I again disabled the Dyndolod Output and traveled to the same spot.  No CTD.

Crash_2022_6_28_15-38-18.txt 41.01 kB · 0 downloads

Seems we are not getting anywhere with this. Maybe time to do the steps explained under "Debug CTD caused by missing or invalid NIF" in the DynDOLOD-README.txt

First check/remove the object LOD mesh *.BTO file to rule them out.
If it still CTDs with all meshes removed, check if disabling DynDOLOD.esp and keeping DynDOLOD.esm makes a difference. In case it does, do the debug=true tests.

If it crashes with only DynDOLOD.esm, let me know for different suggestions.

[Dez756]

1 hour ago, sheson said:

If it still CTDs with all meshes removed, check if disabling DynDOLOD.esp and keeping DynDOLOD.esm makes a difference. In case it does, do the debug=true tests.

Still CTD with meshes removed.  No CTD with DynDOLOD.esp only disabled.

I'll try to get to the debug=true tests later.

[Dez756]

On 6/28/2022 at 5:17 PM, Dez756 said:

I'll try to get to the debug=true tests later.

Here's my Papyrus log with debug=true.  I don't see entries in the Papyrus log exactly like the example in ReadMe.  I tried to search for the Form ID of the last entry referencing DynDOLOD ([06/29/2022 - 03:54:57PM] [SHESON_DynDOLOD_LODObject < (BE028E70)>] [SHESON_DynDOLOD_Firstborn < (BE028E5A)>] Enable #2 [Form < (20004052)>] using dyndolod\lod\effects\fxfirewithembers01_dyndolod_lod.nif TRUE False TRUE) but SSEEdit doesn't seem to find anything (although I'm not sure I'm looking in the right spot - I've used SSEEdit for some things like cleaning, adding keywords or heel sounds but never searched for Form IDs).  See screen shot for the results of filter for BE028E70.  

I note that the entries in the log in the last second all seem to concern Creature Framework.  To test whether that was somehow involved, I removed CF and all associated mods and traveled (on a new game) to the same Riften Gate.  Still CTD'd.  Again, though this mod and its associates have been in my load order for literally years!

It's so weird that this seems to be the only place it CTDs as I had traveled to most of the other cities before I ran across this problem at Riften.

I'm sure this is getting frustrating for you!  Should I start (dread) removing mods and running everything (Xlodgen, TexGen, DynDOLOD) again? 

 

Papyrus.0.log

Edited June 29, 2022 by Dez756

[Dez756]

In case it helps, here's a screenshot right as the game CTD's.

[sheson]

7 hours ago, Dez756 said:

Here's my Papyrus log with debug=true.  I don't see entries in the Papyrus log exactly like the example in ReadMe.  I tried to search for the Form ID of the last entry referencing DynDOLOD ([06/29/2022 - 03:54:57PM] [SHESON_DynDOLOD_LODObject < (BE028E70)>] [SHESON_DynDOLOD_Firstborn < (BE028E5A)>] Enable #2 [Form < (20004052)>] using dyndolod\lod\effects\fxfirewithembers01_dyndolod_lod.nif TRUE False TRUE) but SSEEdit doesn't seem to find anything (although I'm not sure I'm looking in the right spot - I've used SSEEdit for some things like cleaning, adding keywords or heel sounds but never searched for Form IDs).  See screen shot for the results of filter for BE028E70.  

I note that the entries in the log in the last second all seem to concern Creature Framework.  To test whether that was somehow involved, I removed CF and all associated mods and traveled (on a new game) to the same Riften Gate.  Still CTD'd.  Again, though this mod and its associates have been in my load order for literally years!

It's so weird that this seems to be the only place it CTDs as I had traveled to most of the other cities before I ran across this problem at Riften.

I'm sure this is getting frustrating for you!  Should I start (dread) removing mods and running everything (Xlodgen, TexGen, DynDOLOD) again? 

 

Papyrus.0.log 383.58 kB · 0 downloads

Since the debug message from DynDOLOD is 9 seconds before the crash, it is not caused by any of the dynamic LOD objects.

When entering form IDs into the field top left you need to adjust the first two digits to the actual ID a plugin has in xEdit, e.g. 19 in case of the screenshot.

The next step would be to selective remove records from DynDOLOD.esp in a binary search in order to only keep record in the plugins that causes the problem.

1) Make a backup of DynDOLOD.esp
2) Load DynDOLOD.esp in xEdit and remove half records
3) Test if game still CTDs
4) if it crashes start from step 1) If it doesn't crash, restore backup and  continue with step 2) removing the other half of records which are not causing the crash.

Only remove records from [+] Worldspace
First time, remove all worldspaces that are not Tamriel as we expect the error to be in there only.
Second time, remove all Blocks and keepo the 0000D74 Cell persistent worldspace cell. If the crash is a record in the persistent cell, remove all overwrites first (they are listed at the top). If it is one of the overwrites, restore from back, remove all other records. Then do the loop by removing half the overwrites until only the problematic record remains. Otherwise , continue by removing about half the references that do not start with Tamriel_*, e.g remove the reference that start with a pluginname.
If the crash happens in the Blocks, do the loop removing about half the Blocks until only on Block remains, then Sub-Blocks until only one Sub-Block remains, then cells until only one cell remains, then references until only one reference remains. If you are down to the last reference, remove it too to see if it might be actually the cell record.

[jimmywon34]

dyndolod doesn't finish for me when launched and keeps exiting out/crashing wihtout any warning

Edited June 30, 2022 by jimmywon34

[sheson]

7 hours ago, jimmywon34 said:

dyndolod doesn't finish for me when launched and keeps exiting out/crashing wihtout any warning

Read the first post which log, debug log and bugreport.txt to upload.

If none of those files were created check the Windows Event log.

Make sure to use a recent MO2 version and that OS, antivir does not interfere,

[Dez756]

18 hours ago, sheson said:

If the crash happens in the Blocks, do the loop removing about half the Blocks until only on Block remains, then Sub-Blocks until only one Sub-Block remains, then cells until only one cell remains, then references until only one reference remains.

Okay!  No crash if I remove two references at the following path in the DynDOLOD.esp.  

Worldspace>0000003C Tamriel> Block 1,-1> Sub-Block 5,-3>0000BCD0> Temporary

>1902677D Placed Object bluesskyrim_0121FA_DynDOLOD_CHILD IcebergSmall02_DynDOLOD_LOD[STAT:0D004028]

and

 >190266D6 Placed Object bluesskyrim_0121FB_DynDOLOD_CHILD  IcebergSmall02_DynDOLOD_LOD [STAT:0D004028]

I'm sure you recall that the .Net Script Framework crash logs also referred to IcebergSmall02.  BluesSkyrim.esp appears to be a patch I added back in 2020, but I can't find it on Nexus anymore.  The Mod is called Update by KorruptkSwades.

Should I remove that patch and do everything again?  Or can I just play with those references removed?

Edited June 30, 2022 by Dez756