DynDOLOD 3.00 Alpha 182

[skyrim9838]

Sorry if I missed it but does the alpha version not have a mcm menu ? I was updating on my main save and noticed that I don't have a mcm menu for it.

[Mousetick]

Hello,

I'd like to get some clarification on a couple of Large Reference Bugs topics:

1. Large References to PlaneMarker (occlusion plane) STAT objects.

• It seems strange that PlaneMarkers would be large references. What would be the purpose? They don't have models, neither full nor object LODs, right? Is this used as a way for the engine to load them outside the uGrid?
• Is there a way to inspect PlaneMarkers in the console? In cycling through all the objects, I can get the door, wall, building, clouds, underside etc. but not the PlaneMarker, even though I'm right in front of the occlusion plane:

• I'm experiencing very weird behavior with at least one PlaneMarker which is a large reference, flagged by DynDOLOD as overwritten.

Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31B] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))
Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31C] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))

It's defined in Skyrim.esm and overwritten by 3 plugins in the load order:

1. Skyrim.esm places the PlaneMarker.
2. LegacyoftheDragonborn.esm disables it.
3. Occ_Skyrim_Tamriel.esp (from eFPS - Exterior FPS boost, ESM-flagged "out of the box") re-enables it, resizes it and changes its position.
4. Occ_Skyrim-Legacy_of_Dragonborn_patch.esp (from eFPS - Official Patch Hub) re-sizes it and changes its position again.

Without #4, the PlaneMarker is in the wrong position and obscures a building. With #4 everything looks ok.

I tried to flag #4 as ESM in xEdit, which suppressed the DynDOLOD warning, but now the PlaneMarker is in the wrong position again, as if it were loaded from the wrong plugin - which one? I don't know since I can't inspect it with the console. Strangely, it's bugged only when I enter Solitude from the Tamriel worldspace. If I exit from a Solitude building (interior cell) directly to the Solitude child worldspace, the PlaneMarker is not bugged, it's like everything is loaded correctly.

Note: This behavior is observed with or without DynDOLOD plugins enabled in the load order. So the issue is not related to DynDOLOD. DynDOLOD only comes into play because it helpfully warns about the large references.

2. Scope of Large References Bugs.

I want to confirm and be absolutely clear on this point from the online doc:

Quote

Large reference bugs describe the issue that the object LOD for all large references of a cell is not disabled when the full models for the large references are loaded. [...] Again, the entire cell is affected, regardless of the model used by the large reference or its type.

Let's say a cell contains 2 large references, LR1 and LR2. One of them, LR1 is overwritten by at least one ESP plugin, the other, LR2 remaining untouched. LR1 will trigger the LOD bug for both LR1 and LR2. Is this correct?

From this we can conclude that even though an ESP plugin may overwrite only one large reference, its effect may be potentially far more visually "damaging" than one might think if the affected cell contains many large references. In other words, the number of "initially disabled" or "overwritten" warnings issued by DynDOLOD for a given plugin is not a good indication of how seriously bugged the LOD may become because of it. Is this correct?

Thanks.

[sheson]

10 hours ago, z929669 said:

I think I may understand the issue now. Let me test it: Trees in object lod *.bto just outside the walls of the child world unload properly (from player perspective inside child worlds and looking out --so player doesn't see these  trees when inside Riften/Solitude) but traditional tree LOD does not unload and player sees them while inside the child world? Then this makes the player think that LOD isn't working correctly since it's inconsistent from inside the child world?

Yes, for the active cells the child worldspace replaces.

10 hours ago, skyrim9838 said:

Sorry if I missed it but does the alpha version not have a mcm menu ? I was updating on my main save and noticed that I don't have a mcm menu for it.

Nothing changed. The scripts in DynDOLOD Resources are still the same. Test with a new game. Make sure to stay with the plugin limit.

[sheson]

51 minutes ago, Mousetick said:

Hello,

I'd like to get some clarification on a couple of Large Reference Bugs topics:

1. Large References to PlaneMarker (occlusion plane) STAT objects.

• It seems strange that PlaneMarkers would be large references. What would be the purpose? They don't have models, neither full nor object LODs, right? Is this used as a way for the engine to load them outside the uGrid?
• Is there a way to inspect PlaneMarkers in the console? In cycling through all the objects, I can get the door, wall, building, clouds, underside etc. but not the PlaneMarker, even though I'm right in front of the occlusion plane:

• I'm experiencing very weird behavior with at least one PlaneMarker which is a large reference, flagged by DynDOLOD as overwritten.

Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31B] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))
Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31C] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))

It's defined in Skyrim.esm and overwritten by 3 plugins in the load order:

1. Skyrim.esm places the PlaneMarker.
2. LegacyoftheDragonborn.esm disables it.
3. Occ_Skyrim_Tamriel.esp (from eFPS - Exterior FPS boost, ESM-flagged "out of the box") re-enables it, resizes it and changes its position.
4. Occ_Skyrim-Legacy_of_Dragonborn_patch.esp (from eFPS - Official Patch Hub) re-sizes it and changes its position again.

Without #4, the PlaneMarker is in the wrong position and obscures a building. With #4 everything looks ok.

I tried to flag #4 as ESM in xEdit, which suppressed the DynDOLOD warning, but now the PlaneMarker is in the wrong position again, as if it were loaded from the wrong plugin - which one? I don't know since I can't inspect it with the console. Strangely, it's bugged only when I enter Solitude from the Tamriel worldspace. If I exit from a Solitude building (interior cell) directly to the Solitude child worldspace, the PlaneMarker is not bugged, it's like everything is loaded correctly.

Note: This behavior is observed with or without DynDOLOD plugins enabled in the load order. So the issue is not related to DynDOLOD. DynDOLOD only comes into play because it helpfully warns about the large references.

2. Scope of Large References Bugs.

I want to confirm and be absolutely clear on this point from the online doc:

Let's say a cell contains 2 large references, LR1 and LR2. One of them, LR1 is overwritten by at least one ESP plugin, the other, LR2 remaining untouched. LR1 will trigger the LOD bug for both LR1 and LR2. Is this correct?

From this we can conclude that even though an ESP plugin may overwrite only one large reference, its effect may be potentially far more visually "damaging" than one might think if the affected cell contains many large references. In other words, the number of "initially disabled" or "overwritten" warnings issued by DynDOLOD for a given plugin is not a good indication of how seriously bugged the LOD may become because of it. Is this correct?

Thanks.

It's anyone's guess why invisible things like markers or planes are listed as large references in the vanilla plugins. The main criteria seems to be STAT/MSTT and object bounds * scale. The script I wrote ignores markers and planes.
I am not aware of how one could be making occlusion planes/boxes visible/clickable in the game. Use CK.

The last plugin in the load order to change a record wins. LOD or being marked as large reference does not change that. So if something is out of place, it is because of the last plugin. Check how the game loads plugins and which plugin is the last to modify a record with xEdit.

Correct, if the reference bug is triggered in a cell, the LOD for all large references in the cell does not unload.

[Mousetick]

Thanks for the quick answers.

41 minutes ago, sheson said:

I am not aware of how one could be making occlusion planes/boxes visible/clickable in the game. Use CK.

Yeah but CK only shows a static view of the cell  I was looking for a way to check if a PlaneMarker is loaded and from where, at runtime.

45 minutes ago, sheson said:

The last plugin in the load order to change a record wins. LOD or being marked as large reference does not change that. So if something is out of place, it is because of the last plugin. Check how the game loads plugins and which plugin is the last to modify a record with xEdit.

I managed to fix the PlaneMarker weird loading issue by applying a combination of the same remedies with xEdit as other large references: 1) flag the ESP as ESM, 2) overwrite the REFR in a custom ESM patch and bury it underground, and 3) copy the REFR as a new record in a custom ESM patch so that it's no longer an overwritten large reference.

This particular overwritten large reference looked suspicious: the ESP overwriting it not only resizes and repositions it, it also puts it within a different parent cell. I suppose the engine may have been quite confused by this hence the weird behavior.

Which brings some new remarks and questions:

• The whole large reference system looks pretty fragile and not at all compatible with modding. Mod authors should be a lot more aware of it and avoid overwriting large references, opting to bury existing ones underground and create new ones instead.
• The issue is how can mod authors be more aware of large references when they are not clearly made visible by the tools like xEdit or CK?
• If mod authors want to take advantage of the large reference system, they should create new references in an ESM and use your script to generate the RNAM list of the worldspace, rather than modifying existing large references.
• Overwriting large references creates more issues than just LOD unloading: What if the plugin resizes it so that its bounds no longer satisfy the "large" criteria? We end up with a unnecessarily bugged "small" reference. What if the plugin moves it to a different cell? Which cell is now bugged - the original one, the new one, both?

[sheson]

1 hour ago, Mousetick said:

Thanks for the quick answers.

Yeah but CK only shows a static view of the cell  I was looking for a way to check if a PlaneMarker is loaded and from where, at runtime.

I managed to fix the PlaneMarker weird loading issue by applying a combination of the same remedies with xEdit as other large references: 1) flag the ESP as ESM, 2) overwrite the REFR in a custom ESM patch and bury it underground, and 3) copy the REFR as a new record in a custom ESM patch so that it's no longer an overwritten large reference.

This particular overwritten large reference looked suspicious: the ESP overwriting it not only resizes and repositions it, it also puts it within a different parent cell. I suppose the engine may have been quite confused by this hence the weird behavior.

Which brings some new remarks and questions:

• The whole large reference system looks pretty fragile and not at all compatible with modding. Mod authors should be a lot more aware of it and avoid overwriting large references, opting to bury existing ones underground and create new ones instead.
• The issue is how can mod authors be more aware of large references when they are not clearly made visible by the tools like xEdit or CK?
• If mod authors want to take advantage of the large reference system, they should create new references in an ESM and use your script to generate the RNAM list of the worldspace, rather than modifying existing large references.
• Overwriting large references creates more issues than just LOD unloading: What if the plugin resizes it so that its bounds no longer satisfy the "large" criteria? We end up with a unnecessarily bugged "small" reference. What if the plugin moves it to a different cell? Which cell is now bugged - the original one, the new one, both?

Moving large references to different cells probably triggers bugs regardless of plugin type. I am pretty sure I tested it, but are not certain about the result anymore.

I have reported these bugs first time end of 2016. DynDOLOD supports SSE since version 2.22 that came out March 2017 and includes information about the large reference bugs and reports overwritten large references since then. Since half a decade.

I consider every mod that triggers large reference bugs as (visually) incompatible with Skyrim Special Edition since the facts and how to work around them are public knowledge. Hence the beta status of DynDOLOD for SSE (and for VR since it supports large references).

The first version of the xEdit large reference generation script exists since 2017 as well IIRC.

DynDOLOD fixes large reference bugs triggered by size changes by setting the threshold value to 1.

Be honest, first reaction to CK reporting errors for the vanilla game files is to disable them. Problem is, now you do not see errors in modding plugins/mod either.

[Mousetick]

The following is long and broad, so feel free to move the whole conversation into a separate topic if you think it doesn't belong here.

9 hours ago, sheson said:

Be honest, first reaction to CK reporting errors for the vanilla game files is to disable them. Problem is, now you do not see errors in modding plugins/mod either.

Are you saying that CK can properly detect overwritten large references and notify the user, but mod authors disable this option? I know very little of CK. I just looked at it and played with it a bit, I couldn't find anything related to large references. I also tested moving a large reference and saving the overwrite in an ESP. CK happily obliged without a blip.

9 hours ago, sheson said:

I consider every mod that triggers large reference bugs as (visually) incompatible with Skyrim Special Edition since the facts and how to work around them are public knowledge.

I understand your position. It must be very frustrating to be dealing with those issues as long and as best as you have been. However I'm not sure it's fair to put the blame on the mod authors for being ignorant or on the mods for being defective.

1. There is no indication in xEdit or CK that a reference is marked as large in the master, there is no error or warning triggered in those tools when such reference is overwritten either. So mod authors are "blind" to large references. How can someone tell beforehand that modifying a particular reference that looks like any other is going to cause problems?

The RNAM list is hidden by default in xEdit, and even if the option to show it is turned on, the information available is unusable by a mod author. It's a huge "blob" that is stashed away in the parent worldspace, disconnected from the references themselves, and impossible to humanly parse or cross-reference with the references in a given cell.

Would you agree with this, or am I missing something?

2. From an anecdotal perspective as a casual mod user, it seems that more often than not, I'm sorry to say, mod authors are not familiar with DynDOLOD. Looking at the posts in the comments section on Nexus, user questions like "Do I need to re-run DynDOLOD with this mod?" or "Does this mod work with DynDOLOD?" receiving answers from mod authors like "What is DynDOLOD?" or "Maybe, I don't know since I don't use it" is a pretty common sight.

Why that is, I don't know. In an ideal world, everyone involved in modding Skyrim should be familiar with DynDOLOD  But that's not the case, for whatever reasons. Perhaps the large reference issues would be better known if they were considered and publicized separately from DynDOLOD. They exist independently of DynDOLOD but mod authors may be confusing them as a DynDOLOD-specific or DynDOLOD-induced issues.

3. On second thought, I realized that the suggested "best practices" for mod authors dealing with large references (splitting mod into ESM+ESP, burying them, creating new references...) don't appear to be a viable solution. They're merely workarounds, and pretty complicated to implement at that. Beside the extra care and work required, this scheme creates its own set of problems in regards to compatibility between mods and conflict resolution.

For example, ModA and ModB each modify the same large reference. Let's say the object is a big boulder: ModA disables it to place its own building at that location, and ModB changes its dimensions and position slightly for "enhanced" landscaping. Following best practices, both mods bury the original boulder, and ModB places its own new boulder. Now what happens when both mods are present in the load order? There is ModB's boulder clipping into ModA's building.

What used to be a simple load order priority conflict, becomes a complicated issue requiring visual inspection in the console or in CK to figure out where does the boulder come from and what to do with it, in order to resolve and patch the conflict. The complexity increases exponentially as the number of mods increase, since with this scheme, all mods "win" with their new references.

Furthermore, the workarounds are not readily applicable to dynamically swapped references, such as with Base Object Swapper.

I'd love to know your thoughts on the following ideas:

1. Would you consider making the "preflight" validation phase of DynDOLOD a separate tool that could be used independently of DynDOLOD? I'm talking about the validation phase that DynDOLOD performs after loading all the plugins and before generating the LODs and plugins per se, which is then used to produce the HTML diagnostics stored in the Summary folder. This would include the Large Reference Bugs warnings of course, but also all the other useful warnings.

I often find myself wishing to be able to get these diagnostics but renouncing because they require running the full DynDOLOD tool and immobilizing my PC for a while. I did try running DynDOLOD and checking off boxes to speed things up, but all I got was nothing: DynDOLOD simply quit, saying something like "All done, thank you, bye" 

I think such tool could become an essential part of the modding toolkit for mod users and authors alike, similarly to Mod Organizer, xEdit, LOOT or BethINI, but without the pretty steep learning curve of the full DynDOLOD suite. It could also serve as a "trojan" to help familiarize people with large reference issues and to promote DynDOLOD via messages like "DynDOLOD can automatically fix this issue", with pointers to more information and downloads.

2. It would be helpful if xEdit could visually indicate large references by displaying them in a different color or style, so that mod authors are no longer blind.

3. As previously discussed, a more robust and generic solution to the large references issues would be at engine level, via a DLL runtime patcher, by automatically skipping overwritten large references, from a list previously generated by DynDOLOD.

Pushing this idea further, an even better and cleaner solution would be for the DLL runtime patcher to build the in-memory RNAM list of large references from scratch on the fly (and maybe cache it if it's too expensive to do it every time the worldspaces are loaded). In other words, it would work a bit like your xEdit script, but at runtime rather than at plugin creation time, and it would automatically skip those that are overwritten by ESP plugins.

With this approach, large references would always be correct and "clean", no matter the load order, no matter the final dimensions of the overwritten references, it could work cooperatively with Base Object Swapper, and finally there would no longer be a need for DynDOLOD to verify or attempt to fix them. It wouldn't matter if mod authors are aware of them or are specifically working around them. It might also make large references on the fly for ESM plugins that don't explicitly define them. It could make fLargeRefMinSize a runtime configuration setting rather than a fixed setting.

Thanks for reading and again, sorry for the long post. I hope this provided some constructive food for thought.

[z929669]

58 minutes ago, Mousetick said:

Thanks for reading and again, sorry for the long post. I hope this provided some constructive food for thought.

Interesting insights. I would use such a tool(s).

[sheson]

2 hours ago, Mousetick said:

1. Would you consider making the "preflight" validation phase of DynDOLOD a separate tool that could be used independently of DynDOLOD? I'm talking about the validation phase that DynDOLOD performs after loading all the plugins and before generating the LODs and plugins per se, which is then used to produce the HTML diagnostics stored in the Summary folder. This would include the Large Reference Bugs warnings of course, but also all the other useful warnings.

Check dynamic LOD only for a quick check of things.

[sheson]

2 hours ago, Mousetick said:

2. It would be helpful if xEdit could visually indicate large references by displaying them in a different color or style, so that mod authors are no longer blind.

Use xLODGen terrain LOD beta in edit mode. Even if RNAM data is not shown on worldspace records because of Hide "never shown" being checked, large references list the worldspace record on the Referenced by tab.

If you uncheck the Hide "never shown" you will notice the delays the huge amount of data in Skyrim.esm causes whenever the Tamriel worldspace records is somehow involved when checking references or cells etc. That's why normal xEdit ignores the data by default on Skyrim.esm.

[Greg]

2 hours ago, Mousetick said:

Are you saying that CK can properly detect overwritten large references and notify the user, but mod authors disable this option? I know very little of CK. I just looked at it and played with it a bit, I couldn't find anything related to large references. I also tested moving a large reference and saving the overwrite in an ESP. CK happily obliged without a blip.

This is complete off topic for here, but run Creation Kit and load a random mod. Creation Kit will spin its wheels for a while loading the base game masters and then spew out a massive log errors and warnings for Skyrim.esm, Update.esm, etc. I know my naive reaction to this the first time I ran across it was to hunt down a way to disable it.

[Zanderat]

Alpha 78.  Referenced log has literally nothing relevant.  Never happened with previous versions.  Mod set up is stable and hasn't changed (except for removing TPOS2) since last successful run.  Thanks.

Spoiler

Skyrim/Fallout Object/Terrain LOD Generator 3.0.0.0
Created by Ehamloptiran and Zilav
Updated by Sheson

Log started at 2:21:11 PM
Game Mode: TES5
Worldspace: Test
Fix Tangents: False, False, False, False
Generate Tangents: True, True, True, True
Generate Vertex Colors: True, True, True, True
Merge Vertex Colors: False, False, False, False
Merge Meshes: True
Grouping: False
Remove Faces under Terrain: False
Remove Faces under Water: False
Use HD Flag: True
Ignore Materials: False
Alpha DoubleSided: False
Default Alpha Threshold: 128
Use Source Alpha Threshold: False
Use Backlight Power: False
Use Decal Flag: False
Specific level: No
Specific quad: No
Max Level: 32
Output: C:\DynDOLOD\DynDOLOD\DynDOLOD_Output\
Generating object LOD meshes for worldspace Test

 

 

edit:  Windows Even Viewer showed this:

Quote

Faulting application name: LODGenx64.exe, version: 3.0.0.0, time stamp: 0x6223eb75

Faulting module name: KERNELBASE.dll, version: 10.0.19041.1503, time stamp: 0xb2acaea9
Exception code: 0xe0434352
Fault offset: 0x0000000000034f69
Faulting process id: 0xab64
Faulting application start time: 0x01d834b4004f0c85
Faulting application path: C:\DynDOLOD\DynDOLOD\Edit Scripts\LODGenx64.exe
Faulting module path: C:\WINDOWS\System32\KERNELBASE.dll
Report Id: b706b3ce-bbde-4860-a29e-4a5640dfcd86
Faulting package full name:
Faulting package-relative application ID:

and this:

Quote

Fault bucket 2224769302759408942, type 5
Event Name: CLR20r3
Response: Not available
Cab Id: 0

Problem signature:
P1: LODGenx64.exe
P2: 3.0.0.0
P3: 6223eb75
P4: System
P5: 4.8.4360.0
P6: 606e71b3
P7: 2f54
P8: 0
P9: System.InvalidOperationException
P10:

 

Edited March 10, 2022 by Zanderat

[sheson]

54 minutes ago, Zanderat said:

Alpha 78.  Referenced log has literally nothing relevant.  Thanks.

Use the "Click on this link for additional explanations and help for this message" as explained on the first post to open https://dyndolod.info/Help/LODGen 
Read what it has to say for the error code E0434352 returned by LODGen

Use the "Copy to message clipboard" and paste the text instead of posting a screenshot as explained on the first post.

Follow the suggestion to search this thread for the error as suggested by first post.
https://stepmodifications.org/forum/search/?q=E0434352&quick=1&type=forums_topic&item=15606

[Zanderat]

13 minutes ago, sheson said:

Use the "Click on this link for additional explanations and help for this message" as explained on the first post to open https://dyndolod.info/Help/LODGen 
Read what it has to say for the error code E0434352 returned by LODGen

Use the "Copy to message clipboard" and paste the text instead of posting a screenshot as explained on the first post.

Follow the suggestion to search this thread for the error as suggested by first post.
https://stepmodifications.org/forum/search/?q=E0434352&quick=1&type=forums_topic&item=15606

I clicked on said link.  It told me to check the Windows Event Viewer.  Which I did and posted the results.  Screenshot is only the error notice from DynDOLOD.  Why is that problematic?  ANyway here is the results from the "copy to clipboard"

[Window Title]
DynDOLOD

[Main Instruction]
Executing LODGenx64.exe returned error code E0434352.

[Content]
"C:\DynDOLOD\DynDOLOD\Edit Scripts\LODGenx64.exe" --inputfile "C:\DynDOLOD\DynDOLOD\Edit Scripts\Export\LODGen_SSE_Test.txt" --logfile "C:\DynDOLOD\DynDOLOD\Logs\LODGen_SSE_Test_log.txt".

Check the file C:\DynDOLOD\DynDOLOD\Logs\LODGen_SSE_Test_log.txt for additional information.

Click on this link for additional explanations and help for this message

[Exit DynDOLOD]

[Footer]
Online Help | Support Forum | Copy message to clipboard

I also already looked at the search you suggested on my own and came with nothing relevant.  I did try the repair .NET app.  It didn't help.

[sheson]

20 minutes ago, Zanderat said:

 

I clicked on said link.  It told me to check the Windows Event Viewer.  Which I did and posted the results.  Screenshot is only the error notice from DynDOLOD.  Why is that problematic?  ANyway here is the results from the "copy to clipboard"

What happened when you follow the second instructions for the error code "See https://www.admin-enclave.com/en/articles/windows/306" and follow what the website says?
That is what solved the problem for 2 people in the past couple days - as you must know after doing the search. 

Screenshots can not be searched.

If error persist after following the suggestion from  the website https://www.admin-enclave.com/en/articles/windows/306 linked from the help page https://dyndolod.info/Help/LODGen for the error code E0434352, execute the command from the error message:

"C:\DynDOLOD\DynDOLOD\Edit Scripts\LODGenx64.exe" --inputfile "C:\DynDOLOD\DynDOLOD\Edit Scripts\Export\LODGen_SSE_Test.txt" --logfile "C:\DynDOLOD\DynDOLOD\Logs\LODGen_SSE_Test_log.txt"

from a command prompt window to see if it shows more information. Copy and paste the text from the command prompt window in that case.

If it fails because the LODGen_SSE_Test.txt is missing let me know.