DynDOLOD 3.00 Alpha 182

[manguz]

Thanks for MaxLodGen, even if my previous memory issues were my fault for not having page file activated.

This can relieve some stress to our systems

cheers

 

[ButchDiavolo]

Hi Sheson,

I vaguely remember reading on the forums somewhere about a xEdit script you made for large reference errors? If so, where can I find that? I (finally) noticed that some mods I created have large reference errors so I would like to correct them.

Thanks in advance

[z929669]

4 hours ago, Thaumatarge said:

Outside riften this is exactly how it looks, what I am more concerned about is the lack of billboards for when I am in the Riften worldspace. All trees in the cells surrounding Riften lack trees when I generate Ultra trees, but not when I generate regular tree lods. When I generate regular tree lods, it shows up as billboards outside Riften in the Riften worldspace.

Just chiming in on a point: Unticking 'Ultra' gives you "Traditional Tree LOD", which is the vanilla system using one simple diffuse billboard (also created by TexGen I think) and mapped to coordinates. You can still have billboard trees if you tick Ultra. They are just projected onto a simple, flat-plane model in object LOD just like the "3D trees". I always tell people that they are both essentially the same thing but for how simple/complex the models are. Most tree models are low poly trunks with flat, planar branches (or curled ones) showing pictures of branches/leaves. The DynDOLOD Billboard# trees are exactly the same but go a step further by making the whole tree flat in 2-4 planes with pictures of trunk/branches/leaves. So in my opinion, they are both 3D trees and traditional tree LOD is 2D trees ... but I digress

Sheson will shoot me down where I am wrong ...

[sheson]

47 minutes ago, ButchDiavolo said:

Hi Sheson,

I vaguely remember reading on the forums somewhere about a xEdit script you made for large reference errors? If so, where can I find that? I (finally) noticed that some mods I created have large reference errors so I would like to correct them.

Thanks in advance

Large reference bugs triggered by ESP overwriting large reference can not be fixed by a script. The overwrite need to be in a ESM flagged plugin instead if possible.

There is a script Skyrim SE - Generate Large References.pas (get latest version from https://github.com/TES5Edit/TES5Edit/tree/dev-4.1.5/Build/Edit Scripts) included with xEdit to make new references which are added by ESM flagged plugins large references. It's main purpose is for new worlds mods for example

[sheson]

38 minutes ago, z929669 said:

Just chiming in on a point: Unticking 'Ultra' gives you "Traditional Tree LOD", which is the vanilla system using one simple diffuse billboard (also created by TexGen I think) and mapped to coordinates. You can still have billboard trees if you tick Ultra. They are just projected onto a simple, flat-plane model in object LOD just like the "3D trees". I always tell people that they are both essentially the same thing but for how simple/complex the models are. Most tree models are low poly trunks with flat, planar branches (or curled ones) showing pictures of branches/leaves. The DynDOLOD Billboard# trees are exactly the same but go a step further by making the whole tree flat in 2-4 planes with pictures of trunk/branches/leaves. So in my opinion, they are both 3D trees and traditional tree LOD is 2D trees ... but I digress

Sheson will shoot me down where I am wrong ...

This is case is a bit different.

As we know tree LOD does not unload in child words. In the case of Riften (and also Solitude) it happens that a few stuck LOD trees that are just past the walls are not immediately perceived as a bug since in the child world no tree full model is placed, so there is no overlap.

Object LOD unloads properly also in child worlds, so ultra tree LOD - billboard or 3D - does not accidentally show those trees and they might be correctly perceived as missing.

[z929669]

15 minutes ago, sheson said:

This is case is a bit different.

As we know tree LOD does not unload in child words. In the case of Riften (and also Solitude) it happens that a few stuck LOD trees that are just past the walls are not immediately perceived as a bug since in the child world no tree full model is placed, so there is no overlap.

Object LOD unloads properly also in child worlds, so ultra tree LOD - billboard or 3D - does not accidentally show those trees and they might be correctly perceived as missing.

I think I may understand the issue now. Let me test it: Trees in object lod *.bto just outside the walls of the child world unload properly (from player perspective inside child worlds and looking out --so player doesn't see these  trees when inside Riften/Solitude) but traditional tree LOD does not unload and player sees them while inside the child world? Then this makes the player think that LOD isn't working correctly since it's inconsistent from inside the child world?

[skyrim9838]

Sorry if I missed it but does the alpha version not have a mcm menu ? I was updating on my main save and noticed that I don't have a mcm menu for it.

[Mousetick]

Hello,

I'd like to get some clarification on a couple of Large Reference Bugs topics:

1. Large References to PlaneMarker (occlusion plane) STAT objects.

• It seems strange that PlaneMarkers would be large references. What would be the purpose? They don't have models, neither full nor object LODs, right? Is this used as a way for the engine to load them outside the uGrid?
• Is there a way to inspect PlaneMarkers in the console? In cycling through all the objects, I can get the door, wall, building, clouds, underside etc. but not the PlaneMarker, even though I'm right in front of the occlusion plane:

• I'm experiencing very weird behavior with at least one PlaneMarker which is a large reference, flagged by DynDOLOD as overwritten.

Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31B] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))
Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31C] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))

It's defined in Skyrim.esm and overwritten by 3 plugins in the load order:

1. Skyrim.esm places the PlaneMarker.
2. LegacyoftheDragonborn.esm disables it.
3. Occ_Skyrim_Tamriel.esp (from eFPS - Exterior FPS boost, ESM-flagged "out of the box") re-enables it, resizes it and changes its position.
4. Occ_Skyrim-Legacy_of_Dragonborn_patch.esp (from eFPS - Official Patch Hub) re-sizes it and changes its position again.

Without #4, the PlaneMarker is in the wrong position and obscures a building. With #4 everything looks ok.

I tried to flag #4 as ESM in xEdit, which suppressed the DynDOLOD warning, but now the PlaneMarker is in the wrong position again, as if it were loaded from the wrong plugin - which one? I don't know since I can't inspect it with the console. Strangely, it's bugged only when I enter Solitude from the Tamriel worldspace. If I exit from a Solitude building (interior cell) directly to the Solitude child worldspace, the PlaneMarker is not bugged, it's like everything is loaded correctly.

Note: This behavior is observed with or without DynDOLOD plugins enabled in the load order. So the issue is not related to DynDOLOD. DynDOLOD only comes into play because it helpfully warns about the large references.

2. Scope of Large References Bugs.

I want to confirm and be absolutely clear on this point from the online doc:

Quote

Large reference bugs describe the issue that the object LOD for all large references of a cell is not disabled when the full models for the large references are loaded. [...] Again, the entire cell is affected, regardless of the model used by the large reference or its type.

Let's say a cell contains 2 large references, LR1 and LR2. One of them, LR1 is overwritten by at least one ESP plugin, the other, LR2 remaining untouched. LR1 will trigger the LOD bug for both LR1 and LR2. Is this correct?

From this we can conclude that even though an ESP plugin may overwrite only one large reference, its effect may be potentially far more visually "damaging" than one might think if the affected cell contains many large references. In other words, the number of "initially disabled" or "overwritten" warnings issued by DynDOLOD for a given plugin is not a good indication of how seriously bugged the LOD may become because of it. Is this correct?

Thanks.

[sheson]

10 hours ago, z929669 said:

I think I may understand the issue now. Let me test it: Trees in object lod *.bto just outside the walls of the child world unload properly (from player perspective inside child worlds and looking out --so player doesn't see these  trees when inside Riften/Solitude) but traditional tree LOD does not unload and player sees them while inside the child world? Then this makes the player think that LOD isn't working correctly since it's inconsistent from inside the child world?

Yes, for the active cells the child worldspace replaces.

10 hours ago, skyrim9838 said:

Sorry if I missed it but does the alpha version not have a mcm menu ? I was updating on my main save and noticed that I don't have a mcm menu for it.

Nothing changed. The scripts in DynDOLOD Resources are still the same. Test with a new game. Make sure to stay with the plugin limit.

[sheson]

51 minutes ago, Mousetick said:

Hello,

I'd like to get some clarification on a couple of Large Reference Bugs topics:

1. Large References to PlaneMarker (occlusion plane) STAT objects.

• It seems strange that PlaneMarkers would be large references. What would be the purpose? They don't have models, neither full nor object LODs, right? Is this used as a way for the engine to load them outside the uGrid?
• Is there a way to inspect PlaneMarkers in the console? In cycling through all the objects, I can get the door, wall, building, clouds, underside etc. but not the PlaneMarker, even though I'm right in front of the occlusion plane:

• I'm experiencing very weird behavior with at least one PlaneMarker which is a large reference, flagged by DynDOLOD as overwritten.

Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31B] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))
Warning: Overwritten large reference in Occ_Skyrim-Legacy_of_Dragonborn_patch.esp [REFR:0010C31C] (places PlaneMarker [STAT:00000017] in GRUP Cell Temporary Children of SolitudeBluePalaceCourtyard [CELL:00037EE2] (in SolitudeWorld "Solitude" [WRLD:00037EDF] at -13,24))

It's defined in Skyrim.esm and overwritten by 3 plugins in the load order:

1. Skyrim.esm places the PlaneMarker.
2. LegacyoftheDragonborn.esm disables it.
3. Occ_Skyrim_Tamriel.esp (from eFPS - Exterior FPS boost, ESM-flagged "out of the box") re-enables it, resizes it and changes its position.
4. Occ_Skyrim-Legacy_of_Dragonborn_patch.esp (from eFPS - Official Patch Hub) re-sizes it and changes its position again.

Without #4, the PlaneMarker is in the wrong position and obscures a building. With #4 everything looks ok.

I tried to flag #4 as ESM in xEdit, which suppressed the DynDOLOD warning, but now the PlaneMarker is in the wrong position again, as if it were loaded from the wrong plugin - which one? I don't know since I can't inspect it with the console. Strangely, it's bugged only when I enter Solitude from the Tamriel worldspace. If I exit from a Solitude building (interior cell) directly to the Solitude child worldspace, the PlaneMarker is not bugged, it's like everything is loaded correctly.

Note: This behavior is observed with or without DynDOLOD plugins enabled in the load order. So the issue is not related to DynDOLOD. DynDOLOD only comes into play because it helpfully warns about the large references.

2. Scope of Large References Bugs.

I want to confirm and be absolutely clear on this point from the online doc:

Let's say a cell contains 2 large references, LR1 and LR2. One of them, LR1 is overwritten by at least one ESP plugin, the other, LR2 remaining untouched. LR1 will trigger the LOD bug for both LR1 and LR2. Is this correct?

From this we can conclude that even though an ESP plugin may overwrite only one large reference, its effect may be potentially far more visually "damaging" than one might think if the affected cell contains many large references. In other words, the number of "initially disabled" or "overwritten" warnings issued by DynDOLOD for a given plugin is not a good indication of how seriously bugged the LOD may become because of it. Is this correct?

Thanks.

It's anyone's guess why invisible things like markers or planes are listed as large references in the vanilla plugins. The main criteria seems to be STAT/MSTT and object bounds * scale. The script I wrote ignores markers and planes.
I am not aware of how one could be making occlusion planes/boxes visible/clickable in the game. Use CK.

The last plugin in the load order to change a record wins. LOD or being marked as large reference does not change that. So if something is out of place, it is because of the last plugin. Check how the game loads plugins and which plugin is the last to modify a record with xEdit.

Correct, if the reference bug is triggered in a cell, the LOD for all large references in the cell does not unload.

[Mousetick]

Thanks for the quick answers.

41 minutes ago, sheson said:

I am not aware of how one could be making occlusion planes/boxes visible/clickable in the game. Use CK.

Yeah but CK only shows a static view of the cell  I was looking for a way to check if a PlaneMarker is loaded and from where, at runtime.

45 minutes ago, sheson said:

The last plugin in the load order to change a record wins. LOD or being marked as large reference does not change that. So if something is out of place, it is because of the last plugin. Check how the game loads plugins and which plugin is the last to modify a record with xEdit.

I managed to fix the PlaneMarker weird loading issue by applying a combination of the same remedies with xEdit as other large references: 1) flag the ESP as ESM, 2) overwrite the REFR in a custom ESM patch and bury it underground, and 3) copy the REFR as a new record in a custom ESM patch so that it's no longer an overwritten large reference.

This particular overwritten large reference looked suspicious: the ESP overwriting it not only resizes and repositions it, it also puts it within a different parent cell. I suppose the engine may have been quite confused by this hence the weird behavior.

Which brings some new remarks and questions:

• The whole large reference system looks pretty fragile and not at all compatible with modding. Mod authors should be a lot more aware of it and avoid overwriting large references, opting to bury existing ones underground and create new ones instead.
• The issue is how can mod authors be more aware of large references when they are not clearly made visible by the tools like xEdit or CK?
• If mod authors want to take advantage of the large reference system, they should create new references in an ESM and use your script to generate the RNAM list of the worldspace, rather than modifying existing large references.
• Overwriting large references creates more issues than just LOD unloading: What if the plugin resizes it so that its bounds no longer satisfy the "large" criteria? We end up with a unnecessarily bugged "small" reference. What if the plugin moves it to a different cell? Which cell is now bugged - the original one, the new one, both?

[sheson]

1 hour ago, Mousetick said:

Thanks for the quick answers.

Yeah but CK only shows a static view of the cell  I was looking for a way to check if a PlaneMarker is loaded and from where, at runtime.

I managed to fix the PlaneMarker weird loading issue by applying a combination of the same remedies with xEdit as other large references: 1) flag the ESP as ESM, 2) overwrite the REFR in a custom ESM patch and bury it underground, and 3) copy the REFR as a new record in a custom ESM patch so that it's no longer an overwritten large reference.

This particular overwritten large reference looked suspicious: the ESP overwriting it not only resizes and repositions it, it also puts it within a different parent cell. I suppose the engine may have been quite confused by this hence the weird behavior.

Which brings some new remarks and questions:

• The whole large reference system looks pretty fragile and not at all compatible with modding. Mod authors should be a lot more aware of it and avoid overwriting large references, opting to bury existing ones underground and create new ones instead.
• The issue is how can mod authors be more aware of large references when they are not clearly made visible by the tools like xEdit or CK?
• If mod authors want to take advantage of the large reference system, they should create new references in an ESM and use your script to generate the RNAM list of the worldspace, rather than modifying existing large references.
• Overwriting large references creates more issues than just LOD unloading: What if the plugin resizes it so that its bounds no longer satisfy the "large" criteria? We end up with a unnecessarily bugged "small" reference. What if the plugin moves it to a different cell? Which cell is now bugged - the original one, the new one, both?

Moving large references to different cells probably triggers bugs regardless of plugin type. I am pretty sure I tested it, but are not certain about the result anymore.

I have reported these bugs first time end of 2016. DynDOLOD supports SSE since version 2.22 that came out March 2017 and includes information about the large reference bugs and reports overwritten large references since then. Since half a decade.

I consider every mod that triggers large reference bugs as (visually) incompatible with Skyrim Special Edition since the facts and how to work around them are public knowledge. Hence the beta status of DynDOLOD for SSE (and for VR since it supports large references).

The first version of the xEdit large reference generation script exists since 2017 as well IIRC.

DynDOLOD fixes large reference bugs triggered by size changes by setting the threshold value to 1.

Be honest, first reaction to CK reporting errors for the vanilla game files is to disable them. Problem is, now you do not see errors in modding plugins/mod either.

[Mousetick]

The following is long and broad, so feel free to move the whole conversation into a separate topic if you think it doesn't belong here.

9 hours ago, sheson said:

Be honest, first reaction to CK reporting errors for the vanilla game files is to disable them. Problem is, now you do not see errors in modding plugins/mod either.

Are you saying that CK can properly detect overwritten large references and notify the user, but mod authors disable this option? I know very little of CK. I just looked at it and played with it a bit, I couldn't find anything related to large references. I also tested moving a large reference and saving the overwrite in an ESP. CK happily obliged without a blip.

9 hours ago, sheson said:

I consider every mod that triggers large reference bugs as (visually) incompatible with Skyrim Special Edition since the facts and how to work around them are public knowledge.

I understand your position. It must be very frustrating to be dealing with those issues as long and as best as you have been. However I'm not sure it's fair to put the blame on the mod authors for being ignorant or on the mods for being defective.

1. There is no indication in xEdit or CK that a reference is marked as large in the master, there is no error or warning triggered in those tools when such reference is overwritten either. So mod authors are "blind" to large references. How can someone tell beforehand that modifying a particular reference that looks like any other is going to cause problems?

The RNAM list is hidden by default in xEdit, and even if the option to show it is turned on, the information available is unusable by a mod author. It's a huge "blob" that is stashed away in the parent worldspace, disconnected from the references themselves, and impossible to humanly parse or cross-reference with the references in a given cell.

Would you agree with this, or am I missing something?

2. From an anecdotal perspective as a casual mod user, it seems that more often than not, I'm sorry to say, mod authors are not familiar with DynDOLOD. Looking at the posts in the comments section on Nexus, user questions like "Do I need to re-run DynDOLOD with this mod?" or "Does this mod work with DynDOLOD?" receiving answers from mod authors like "What is DynDOLOD?" or "Maybe, I don't know since I don't use it" is a pretty common sight.

Why that is, I don't know. In an ideal world, everyone involved in modding Skyrim should be familiar with DynDOLOD  But that's not the case, for whatever reasons. Perhaps the large reference issues would be better known if they were considered and publicized separately from DynDOLOD. They exist independently of DynDOLOD but mod authors may be confusing them as a DynDOLOD-specific or DynDOLOD-induced issues.

3. On second thought, I realized that the suggested "best practices" for mod authors dealing with large references (splitting mod into ESM+ESP, burying them, creating new references...) don't appear to be a viable solution. They're merely workarounds, and pretty complicated to implement at that. Beside the extra care and work required, this scheme creates its own set of problems in regards to compatibility between mods and conflict resolution.

For example, ModA and ModB each modify the same large reference. Let's say the object is a big boulder: ModA disables it to place its own building at that location, and ModB changes its dimensions and position slightly for "enhanced" landscaping. Following best practices, both mods bury the original boulder, and ModB places its own new boulder. Now what happens when both mods are present in the load order? There is ModB's boulder clipping into ModA's building.

What used to be a simple load order priority conflict, becomes a complicated issue requiring visual inspection in the console or in CK to figure out where does the boulder come from and what to do with it, in order to resolve and patch the conflict. The complexity increases exponentially as the number of mods increase, since with this scheme, all mods "win" with their new references.

Furthermore, the workarounds are not readily applicable to dynamically swapped references, such as with Base Object Swapper.

I'd love to know your thoughts on the following ideas:

1. Would you consider making the "preflight" validation phase of DynDOLOD a separate tool that could be used independently of DynDOLOD? I'm talking about the validation phase that DynDOLOD performs after loading all the plugins and before generating the LODs and plugins per se, which is then used to produce the HTML diagnostics stored in the Summary folder. This would include the Large Reference Bugs warnings of course, but also all the other useful warnings.

I often find myself wishing to be able to get these diagnostics but renouncing because they require running the full DynDOLOD tool and immobilizing my PC for a while. I did try running DynDOLOD and checking off boxes to speed things up, but all I got was nothing: DynDOLOD simply quit, saying something like "All done, thank you, bye" 

I think such tool could become an essential part of the modding toolkit for mod users and authors alike, similarly to Mod Organizer, xEdit, LOOT or BethINI, but without the pretty steep learning curve of the full DynDOLOD suite. It could also serve as a "trojan" to help familiarize people with large reference issues and to promote DynDOLOD via messages like "DynDOLOD can automatically fix this issue", with pointers to more information and downloads.

2. It would be helpful if xEdit could visually indicate large references by displaying them in a different color or style, so that mod authors are no longer blind.

3. As previously discussed, a more robust and generic solution to the large references issues would be at engine level, via a DLL runtime patcher, by automatically skipping overwritten large references, from a list previously generated by DynDOLOD.

Pushing this idea further, an even better and cleaner solution would be for the DLL runtime patcher to build the in-memory RNAM list of large references from scratch on the fly (and maybe cache it if it's too expensive to do it every time the worldspaces are loaded). In other words, it would work a bit like your xEdit script, but at runtime rather than at plugin creation time, and it would automatically skip those that are overwritten by ESP plugins.

With this approach, large references would always be correct and "clean", no matter the load order, no matter the final dimensions of the overwritten references, it could work cooperatively with Base Object Swapper, and finally there would no longer be a need for DynDOLOD to verify or attempt to fix them. It wouldn't matter if mod authors are aware of them or are specifically working around them. It might also make large references on the fly for ESM plugins that don't explicitly define them. It could make fLargeRefMinSize a runtime configuration setting rather than a fixed setting.

Thanks for reading and again, sorry for the long post. I hope this provided some constructive food for thought.

[z929669]

58 minutes ago, Mousetick said:

Thanks for reading and again, sorry for the long post. I hope this provided some constructive food for thought.

Interesting insights. I would use such a tool(s).

[sheson]

2 hours ago, Mousetick said:

1. Would you consider making the "preflight" validation phase of DynDOLOD a separate tool that could be used independently of DynDOLOD? I'm talking about the validation phase that DynDOLOD performs after loading all the plugins and before generating the LODs and plugins per se, which is then used to produce the HTML diagnostics stored in the Summary folder. This would include the Large Reference Bugs warnings of course, but also all the other useful warnings.

Check dynamic LOD only for a quick check of things.