ACCEPTED Worldspace Transition Tweaks (by AndrealphusVIII)

[TechAngel85]

On 7/2/2022 at 10:05 PM, z929669 said:

Technically, a mountain range would obscure it at RL distance analogous to game world due to curvature of the Earth. If you were on top of the mountain range, you would see it though.

That said: It IS a fantasy game ... but we DO always talk about 'realism' too, and all we have is our reality as a basis

True, but that is also going to depend on distance to said mountain range and your own altitude. It'll never resemble RL so the question is to either place it or not. I'm fine with either, to be honest. I just wanted to make a case for opposite side so that we have all our basis covered. 

[z929669]

27 minutes ago, TechAngel85 said:

True, but that is also going to depend on distance to said mountain range and your own altitude. It'll never resemble RL so the question is to either place it or not. I'm fine with either, to be honest. I just wanted to make a case for opposite site so that we have all our basis covered. 

I pretty much agree and am fine including this one, plume over mountains or not. @DoubleYou Is this a deal breaker for you?

[DoubleYou]

Not a deal breaker, just something to keep in mind. The other improvements far outweigh this IMHO.

[z929669]

Thanks. Marked accepted for 2.1.0. I will create the mod page and add this one to the guide. It's been a while since I installed this, and I think it's just a straight up install with no fussing about.

[Sacralletius]

Cool. Good to know. Since there's a debate with some people wanting the plume to be visible and others don't. I'm probably making it optional.

[Sacralletius]

I've updated the mod with some additional customizable settings for the plume.

[TechAngel85]

17 hours ago, Sacralletius said:

I've updated the mod with some additional customizable settings for the plume.

If you could contact one of the Admin or Step Staff on Nexus, we'll set you up with the Mod Author badge and user group on the forums.

[z929669]

Hey @Sacralletius,

I'm working on some of our latest patches and wondering about the late-loading WTT ESL and it's purpose. I did see your note that it "ensures all worldspace data gets forwarded", but I guess I don't understand.

We'd rather provide the option to hide the plume from Tamriel via a optional plugin patch than deal with the console (it's just simpler for our guide instructions and support headaches). When I look at the ESL though, the only conflict between your ESL and the ESM-flagged ESP is that one single record.

So two questions on this:

1. This one conflict seems to disable the plume from DLC2SolsteimWorld PoV (04000800) just to make it visible from that PoV with the ESL. What's happening here?
2. Why are all of the other worldspace blocks/records included in the ESL when they are identical to those in the ESM (and they have no intervening conflicts with other mods in our lineup, which tells me that intervening conflicts would be very rare for these records). I'm guessing this has something to do with PoV from either side (Tamriel vs Solstheim), but I don't understand it!

---

Now to the plume visibility from Tamriel (0000003C) ... we are hiding it via optional plugin like so:

It seems to work fine, but is there any theoretical issue you know of doing it this way? I ask because you added a method to do it from the console rather than like this.

[Sacralletius]

Quote

This one conflict seems to disable the plume from DLC2SolsteimWorld PoV (04000800) just to make it visible from that PoV with the ESL. What's happening here?

It's a bug. I think I forgot to forward the initially disabled flag to the ESL, during the last update. I will do so now and upload an update later today. Sorry for the inconvenience. Thanks for telling me though.

Quote

Why are all of the other worldspace blocks/records included in the ESL when they are identical to those in the ESM (and they have no intervening conflicts with other mods in our lineup, which tells me that intervening conflicts would be very rare for these records). I'm guessing this has something to do with PoV from either side (Tamriel vs Solstheim), but I don't understand it!

The main purpose of the ESL is to make sure all the worldspace data wins any conflict. That data is needed for DynDOLOD to generate LODs. (will show an error messsage otherwise) In particular the data within the WRLD records, involving the MHDT - Max Height Data and Object Bounds (NAM0 - Min and NAM9 - Max) subrecords.

In addition, the edits to the cells within the blocks are needed to forward the region data (for sounds) and location data. (WTT_SkyrimToSolstheimLocation "Skyrim To Solstheim Transition Location" [LCTN:0606F26D] for instance) The latter serves as a failsafe for when the player gets outside the bounds of the map through any means, bypassing the PLDs. There's a dormant constant effect spell (WTT_ControlSpell "WTT Control Spell" [SPEL:0606F273]) to check if the player is within these areas, and if they are, the spell will trigger the PLD script to teleport them over.

Quote

It seems to work fine, but is there any theoretical issue you know of doing it this way? I ask because you added a method to do it from the console rather than like this.

Personally, to hide the plume, I'd recommend making an override patch that sets the global value of WTT_SkyrimVvardenfellPlumeMaxDistance to around 100. (This will make sure the constant effect spell will never start.) That way the player can still use console commands to change this later on, if they chose to, without the need to disable any plugins, eliminating the need to rebuild any dynamic patches that would require said plugin as master. That's how I would do it. I'm not sure whether or not this is actually the best way though.

I hope this answers your questions.

If you have any suggestions/ideas as to how to improve the mod, feel free to share. I'm always open to improve. :)

[z929669]

5 hours ago, Sacralletius said:

It's a bug. I think I forgot to forward the initially disabled flag to the ESL, during the last update. I will do so now and upload an update later today. Sorry for the inconvenience. Thanks for telling me though.

The main purpose of the ESL is to make sure all the worldspace data wins any conflict. That data is needed for DynDOLOD to generate LODs. (will show an error messsage otherwise) In particular the data within the WRLD records, involving the MHDT - Max Height Data and Object Bounds (NAM0 - Min and NAM9 - Max) subrecords.

In addition, the edits to the cells within the blocks are needed to forward the region data (for sounds) and location data. (WTT_SkyrimToSolstheimLocation "Skyrim To Solstheim Transition Location" [LCTN:0606F26D] for instance) The latter serves as a failsafe for when the player gets outside the bounds of the map through any means, bypassing the PLDs. There's a dormant constant effect spell (WTT_ControlSpell "WTT Control Spell" [SPEL:0606F273]) to check if the player is within these areas, and if they are, the spell will trigger the PLD script to teleport them over.

Personally, to hide the plume, I'd recommend making an override patch that sets the global value of WTT_SkyrimVvardenfellPlumeMaxDistance to around 100. (This will make sure the constant effect spell will never start.) That way the player can still use console commands to change this later on, if they chose to, without the need to disable any plugins, eliminating the need to rebuild any dynamic patches that would require said plugin as master. That's how I would do it. I'm not sure whether or not this is actually the best way though.

I hope this answers your questions.

If you have any suggestions/ideas as to how to improve the mod, feel free to share. I'm always open to improve.

This is very helpful. I'm going to let this sink in a bit and read over your WTT content again. Thank you!

[z929669]

9 hours ago, Sacralletius said:

The main purpose of the ESL is to make sure all the worldspace data wins any conflict. That data is needed for DynDOLOD to generate LODs. (will show an error messsage otherwise) In particular the data within the WRLD records, involving the MHDT - Max Height Data and Object Bounds (NAM0 - Min and NAM9 - Max) subrecords.

In addition, the edits to the cells within the blocks are needed to forward the region data (for sounds) and location data. (WTT_SkyrimToSolstheimLocation "Skyrim To Solstheim Transition Location" [LCTN:0606F26D] for instance) The latter serves as a failsafe for when the player gets outside the bounds of the map through any means, bypassing the PLDs. There's a dormant constant effect spell (WTT_ControlSpell "WTT Control Spell" [SPEL:0606F273]) to check if the player is within these areas, and if they are, the spell will trigger the PLD script to teleport them over.

 

I checked 06F26D, and this isn't forwarded into the ESL. Did you intend otherwise? I'm not challenging your logic, but rather trying to understand by relating to what I'm seeing:

I also did some testing for DynDOLOD errors by creating/enabling a test plugin and disabling your ESL then ran DynDOLOD:

1. Forwarded a few of the REFR records (persistent and temporary) into a test ESP (NULL flagged, "WTT test.esp")
2. Made edits to a few of these records:
   1. Left most of the persistent records alone (just forwarded into ESP)
   2. Added Deleted, Initially Disabled flags to 06A131 (persistent)
   3. Added Initially Disabled flag to 06A132 (persistent)
   4. Removed Initially Disabled flag from 06F294 (persistent)
   5. Added Initially Disabled flag, set Enable parent to opposite of player, set z to -30000 on 034355 (temporary) - essentially created a UDR as I understand it
   6. Added Initially Disabled flag to 06F295 (temporary)

Here's the overview:

The DynDOLOD warning/error report only identified the Deleted reference (#2) as an issue:

Error: Deleted reference WTT test.esp [REFR:1006A131] (in GRUP Cell Persistent Children of [CELL:00000D74] (in Tamriel "Skyrim" [WRLD:0000003C]))

Since the other worldspace records I tested in the test ESP did not trigger any DynDOLOD large-reference warnings, I'm thinking that your ESL isn't really needed to suppress DynDOLOD errors. Admittedly, I find the Large Reference triggers and issues a bit confusing still, so have a look at my answered questions on this topic if you want to learn more. It may make more obvious sense to you.

Although not all/any of the records I forwarded are necessarily large references, DynDOLOD should call out issues regardless. I was expecting a couple of the records I forwarded to trigger DynDOLOD errors (e.g., #3 and #5), but they didn't. Maybe someone who understands this better like @Mousetick or @sheson can provide some instructive insights. I haven't had my "aha" moment yet on LRs (but I'm close I think). I assume it has something to do with ESM setting this in memory at runtime 'permanently' versus ESP (ESL?) doing so dynamically/temporarily, but this is a wild guess.

Ultimately, I'm questioning the current implementation of the ESL and if it's necessary (I had thought one should be creating an ESM to hold the LRs and an ESP to hold their overrides into distinct, new records). Take no advice from me at this point!

... next, I will see about implementing the plume option as you suggested. Haven't taken that up yet, so I cannot speak to it.

[Mousetick]

Edit: This was meant as a reply to Z's post above but I forgot to quote it and I'm unable to do so now for technical reasons.

I don't use this particular mod*, so take my comments with a grain of salt.

• I don't know the exact reason for the ESM, but it's probably not for alleviating LR issues. The main mod content being ESM is a good thing anyway, for whatever reason, so let's just not question it.
• This mod's worldspace and cell settings are critical to its function. They're also used by DynDOLOD for terrain LOD and occlusion, as such they need to be correct.
• Since the main mod content is ESM, it's going to be sorted high in the loader order, as such it's likely other mods will be loaded below it. Those other mods below which make changes in the same cells as this mod, will clobber this mod's specific worldspace and cell settings, because they don't know anything about it.
• The ESL-flagged ESP is there to ensure those worldspace and cell settings are correct. It simply forwards the worldspace and cells records of this mod's ESM and is meant to be loaded very very low (i.e. with high priority) so that it wins over any other mod that have clobbered them. LOOT may have some metarules to automatically sort it very low, or it may need to be manually sorted very low - I don't know.
• As an analogy, consider another mod you're familiar with: ACMOS. It's the same principle: it must be loaded very low because it makes changes to worldspace records that must win over all other mods. Except ACMOS is a single ESP plugin, so it doesn't need another plugin to forward its changes.
• It looks like you assumed the ESM/ESP split was LR-related, which sent you down a LR rabbit hole. The 'WTT test.esp' experiment yielded no interesting results because there were no LRs to begin with.
• The Location records are not forwarded by this mod's ESP because they don't need to be: they're specific, unique to this mod. Nothing else is going to override them. On the other hand, each cell record can have a location property, pointing to a Location record. It's this location property that needs to preserved, among other settings, and forwarded with the highest priority, by the ESP.
• Even if there are no conflicts with the ESM in your current load order (STEP guide), the ESL should nevertheless still be included, enabled and sorted very low, to avoid issues for users adding their mods to the list.

(*) Though I do use many other excellent mods by Sacralletius

Edited December 28, 2022 by Mousetick
Replying to Z

[z929669]

15 minutes ago, Mousetick said:

• It looks like you assumed the ESM/ESP split was LR-related, which sent you down a LR rabbit hole. The 'WTT test.esp' experiment yielded no interesting results because there were no LRs to begin with.

Thanks .. but it shouldn't matter if they are LRs or not. They are REFR records in worldspace, so shouldn't some of my changes trigger the DynDOLOD warning regardless? This is the gist of the whole debate around which some people cry foul for DynDOLOD reporting non-issues ... the issue being "general bad practice" and making the LR bug possible whether manifest or not.

I don't question the ESM in the least. I was just wondering about the ESL and the need to forward the ESM records. I also forgot to acknowledge that it's not a light master but just a light plugin (ESM flag not set). So I'm probably fixating on it being an ESL and was incorrectly presuming it to be a light master.

So regarding ESM/ESM-flagged plugins: I think I understand the significance that ESM records are locked into memory at runtime and for the duration, while non-master changes are transient in memory, correct? Using a master-flagged plugin ensures that the changes are 'baked' at runtime, I assume? Which is important for this particular mod's functionality (and any mod that adds new worldspace records or possibly 'new' anything)?

[Mousetick]

1 hour ago, z929669 said:

They are REFR records in worldspace, so shouldn't some of my changes trigger the DynDOLOD warning regardless?

Absolutely not. What kind of warning were you expecting then?

Look at the list of warnings and errors that can be reported by DynDOLOD: https://dyndolod.info/Messages. All messages which have 'Reference' in their title, except 3, only apply to Large References.

1 hour ago, z929669 said:

So regarding ESM/ESM-flagged plugins: I think I understand the significance that ESM records are locked into memory at runtime and for the duration, while non-master changes are transient in memory, correct? Using a master-flagged plugin ensures that the changes are 'baked' at runtime, I assume? 

Actually it's the other way around. There are 2 kinds of references: persistent or temporary. Persistent is a flag in the REFR header, which can be seen in xEdit.

ESM:

• Persistent references are loaded permanently in memory and processed* by the engine whether they are placed in active cells or not. The object they represent can be accessed and used at any time from anywhere in the world.
• Temporary references are loaded only when the cell in which they are placed is active/loaded, and are unloaded otherwise. The object or NPC they represent only exist when their parent cell is loaded, and they don't exist otherwise. Attempting to access an unloaded temporary reference leads to an engine crash.

Non-ESM:

• Temporary references behave like persistent references.
• Furthermore, a temporary reference created by an ESM and overridden by a non-ESM loses its temporary status.

There are only a few types of objects that need to be persistent. All others should be (truly) temporary. It doesn't make sense generally for a tree or a rock to be persistent, or for an NPC staying inside a building, for example. The engine has a hard limit on the number of references that can be loaded at the same time (~1 million). The engine processes the AI packages of all persistent NPCs and creatures even if they're far away from the player. References overridden by non-ESM cause LR issues.

The engine even supports a finer grained level of reference persistence: persistent location. A temporary reference can be assigned a persistent location: it remains loaded as long as the player is in that location, and behaves like a temporary reference otherwise. For example, a temporary NPC reference could be given the persistent location of Solitude. While the player is anywhere in Solitude, the NPC is loaded and active. Once the player exits Solitude, the NPC is unloaded.

Mods that add or change references should be designed and released as ESMs. The fact that 99% of mods containing references are released on Nexus in non-ESM form is an anomaly. CC content, which is nothing more than officially released and supported mods, is all ESM for a good reason.

2 hours ago, z929669 said:

Which is important for this particular mod's functionality (and any mod that adds new worldspace records or possibly 'new' anything)?

I don't know the specific reason behind this mod being ESM.

It's not "important" or "necessary" to be ESM. Everything still "works" more or less the same with non-ESM, from a user point of view. But it's an inefficient and unintended way to create or modify references, not in the way the engine is designed to be used.

(*) But not rendered.

[Sacralletius]

Alright, so there are a few more questions that were being raised. I'll try to answer them.

Why is the main ESP flagged as an ESM?

Because it contains several hundreds of thousands of new references. (many references from Solstheim copied to Skyrim and vice versa.) In a normal non-ESMified ESP, all of those references count as persistent and will contribute towards the ~1 million persistent reference limit.

Why the ESPFE?

As the ESMs load very earlier in one's load order, it is very likely that any mod that makes edits to either Tamriel or DLC2SolstheimWorld resets the edits I made. These edits are critical to win each conflict:

Within the WRLD records: MHDT - Max Height Data and Object Bounds (NAM0 - Min and NAM9 - Max). If any mod reverts these and the user attempts to run DynDOLOD it will cause an out of bounds error.

Quote

I checked 06F26D, and this isn't forwarded into the ESL. Did you intend otherwise? I'm not challenging your logic, but rather trying to understand by relating to what I'm seeing:

You're looking at the wrong place. I didn't mean direct Location record, but instead any cell that references them. I marked a group of cells by giving them a location and if the player somehow gets to that location they will be teleported to corresponding worldspace.

The ESPFE is absolutely essential unfortunately AFAIK. I cannot be skipped.

Quote

Forwarded a few of the REFR records (persistent and temporary) into a test ESP (NULL flagged, "WTT test.esp")
Made edits to a few of these records:
    Left most of the persistent records alone (just forwarded into ESP)
    Added Deleted, Initially Disabled flags to 06A131 (persistent)
    Added Initially Disabled flag to 06A132 (persistent)
    Removed Initially Disabled flag from 06F294 (persistent)
    Added Initially Disabled flag, set Enable parent to opposite of player, set z to -30000 on 034355 (temporary) - essentially created a UDR as I understand it
    Added Initially Disabled flag to 06F295 (temporary)

Could you send me the ESP containing your edits? It's hard to go off of descriptions. But from what I see adding a Deleted flag is NEVER a good idea.

Never mind, misread.