The case for flagging every plugin as an ESM

[Daemonjax]

If you use xEdit to flag every plugin in your load order as an ESM, you gain the following benefits:

1) "Fixes" the _Overwritten_ large reference bug since ESM's can overwrite large references.

2) "Fixes" the esp temporary reference bug*

* All temporary references in esp are treated as persistent references which are kept in memory forever (loaded before you get to the main menu) and there's a relatively low max of 10^20 in memory (CTD when exceeded)

Downside(s):

Seriously I can't think of any.

 

What am I missing here?  Of course it would be ideal to have the game or MO2/Vortex/Whatever automatically promote esp's to esm when loading the game, but that functionality doesn't exist.

 

EDIT: I'm working on a program that will wrap (I believe I can do that part simply with a batch file) around the exe which flags non-esm's as esm and then puts them back after the game exits.  

Edited February 13, 2022 by Daemonjax
words

[Mousetick]

In theory this should work.

I've flagged a few mods as ESM in xEdit to fix the overwritten large reference issue with LODs, but only on an as-needed basis.

If you can make a tool that automates the ESM-flagging to the whole load order that would be useful. The problem with flagging mods piecemeal is that one needs to keep track of which ones have been flagged, so they can be re-flagged in case they're updated later.

I don't see how it would be necessary to redo and undo this process every time the game starts and quits. What would be the purpose? It could be done once and for all.

[DoubleYou]

Or you could simply set TreatAllModsAsMasters=True in SSE Engine Fixes.

My gut tells me this is a bad idea, but I don't really have a good argument against it. If you ask on the xEdit Discord, they could probably give you a better idea about this.

[Mousetick]

9 hours ago, DoubleYou said:

Or you could simply set TreatAllModsAsMasters=True in SSE Engine Fixes.

Oh wow, that certainly makes things easier. Not any safer, mind, but it does exactly what the OP had in mind.

9 hours ago, DoubleYou said:

My gut tells me this is a bad idea, but I don't really have a good argument against it.

I'm with you DY. I think it's a good idea in principle, as it solves both issues mentioned in the OP, but it could be troublesome in practice.

My knowledge of references is quite limited, but my understanding is:

• Persistent references are permanently loaded in memory as soon as the game starts and never released until it exists.
• Temporary references are allocated and released in memory whenever their parent cell is loaded or unloaded respectively.
• All references loaded from an ESP plugin (vs. ESM) are treated as persistent by the engine, whether they're defined as temporary or persistent in the plugin.

So making an ESP plugin into an ESM could expose "use-after-free" bugs in some plugins. For instance a Papyrus script triggered by a background quest could (incorrectly) access a temporary reference while its parent cell is not loaded. In an ESP this would not be an issue since the temporary reference is actually persistent in memory. Once the ESP is loaded as an ESM however, if the parent cell is not loaded, the reference could point to another object, causing unexpected behavior, or in the worst case cause a memory access violation CTD.

I'm assuming CK has checks in place to ensure all references potentially used outside of their parent cells are made explicitly persistent in the plugin. However there might be cases with manually written scripts accessing references directly without using properties/aliases or with plugins manually edited in xEdit, where such validation checks are not enforced.

In the best scenario where the plugins and their scripts are defining and accessing temporary vs. persistent references properly, it should make no difference whether they're loaded as ESP or ESM. That being said, it's probably a bad idea to switch ESP to ESM mid-game, due to potentially unknown differences in how temporary vs. persistent references are stored in a save file.