Guide:Plugins Files

Template:Construction

This guide will walk you through various level of understanding of the "plugins files" used by Skyrim and other recent Bethesda titles, from their definition/content to the interaction they have with each other, and their result in-game.

Global game architecture

Skyrim is firstly its executable, the actual program that run the game. It is located in the installation folder, and goes by the name of TESV.exe

Most of what will actually be seen and used in game isn't included in the executable itself, but in the Data folder placed next to it. The Data folder is where the original Skyrim content is defined, and where mods are usually installed to add their own content.

Files found under the data folder can be categorized into 5 categories :

Plugins Files, the object of this guide. They have the .esp or .esm file extension, and are the files read by the executable.
Ressource files, which can be either "loose", or packed in a Bethesda-specific archive format named BSA, with the .bsa extension. Ressources files aren't accessed/read by themselves, instead they're "pointed at" by the plugin file. (Or pointed by an other ressource, which is in turn pointed by a plugin). There would be a lot to say about all of this, but this is out of this guide's scope.
Hard-coded ressource files, read directly by the executable without being pointed at by plugins. This is the case of the intro video (the Bethesda logo).
Non-relevant files, either put there by user-mistake, or here to serve development purpose (such as scripts-source files, used to create scripts).
Plugin-linked files, those are used when the relevant plugin is loaded without being pointed directly. This is the case of INI files and localization files.

Basics

This part will cover some of the basic concept of plugins files.

If you're looking for more technical informations, refer to the UESP Wiki

Definitions

Here are several definitions that need to be properly understood for understanding the rest of the guide

Form : A form is an "object" (in the programming-sense of the term) used by the game engine. There is multiple type of forms, such as NPCs, weapons, weathers, quests, cell, visual effect, etc... there is a bit more than 120 different types of forms used in Skyrim.

FormID : A FormID is an 8-digit number in hexadecimal identifying a form. Each formID is unique to the form it define. A FormID is completely independent of the family of the form defined.

Record : A record is the set of data defining a form. A record is stored in a plugin.

Run-time : This is the game-launching process. Among several other things, part of this process involve reading records stored in plugins and generating the form they defines for the game.

Group : A group is a collection of records (or of other sub-groups), optimizing the way records are stored/accessed.

Top-Group : A top-group is a group containing a complete family of records defining the same type of forms. If not for an oddity in the vanilla files, one could say that a top-group define a family of form.

Sub-Group : A sub-group is a group contained in a top-group or another sub-group, dividing its parents "family" into "sub-family", used to further optimize complex set of records.

Signature : A signature is the ID-card of a record. A record's signature represent which kind of form this record will define, and thus, which top-group it belong to and what kind of data it should contains. A signature is a 4-character word.

Field : A field is a data from a record. Records are a collection of fields. The number and the nature of those fields depends on the record's type (and thus, signature). Like records, fields also have a signature, though its meaning depend of which kind of record this field belong to.

Plugin : A plugin is a collection of records organized in groups, each of them defining a form.

Header : There is exactly one header per plugin. The header is a special record that doesn't belong to any top-group, and doesn't define a form. It contain technical informations related to the plugin, along with its author's name and a description.

Referencing record : A referencing record is a record that use one (or several) other records in one or more of its fields.

Referenced record : A record referenced by an other one.

Base object : A record defining a form that can be placed in the world.

Reference : A reference (not to be confused with a referenced record) is a record defining an instance of a Base Object in the world. The base object is referenced by this reference (yes, English sometime sound a bit weird in this context). Reference are always under a CELL sub-group.

Interpretation

With what have been defined so far, you should now be able to visualize the content of a plugin like a main folder containing :

A kind of "readme" (header) and several other folders (top-groups).
Each of those folders contain either : Sub-folders (sub-groups), or files (records).
Each of those files have a unique name (formID), and an extension depending on which folder it is placed in (signature).
Those files contains multiple lines of informations (fields), each line starting with a signature as well. Some lines are a path to an other of those files (referencing records to referenced records).

That's it, you can now visualize a complex game's file architecture as a simple folder on your desktop. This is the first towards understanding how the game work.

Random Facts

It is technically possible that two top-groups defining the same type of forms exist in the same file, which is why "top-group" haven't simply been defined as form-category. However in mods, this situation isn't expected and can't be produced with standard modding tools, and will be regarded as potential plugin-corruption. However Skyrim.esm contains a duplicated top-group (but one of the two is empty).

Empty top-groups exists. Left-over from previous Bethesda titles, or unimplemented projects, there is 6 of them (7 with the empty duplicated top-group).

There is very few sub-groups. They're only used to organize world-spaces/cells and their content into logical blocks, and to regroup dialogues info (INFO) under a same dialogue topic (DIAL). (A DIAL is an NPC's line of dialogue, while the INFO is the set of answers the player can choose in the dialogue menu).

A form can exist in-game without being defined by any record. Such form is called a "Created Form", only exist in the savegame, and have its formID index equal to FF. Those forms are similar to references in the sense that they're instance of a base object, but have been created dynamically by the game.

Masters

There is a little bit more into FormID than just 8 hexadecimals numbers. FormIDs are actually constituted of two parts :
The first 2 digits, called "index", and the 6 last digits, which doesn't have a specific name. Here is an other definition :
FormID's Index : Or Index for short, the first two digits of a FormID.

In order to prevent multiple plugins to randomly "overlap" each other in the FormID they use, and cause critical issues, the FormID index won't be used as he is saved in the plugin. It will be interpreted at run-time to be restrained to a specific value. The value depending on the masters of the plugin, and the index itself.

Plugins can have masters. Most plugins for Skyrim actually have at least Skyrim.esm as master, including the official DLCs. Masters of a plugin are listed in its header, and this list of master is sorted. Which give us some more definitions :

Master File : A plugin listed in the list of masters of an other plugin.

Master's index : The place of a master in the masters' list of a plugin (starting to "0", up to "number_of_masters - 1")

The number of masters, and their index in the master list, will determine how the actual FormID's index will be interpreted at run-time :

If the formID's index is equal to the number of masters, the form with this formID is considered as "defined in the plugin"
If the formID's index is superior to the number of masters, it'll be "clamped", i.e. treated as if it was equal.
If the formID's index is inferior to the number of masters : then it match one of the master's index. The form with this formID is considered as defined in the matching master.

Then again, there is two scenarios :

The form is actually defined in the master, in this case the record in the master is overridden.
The form doesn't exist. The record will be considered as "injected".

Which give us 3 new definitions :

New record : A record with a formID's index equal or superior to the number of master. This record define a new form to be used by the game.

Override : A record with a formID's index equal to one of the master's index, if the master also have a record with this formID. This record modify a form already defined by an other plugin.

Injected record : A record with a local formID's index equal to one of the master's index, but the master do not have a record with this formID. This record also define a new form to be used by the game.

In a less abstract fashion :
We've now learned how a new form is created (by defining a new record), and how we can modify an already existing form (by overriding its record in the master it's defined into). We also learned that it's possible to create "injected" records, though the purpose of this remain unclear.

Because definitions are so practical when it come to understanding things in depth, we can extrapolate a little bit and give a few more to carve our knowledge in stone :

Master record : A new record in a plugin that is later referenced into an other.

Multiple overrides : A scenario where two or more plugins would override the same master record.

Critical conflict : A world-end scenario, where a plugin override a record in an uncommon fashion, using its formID to re-define something completely different instead of simply altering it (typically, the override doesn't have the same signature as the master record).

The exhaustive list of overrides contained in a plugin is saved directly in its header, under the ONAM field.

Furthermore, it is now clear that a form, with its unique formID, is only defined once (by a master record), but can be altered by records in plugins dependent on the master. Thus, for the next part of this guide, we will in most case ignore the subtlety of the interpreted formID's index, and consider talking about "record's formID", designating the formID of the record post-interpretation.

Load Order

Since there is multiple plugins, they're loaded in a specific order. The load order can be adjusted manually and is stored in the %LocalAppData%\Skyrim\plugins.txt file.
Note that contrary to popular belief, the loadorder.txt that can be found in this folder isn't an "official" file, but is generated/used by the loadorder utility LOOT (and its predecessor BOSS). This file is commonly used by popular modding utilities such as TES5Edit as reference for the load order if present, in place of the actual plugins.txt.

The position of Skyrim.esm in the load order is hard-coded, its index will always be [00]. Other plugins, including official patches and DLCs are loaded in the order they appear in plugins.txt, and their index attributed accordingly : starting from [01] for the first plugin to the list, and up to [FE] if there is 255 plugins listed.

Since I just introduced the concept of "position in the load order" and named it "index", let's define this right away, in order to keep track of what we're talking about :

Load Order Index : The position of a plugin in the load order. Not to be confused with the formID index definition from the previous chapter. A load order index will usually be written between square brackets like this : [01]

When booting the game, the engine will interpret the formID's index we've been talking about in the previous chapter, in order to attribute them to their respective plugin : New records will have their FormID's index set to their plugin's index in the load order. Overrides and injected records will have their FormID's index set to the corresponding master's index.

Atop of that, if one plugin's master is missing, the game will crash to desktop after the official Bethesda logo. We now know enough to understand that this is because it is trying to find a master in order to give some formID's a proper index, but cannot. To prevent any critical bug, the game shut itself down.

We also know enough to understand that this would be possible for a plugin to be loaded before its master (overrides would just have a higher index than new records in the same plugin). While the game is actually able to load, and some overrides can work properly, several other might not behave as expected, and scenarios of multiple overrides will lead to random results.
Atop of that, such disordered load order is extremely difficult for a human or a basic algorithm to be read, thus it is highly recommended that the load order is... in order, i.e. that any plugin having masters should be loaded after all of them. Utilities such as LOOT and NMM will automatically modify the load order in order to have this behavior. Others such as MO or WB will warn about miss-ordered load order. TES5Edit will just refuse to launch with an error message explaining that the load order is unexpected.

More about masters

There is several reasons why a plugin have a masters, and thus various type of "dependency".

Explicit dependency : This plugin need its master records in order to function properly. The master is in the masters' list of the plugin. This is either because it is overriding some of its master's records in order to make its modifications, or because one or more of its record are referencing at least one of its master's record.

Implicit dependency : An implicit dependency is when a plugin doesn't reference a master, but is designed expecting this master to be present. Typical examples includes :

A mod needing an other mod's ressources, but none of its records.
A mod expecting bugfixes to be in place to function properly.

Forced dependency : This is an implicit dependency, but the expected master have been forced as explicit master (added to the masters' list of this plugin). This is more intended as an user warning, as almost any modding utilities available will warn the user about the missing master if this isn't present. However the game won't have to attribute any specific formID's index linked to this master and thus will proceed normally.

Obsolete dependency : Same as a forced dependency, except that the mod doesn't necessarily expect the other mod to be installed at all, not even implicitly. This is usually an error on the modder's end that happens when loading unnecessary masterfiles in the creation kit alongside the plugin worked-on.

A common misunderstanding in the modding community is that master files are files with a .esm extension, and .esp files are regular plugins. This is wrong in several levels. The file extension is just a convention supposed to identify them, but have no technical meaning.

A master file is a plugin which have the "ESM" flag set in its header. Whether it have a .esp or .esm extension doesn't matter, only the flag is to be taken into consideration. And this is the only actual difference between a master file and a regular plugin. (A plugin with the "ESM" flag set, but which still have the .esp extensions is usually refered to as an "esmyfied plugin").
The game engine have no consideration for this file, nor for the file's extensions. No matter if the "ESM" flag is set, as long as the plugin have either the .esp or .esm extension and is listed in Plugins.txt, it'll be loaded at its index.
.esm files and/or files with the ESM flag set doesn't necessarily need to be loaded before any regular files. This is only a convention followed by the majority of the community.
A regular plugin file (with the .esm extension, and without the "ESM" flag set) can be a master as much as any other plugins, this isn't a limitation.

The only difference between files that have the flag, and those that don't, is the way the Creation Kit is handling them, because it can only edit one plugin at a time. If a file with a master is loaded in the Creation Kit :

If the master have the "ESM" flag : the modder will be able to edit his plugin without modifying anything in the master(s) file(s). No matter if the plugin need records from the loaded master files, all of them will be added in its masters' list (this is how obsolete dependency happens)
If not, then the CK will simply not be able which file it is supposed to edits (since none have the "ESM" flag), and thus break the dependency in order to work on them "separately". This will result in completely corrupted plugins, as every referencing records pointing to a record in the master file will now point to the void.

And this is the end of "the basics". Hopefully this was clear enough. If not, step back and try to understand everything, or the following could be pretty hard to figure out.

The Rule of One, And conflicts

This chapter will walk you through the basic concept of conflicts detection, understanding, and resolution

The source of all problems

The rule of one is the community-given name to the way records are processed (or with our brand new fancy vocabulary, how form behave in the game based on the records defined in the plugins and the load order).

Long story-short : Only the override loaded last in the load order is taken in consideration to generate the form used in-game. This have all kind of ramifications and exceptions, one way or the other.

As discussed in a previous chapter, this isn't 100% accurate, as it is possible to load plugins before they're master and still have their overrides take precedence in game. However, the un-reliability of this behavior, and the fact that it make any consistent analysis near-impossible for a human brain, we will consider for the rest of the guide that the load order is always sorted with masters loaded before plugins dependent on them.

Thus, we can negate the distinction between override loaded last in the load order, and winning override.

Now is the time to introduce conflicts. Not critical conflicts that completely destroy the game like introduced previously, but regular conflicts, those easily resolved by some manual patching.
And in case this haven't been clear enough : Records conflicts are about overrides, and overrides only. If a plugin is constituted of new records only, it doesn't need any kind of record-conflict resolution.

Until now, the guide have been written in hope to be clear and abstract enough to rely on your mental representation only. However, now is a good time to open TES5Edit, load your whole load order into it, and use it as a visual support to follow the next part.

A simple yet concrete example

We vaguely defined the concept of multiple overrides : When two plugins override the same master record. For a simple use case example, let's say :

Skyrim.esm is the master file, and the master record the infamous Iron Sword.
ModA.esp reference Skyrim.esm as master, and override the Iron Sword record in order to change its gold value from 25 to 70.
ModB.esp reference Skyrim.esm as master, and override the Iron Sword record as well, but instead change its Damage from 7 to 5.

The rule of one imply that only the winning override will have its changes applied. So :

If only ModA.esp is loaded, the Iron Sword will have a gold value of 70 instead of 25. This is a case of "override without conflict". The original record from Skyrim.esm isn't used, but this is considered as the expected behavior, because ModA.esp is referencing it as a master.
If both ModB.esp and ModA.esp are loaded, and ModB.esp have a higher priority than ModA.esp then the Iron Sword will have 5 damages, but will still be at the vanilla value of 25. The Iron Sword isn't using the informations from the record in ModA.esp at all.

In case this isn't clear enough : this isn't using the record from Skyrim.esm either. Data others than the damages are identical to those in Skyrim.esm only because they've been copied "as is" in ModB.esp

This is a conflict, due to the "rule of one". There is multiple approach to solving this :

Creating a custom patch. This would be a 3rd plugin, Patch.esp for instance, that would modify both the gold value to 60, and the damage to 5. To make things "clean", our patch would have both ModA.esp and ModB.esp as masters. While having those masters is not technically required in this case, it'll help you remember why this patch have been created in the first place (one reason to have forced dependencies). By loading Patch.esp after ModA.esp and ModB.esp, we would have changes from both mods. This is what common conflict-resolution is about : Copying all the changes to conflicting records into an additional patch to have all the changes taking effect.
On the other hand, since ModA.esp and ModB.esp doesn't do anything else, we could just create the same plugin as we just did, but not add the forced dependencies and just use it instead of the other two. In such case we're not talking about "patching", but recreating a mod from scratch including the features from two others. Although this seems like the best solution in this example, such practice is quickly limited when dealing with actual large-scales mods.
We could also consider that this is ok to "loose" the change from ModA.esp : since ModB.esp is reducing the effectiveness of the sword, it'd make sense that it doesn't worth more than the original value. In which case, ModA.esp is obsolete and be get rid off. However, if ModA.esp was a larger-scale mod (for instance, overhauling the gold value of any weapons in the game), it might be better to just keep the conflict and bear with it.
Trying to be smart : This is usually more something that should be done on the modder's end, but as a power user, you could figure out that ModA.esp purpose is to set the gold value to 10xDamage. So with the damages reduced to 5 in ModB.esp, following ModA.esp pattern, a proper "ModB - A Edition.esp" would have its damage set to 5, and its gold value to 50.

So much blabla for such a simple thing you think... and you'd be right. But hopefully, this little example gave you enough to think about how a proper and complete conflict-resolution procedure can be hair-pulling on an actual load order. We've just been analyzing the consistency between gold value and damages on a single basic weapon.

So, this ITM thing ?

I never mentioned ITM anywhere in this guide up to know, where do you even got this acronym ? Nevermind. You probably heard here and there it was a bad thing. To tell the truth, you're right.

ITM stand for "Identical To Master". In our previous example, the gold value of ModB.esp was identical to the one in its master (Skyrim.esm), thus preventing the changes proposed by ModA.esp to take effect. But this was only on a field-level, there was still a good reason to have ModB.esp in here : It changed the damage to 5.

Now, imagine that ModB.esp wouldn't even to that. Each field identical to its master record (the original Iron Sword from Skyrim.esm). In-game, the form would be the very same as if nothing was installed. The record in ModB.esp is useless. This, is what is called an "Identical To Master record", and usually referred to as ITM. But since ModB.esp is loaded after ModA.esp, it's not only useless : It prevent ModA.esp to have have its change taking effect.

Why would something like this happens ? There could be two reasons :

The most common case of ITM is a beginner modder not at ease with the creation kit, and the ITM being non-intentional. Example : He want to create a custom sword name "Super Sword". He open the creation kit, and change the name of the Iron Sword to "Super Sword". But after testing, he notice that every swords in the world have been renamed "Super Sword". So he read some tutorials about modding, and understand that he need to make a copy of the sword before renaming it to make it "unique". Then, he set the name of the default Iron Sword back to "Iron Sword", and everything look fine. However, even if back in its original state, the record of the Iron Sword is still saved into the plugin, and constitue an ITM. Such ITM should be cleaned and reported to the mod authors so he can clean the original mod as well.
An intentional ITM. For instance, an optional plugin designed specifically to revert the changes on the Iron Sword made by ModA.esp, while keeping the rest of ModA.esp intact (assuming there is more into it than just changing the gold value). This is usually where you recognize modders who actually know what they're doing, and the others.
- If ModA.esp is added as a forced dependency of ModB.esp, then ModB.esp doesn't have an ITM anymore : It actually is different from ModA.esp which is now its master. You can run the cleaning procedure of TES5Edit as much as you want, nothing will be "cleaned", because the "intentional ITM" have been generated properly to serve its purpose, and just its purpose.
- If ModB.esp is actually designed to be "the ultimate vanilla sword", designed to make sure that no matter how heavy and messy your load order is, you'll still have the vanilla sword Bethesda wanted you to have. (This is the only valid reason a mod would generate a complete ITM without targeting a specific mod). In this specific case, a specific workaround have been set up : Modifying the EditorID of the record would prevent it to be an ITM, though the form in-game will be the very same as EditorIDs are unused in-game.

Records merged at runtime

I mentioned exceptions to the rule of one, and this is the one. Some records are excluded from this behavior, they are automatically "conflict-resolved" when the game kick-in.

For references, those records are listed on the AFKMods KnowledgeBase

Note that it is possible that only a handful of fields are merged at runtime, and not the whole record.

Some specifics records will be reviewed in an other chapter of this guide.

Records analysis

Coming... soon ?