Guide:Plugins Files

Template:Construction

This guide will walk you through various level of understanding of the "plugins files" used by Skyrim and other recent Bethesda titles, from their definition/content to the interaction they have with each other, and their result in-game.

Global game architecture

Skyrim is firstly its executable, the actual program that run the game. It is located in the installation folder, and goes by the name of TESV.exe

Most of what will actually be seen and used in game isn't included in the executable itself, but in the Data folder placed next to it. The Data folder is where the original Skyrim content is defined, and where mods are usually installed to add their own content.

Files found under the data folder can be categorized into 5 categories :

Plugins Files, the object of this guide. They have the .esp or .esm file extension, and are the files read by the executable.
Ressource files, which can be either "loose", or packed in a Bethesda-specific archive format named BSA, with the .bsa extension. Ressources files aren't accessed/read by themselves, instead they're "pointed at" by the plugin file. (Or pointed by an other ressource, which is in turn pointed by a plugin). There would be a lot to say about all of this, but this is out of this guide's scope.
Hard-coded ressource files, read directly by the executable without being pointed at by plugins. This is the case of the intro video (the Bethesda logo).
Non-relevant files, either put there by user-mistake, or here to serve development purpose (such as scripts-source files, used to create scripts).
Plugin-linked files, those are used when the relevant plugin is loaded without being pointed directly. This is the case of INI files and localization files.

Basics

This part will cover some of the basic concept of plugins files.

If you're looking for more technical informations, refer to the UESP Wiki

Definitions

Here are several definitions that need to be properly understood for understanding the rest of the guide

Form : A form is an "object" (in the programming-sense of the term) used by the game engine. There is multiple type of forms, such as NPCs, weapons, weathers, quests, cell, visual effect, etc... there is 120 different types of form. (think "family")

FormID : A FormID is 8-digit number in hexadecimal identifying a form. Each formID is unique to the form it define. A FormID is completely independent of the family of the form defined.

Record : A record is the set of data defining a form. A record is stored in a plugin.

Group : A group is a collection of records (or sub-groups) defining the same type of form in a plugin.

Top-Group : A top-group is a group containing the "family" of records defining the same type of forms. If not for an oddity in the vanilla files, one could say that a top-group define a family of form.

Sub-Group : A sub-group is a group contained in a top-group or another sub-group, dividing its parents "family" into "sub-family".

Signature : A signature is the ID-card of a plugin. A record's signature represent which kind of form this record will defines, and thus, which top-group it belong to and what kind of data it should contains. A signature is a 4-character word.

Field : A field is a data from a record. Records are a collection of fields. The number and the nature of those fields depends on the record's signature. Like records, fields also have a signature, though its meaning depend of which kind of record this field belong to.

Plugin : A plugin is a collection of records organized in groups, each of them defining a form.

Header : There is exactly one header per plugin. The header is a special record that doesn't belong to any top-group. It carry technical informations related to the plugin, along with its author's name and a description.

Referencing record : A referencing record is a record that use one (or several) other records in one or more of its fields.

Referenced record : A record referenced by an other one.

Base object : A record defining a form that can be placed in the world.

Reference : A reference (not to be confused with a referenced record) is a record defining an instance of a Base Object. The base object is referenced by this reference (yes, English sometime sound a bit weird in this context). Reference are always under a CELL sub-group.

Interpretation

With what have been defined so far, you should now be able to visualize the content of a plugin like a main folder containing :

A kind of "readme" (header) and several other folders (top-groups).
Each of those folders contain either : Sub-folders (sub-groups), or files (records).
Each of those file having a unique name (formID), and an extension depending on which folder it is placed in (signature).
Those files containing multiple lines of informations (fields), starting with a signature as well. Some line are a path to an other of those files (referencing records to referenced records).

That's it, you can now visualize a complex game's file architecture as a simple folder on your desktop. You've made the first step towards understanding how your game work. You can now reward yourself with a cup of coffee, but should definitely avoid alcohol if you intend to read further.

Random Facts

It is technically possible that two top-groups defining the same type of forms exist in the same file, which is why "top-group" haven't simply been defined as form-category. However in mods, this situation isn't expected and can't be produced with standard modding tools, and will be regarded as potential plugin-corruption. However Skyrim.esm contains a duplicated top-group (but one of the two is empty).

Empty top-groups exists. Left-over from previous Bethesda titles, or unimplemented projects, there is 6 of them (7 with the empty duplicated top-group).

There is very few sub-groups. They're only used to organize world-spaces/cells and their content into logical blocks, and to regroup dialogues info (INFO) under a same dialogue topic (DIAL). (A DIAL is an NPC's line of dialogue, while the INFO is the set of answers the player can choose in the dialogue menu).

A form can exist in-game without being defined by any record. Such form is called a "Created Form", only exist in the savegame, and have its formID index equal to FF. Those forms are similar to references in the sense that they're instance of a base object, but have been created dynamically by the game.

Masters

There is a little bit more into FormID than just 8 numbers. They're actually constituted of two parts :
The first 2 digits, called "index", and the 6 last digits called... I don't know, but it doesn't matter. Let's just define this :
Index : The first two digits of a FormID.

In order to prevent multiple plugins to randomly "overlap" each other in the FormID they use, and cause critical issues, the index won't be used "as is". It will be interpreted at run-time to be restrained to a specific value. The value depending on the masters of the plugin, and the index itself.

Plugins can have masters. Most skyrim mods actually have at least Skyrim.esm as master, including the official DLCs. Masters of a plugin are listed in its header. This list of master is sorted. Which give us some more definitions :

Master File : A plugin listed by an other in its list of master files.

Master's index : The place of a master in the master files list of a plugin (starting to "0", up to "number_of_masters - 1")

The number of masters, and their index in the master list, will define how the actual FormID's index will be interpreted :

If the formID's index is equal to the number of masters, the form with this formID is considered as "defined in the plugin"
If the formID's index is superior to the number of masters, it'll be "clamped", i.e. treated as if it was equal.
If the formID's index is inferior to the number of masters : then it match one of the master's index. The form with this formID is considered as defined in the matching master.

Then again, there is two scenarios :

The form is actually defined in the master, in this case the record in the master is overridden.
The form doesn't exist. The record will be considered as "injected".

Which give us 3 new definitions :

New record : A record with a formID's index equal or superior to the number of master.

Override : A record with a formID's index equal to one of the master's index, if the master also have a record with this formID.

Injected record : A record with a local formID's index equal to one of the master's index, if the master do not have a record with this formID.

In a less abstract fashion :
We've now learned how a new form is created by defining a new record, and how we can modify an already existing form by overriding its record in the master it's defined into. We also learned that it's possible to create "injected" records, though the purpose of this remain unclear.

Because definitions are so practical when it come to understanding things in depth, we can extrapolate a little bit and give a few more to carve our knowledge in stone :

Master record : A new record in a plugin that is later referenced into an other.

Multiple overrides : A scenario where two or more plugins override the same master record.

Critical conflict : A world-end scenario, where a plugin override a record in an uncommon fashion, using its formID to define something completely different (typically, the override doesn't have the same signature as the master record).

The exhaustive list of override contained in a plugin is saved directly in its header, under the ONAM field.

Load Order

Since there is multiple plugins, they're loaded in a specific order. The load order can be adjusted manually and is stored in the %LocalAppData%\Skyrim\plugins.txt file.
Note that contrary to popular belief, the loadorder.txt that can be found in this folder isn't an "official" file, but is generated/used by the loadorder utility LOOT (and its predecessor BOSS). This file is commonly used by popular modding utilities such as TES5Edit as reference for the load order if present, in place of the actual plugins.txt.

The position of Skyrim.esm in the load order is hard-coded, its index will always be [00]. Other plugins, including official patches and DLCs are loaded in the order they appear in plugins.txt, and their index attributed accordingly : starting from [01] for the first plugin to the list, and up to [FE] if there is 255 plugins listed.

Since I just introduced the concept of "position in the load order" and named it "index", let's define this right away, in order to keep track of what we're talking about :

Load Order Index : The position of a plugin in the load order. Not to be confused with the formID index definition from the previous chapter. A load order index will usually be written between square brackets like this : [01]

When booting the game, the engine will interpret the formID's index we've been talking about in the previous chapter, in order to attribute them to their respective plugin : New records will have their FormID's index set to their plugin's index in the load order. Overrides and injected records will have their FormID's index set to the corresponding master's index.

Atop of that, if one plugin's master is missing, the game will crash to desktop after the official Bethesda logo. We now know enough to understand that this is because it is trying to find a master in order to give some formID's a proper index, but cannot. To prevent any critical bug, the game shut itself down.

We also know enough to understand that this would be possible for a plugin to be loaded before its master (overrides would just have a higher index than new records in the same plugin). While the game is actually able to load, and some overrides can work properly, several other might not behave as expected, and scenarios of multiple overrides will lead to random results.
Atop of that, such disordered load order is extremely difficult for a human or a basic algorithm to be read, thus it is highly recommended that the load order is... in order, i.e. that any plugin having masters should be loaded after all of them. Utilities such as LOOT and NMM will automatically modify the load order in order to have this behavior. Others such as MO or WB will warn about miss-ordered load order. TES5Edit will just refuse to launch with an error message explaining that the load order is unexpected.

More about masters

There is several reasons why a plugin have a masters, and thus various type of "dependency".

Explicit dependency : This plugin need its master records in order to function properly. The master is in the masters' list of the plugin. This is either because it is overriding some of its master's records in order to make its modifications, or because one or more of its record are referencing at least one of its master's record.

Implicit dependency : An implicit dependency is when a plugin doesn't reference a master, but is designed expecting this master to be present. Typical examples includes :

A mod needing an other mod's ressources, but none of its records.
A mod expecting bugfixes to be in place to function properly.

Forced dependency : This is an implicit dependency, but the expected master have been forced as explicit master (added to the masters' list of this plugin). This is more intended as an user warning, as almost any modding utilities available will warn the user about the missing master if this isn't present. However the game won't have to attribute any specific formID's index linked to this master and thus will proceed normally.

Obsolete dependency : Same as a forced dependency, except that the mod doesn't necessarily expect the other mod to be installed at all, not even implicitly. This is usually an error on the modder's end that happens when loading unnecessary masterfiles in the creation kit alongside the plugin worked-on.

A common misunderstanding in the modding community is that master files are files with a .esm extension, and .esp files are regular plugins. This is wrong in several levels. The file extension is just a convention supposed to identify them, but have no technical meaning.

A master file is a plugin which have the "ESM" flag set in its header. Whether it have a .esp or .esm extension doesn't matter, only the flag is to be taken into consideration. And this is the only actual difference between a master file and a regular plugin. (A plugin with the "ESM" flag set, but which still have the .esp extensions is usually refered to as an "esmyfied plugin").
The game engine have no consideration for this file, nor for the file's extensions. No matter if the "ESM" flag is set, as long as the plugin have either the .esp or .esm extension and is listed in Plugins.txt, it'll be loaded at its index.
.esm files and/or files with the ESM flag set doesn't necessarily need to be loaded before any regular files. This is only a convention followed by the majority of the community.
A regular plugin file (with the .esm extension, and without the "ESM" flag set) can be a master as much as any other plugins, this isn't a limitation.

The only difference between files that have the flag, and those that don't, is the way the Creation Kit is handling them, because it can only edit one plugin at a time. If a file with a master is loaded in the Creation Kit :

If the master have the "ESM" flag : the modder will be able to edit his plugin without modifying anything in the master(s) file(s). No matter if the plugin need records from the loaded master files, all of them will be added in its masters' list (this is how obsolete dependency happens)
If not, then the CK will simply not be able which file it is supposed to edits (since none have the "ESM" flag), and thus break the dependency in order to work on them "separately". This will result in completely corrupted plugins, as every referencing records pointing to a record in the master file will now point to the void.

And this is the end of "the basics". Hopefully this was clear enough. If not, step back and try to understand everything, or the following could be pretty hard to figure out.

The Rule of One

Coming... soon ?