Guide:Merging Plugins

From Step Mods | Change The Game

Creating New Plugins that Merge Multiple existing Plugins or Provide Conflict Resolution across Multiple Mods

For TES and Fallout Bethesda games, primarily Skyrim

Introduction

Bethesda role playing games using the Gamebryo engine, including The Elder Scrolls games (TES4 and TES5) and Fallout games (Fallout 3, Fallout NV, and Fallout 4), have a limit of 255 plugins (roughly 140 for Fallout New Vegas). While this may seem like a lot, most heavily modded games need more plugins than allowed with this limit. This small guide outlines the primary methods for merging existing plugins and for creating plugins that merge conflict resolutions and content for multiple mods.

There are multiple methods that can be used to merge all or portions of esp plugin files to allow large collections of plugins to fit inside the plugin limit of Bethesda games. Some of these methods create or merge compatibility patches for multiple mods using a single esp plugin, thus reducing the need for multiple esp plugins to provide compatibility patches for various pairs of mods.

The techniques below are listed in order of increasing complexity. With the exception of the Wrye Bash methods below, all the techniques described here require a good understanding of TES5Edit (or equivalents for other games including TES3Edit for Morrowind, TES4Edit for Oblivion, FO3Edit for Fallout 3, FNVEdit for Fallout New Vegas, FO4Edit for Fallout 4), and the record structures of plugins.

Until recently the only tools that could provided automatic merging of mods for Bethesda games were TES4Gecko which automatically merges Oblivion plugins, and FNVPlugin which merges Fallout New Vegas plugins. Mators Merge Plugins provides very capable mod merging tools that can be used across all the Bethesda games that are much more capable than any older merging tools. The x in xEdit stands for the first three letters of each of the separate editors for each of the Bethesda games. Merge Plugins requires the most recent version of xEdit which is available at the Fallout 4 reference above.

This guide does not cover the use of patches created by SkyProc or other similar tools. In Skyrim, SkyProc is used by several active Skyrim mods (e.g., ASIS, Dual Sheath Redux) to create mod-specific patches.

The rest of this guide discusses several tools used for mod merging and conflict resolution.

Merging Mods Without Conflicts

Automatic Merging of Multiple plugins with Merge Plugins

Mator's Merge Plugins utility has evolved to where it provides a simple way to do even complex merges. It is now possible to merge Fallout and Oblivion plugins, a particularly useful feature since Fallout NV cannot handle nearly as many plugins as other Bethesda games. The FAQ in the Nexus description for the Merge Plugins is also quite valuable since it covers the major issues associated with merging plugins including problems caused by changing FormIDs (as discussed later in this guide). Support for this utility is available as part of Mator Utilities Support on the STEP site.

Merge Plugins

Mator's Merge Plugins utility can be used to merge theoretically any installed mod with any other installed mod following "The Rule of One", which requires that the behaviour of the assets after the merge are identical to their behaviour before. This includes mods containing Papyrus scripts (.pex files), NAVM/NAVI (navmesh) records and FormIDs (new objects created by the plugin).

A very thorough tutorial on installing, configuring and using Mator Merge Plugins Standalone by Gamer Poets can be found here Merge Plugins: Start to Finish.

For mods that don't create objects used by other mods that aren't being merged, the resulting merged mod should be free of any problems. Examples of mods like this are (note the examples are old and will be replaced shortly with some newer examples)

The current version of the Merge Plugins utility can also merge mods without changing the FormIDs, thus preserving the capability to use items in these mods without the need to revise all the mods that use these items. This also preserves items that already exist in saved games.

The Merge Plugins utility also works well with mods that provide items like armor and clothing. The only issue with these mods is that if the mods being merged change sufficiently when updated then a new merge of the underlying mods might not preserve all of the objects in a players inventory, as discussed in the Q: My hair's gone?! My armor's gone?! My weapons are gone?! My face is gone?!?!WHAAATTTT?!!!?! question in the FAQ for the older script version of the mod. This won't be an issue if FormIds were not renumbered.

Merges of mods with conflicts uses the standard Rule of One approach that is also used in the Bethesda engine itself. The latest loading plugin that affects each object will be included in the merged plugin. A check for conflicts can be done manually by looking at the records and sub-records in xEdit or by using the Conflict Filters capability in xEdit described in section 4.3 of the FNVEdit Training Manual.

If the plugins being merged are compatibility patches, particular care must be taken since these patches typically need to load later in the mod load order and because there might be conflicts across different compatibility patches that need to be resolved. If this automated technique is used, it is a good idea to create one plugin that combines the optional patches for an individual mod vs. creating one plugin across many different mods since the automated method uses the simple Rule of One to handle conflicts. If the plugins being combined are from the same mod, always start with the plugin for the main portion of the mod if it is being merged. For large mods with multiple compatibility patches it's usually best to merge only the patches. The advantage of including the main mod is that there may be intentional overwrites in the DLC or optional capability plugins, and if the main mod is included then by adding the main plugin first the combined plugin will have the correct records.

The merged mods created by the Merge Plugins mod can merge plugins with navmesh records, but if more than one of the plugins includes navmesh overides some additional steps are needed before using the newly merged plugin. These steps are described in this post and this post in the A Real Explorer's Guide to Skyrim topic. These require deleting navmesh records in the plugin and then opening the merged plugin in the Creation Kit. An excellent detailed example of how to do this is available in one of the documentation forums for the Skyrim Expanded Towns and Villages mod.

Using LOOT (Load Order Optimization) to maintain proper load order

The patches created by this method will not be in the normal "LOOT" list, so it is recommended that the load order assigned by LOOT should be checked, and if necessary the LOOT GUI can be used to cause the mod to load later. The STEP guide includes a set of LOOT rules for sorting Skyrim plugins.

Automatic Merging of Plugins that can have Conflicts

Automatic Merging by the Skyrim Executable

This STEP forum topic, which includes a pointer to this Afkmods post by Arthmoor, discusses the record types that are automatically merged by Skyrim itself. For these there is no need for any additional merging. There are not many such record types, but it is important to know which ones are included.

Wrye Bash for TES games and Wrye Flash for Fallout games

Wrye Bash can create a bashed patch containing resolutions for some of the conflicts between plugins. Key capabilities of the bashed patch are:

  • some plugins can fully merged into the bashed patch, and these plugins can then be disabled,
  • some tweaks can be added to the bashed patch, avoiding the need for a plugin to set these (e.g., crime radius and maximum number of companions in Skyrim), and
  • a few data record types, such as leveled lists, can be merged automatically in the bashed patch avoiding the need to manually create a compatibility patch for these.

Documentation on creating a bashed patch with Wrye Bash is available in the Bashed Patch section of the General Readme for Wrye Bash (use the "?" icon on the bottom of Wrye Bash to display this file) and in the Wrye Bash Pictorial Guide. There is also a video on this and additional discussion on Wrye Bash in the STEP Wrye Bash guide. The bashed patch can automatically resolve and merge several types of conflicts between mods.

The data record merging used in the bashed patch handles several types of conflicts. Like the xEdit Merge Patch discussed below, for records that are part of "leveled lists" it can properly handle situations when one mod changes a sub-record and another mod changes a different sub-record and also when when two or mods change the same sub-record. In addition to resolving conflicts in leveled lists, Wrye Bash can also automatically resolve some non-leveled list record types when two mods change the same sub-record. For example, if one mod changes an NPC skill level to 50 and another mod changes the same skill to 60, the bashed patch will choose one of these changes or use a compromise value based on both (e.g., 55). In performing this task it uses Bash tags (for more details on the tags see here) to help determine which mod's records should have precedence when there is a conflict. These tags can be assigned manually in Wrye Bash, by using the BASH tags sutodetection script for xEdit, and (when implemented) automatically assigned to a mod when LOOT is used.

The Skyrim version of Wrye Bash does not merge nearly as many record types as the versions for TES4 (Oblivion) or Fallout (Fallout 3 and Fallout New Vegas). Wrye Bash for Skyrim handles record types for leveled items, leveled NPC, Names, Item Stats for armor and weapons, and a very small set of game settings (GMST). The only game settings it handles are ones explicitly included in the bashed patch menu; it does not include any GMST records from mods. It remains, however, the best way to merge leveled lists and leveled NPCs, but with the WB limitations in Skyrim other utilities are often needed to supplement its capabilities. An experimental version of Wrye Bash for Skyrim is available that handles a much larger set of hash tag types; once this has been tested Wrye Bash will be able to handle some of the merging that is especially tedious to do manually.

Mator Smash

Mator Smash currently in development, as discussed in the Mator Smash subforum on the STEP site, is being developed to provide a much more complete capability for merging mods managed with tags.

xEdit Merge Patch

TES5Edit (and the equivalent xEdit programs for the other Bethesda games) can automatically create a Merge Patch that merges some potentially conflicting records across multiple mods. In Skyrim it can only work with a few types of records and is primarily useful as a starting point for creating manual patches. With the emergence of the Merge Plugins tool discussed previously there does not seem to be any need to use the xEdit Merge Patch.

Using this capability of TES5Edit is quite different than using the Merge Plugins mod described above since here TES5Edit is being using to create a set of merged records that only include records for which the mods being merged have conflicting entries while above it was merging entire mods when the mods had no conflicts. It can handle plugins that include scripts. This is described in section 4.8 of the FNVEdit Training Manual (the most recent available documentation for TES5Edit) and in Sharlikran's "Merge Patch" video. As the video points out, when using this TES5Edit (and the equivalent programs for the other Bethesda games) feature make sure to remove the Leveled List records and the leveled NPCs from the Merge Patch (note that unlike the edit programs for other Bethesda games the TES5Edit Merge Patch does not currently include leveled NPCs). It is recommended that Wrye Bash be used to take care of leveled list merging, as described above. Like the Wrye Bash bashed patch, the Merge Patch needs to be recreated whenever there is a plugins are added or updated that would change the patches automatically produced in the Merge Patch. The documentation on the Merge Patch suggest manual review of the patch to fix conflicts that the automated Merge Patch process was unable to fix.

In TES4 and the Fallout games the Wrye Bash bashed patch is sufficiently comprehensive , more configurable, and intelligent that the Merge Patch isn't typically needed. In Skyrim, however, the TES5Edit Merge Patch is quite useful, especially since it takes care of portions of compatibility patching that Skyrim Wrye Bash doesn't. In addition to Leveled Item, the TES5Edit Merge Patch includes the following record types: Ammunition, Armor, Container, Faction, Magic Effect, Misc. Item, Scroll, Weapon, and Non-Player Character (Actor).

Creating a Merge Patch is straightforward. Load TES5Edit (or the equivalent programs for the other Bethesda games), and select "OK" after removing the checkmark for the bashed patch if there is one. Right click in the left window and select "Other" and then "Create Merged Patch". When prompted for a new module name enter a new name for the merged patch. After the patch is created disable and/or delete any previous Merged Patches. In addition, the Leveled Item object class should be removed and the patches reviewed to make sure they are reasonable changes; most of the time they are.

Manual Merging of Mods with Conflicts

Manually merging plugins with TES5Edit

TES5Edit can be used to manually create a single compatibility patch merge plugin that provides compatibility fixes and parameter changes across a large set of mods, that merge complete mods, or combinations of these. Manual editing is the most complicated and flexible of the methods being discussed here. It is briefly described in section 4.7 of the FNVEdit Training Manual. Since this is a manual process it can be used with plugins that have conflicts and those that have scripts; script records can be copied to the patch merge plugin like any other record. Manual Editing is easiest with plugins that do not have new FormIDs; if there are FormIDs they need to be renumbered. Manual editing is especially simple with small plugins that change a few parameters originally set by the main game esm file; of course, these are also the plugins that are easiest to combine with the automated methods discussed in the previous subsection. For example, in Skyrim the 26 Moon Size Tweak mod has two plugins each containing a single record. Merging these two plugins into a single plugin or even into an overall plugin containing Patches is particularly easy. Of course, the Merge Plugins TES5Edit Script mod described above can be used to further simplify this merge process as long as there are no conflicts. Generally speaking, actual manual editing (vs. using tools like the Merge Plugins TES5Edit Script) is needed when there are conflicts that can't be resolved automatically. This occurs frequently, but not always, when merging compatibility patches, and when creating compatibility records for mods that have compatibility problems but don't have existing compatibility patches.

In some ways the most difficult problem with manual merging is to decide which plugins to merge; it can be difficult to correctly merge complex plugins when there are conflicts on multiple sub-records. There is limited documentation on the types of errors that are typically encountered when doing this. In general the easiest mods to manually merge are ones that are small or don't have complex record structures.

Manual merging involves four types of actions:

  • "Copy as Override", as described on page 33-34 in the FNVEdit Training Manual, is used to copy a single record from a mod into a patch plugin. When there are conflicts this is used to get all the sub-records from a mod into a patch plugin, typically followed by manual editing of selected individual sub-records in the patch as described below. The FNVEdit Training Manual suggests that this is done after using a conflict filter; this is useful when a lot of mods are being edited at the same time but isn't needed when a patch is being created or edited for a few specific mods.
  • "Deep Copy as Override" is used to simultaneously copy multiple sub-records under one or more parent records of a mod into a patch plugin. When there are conflicts this is used to get all the sub-records from multiple records of a mod into a patch plugin, typically followed by manual editing of selected individual sub-records in the patch as described below.
  • "Edit" an individual sub-record, as described briefly on the bottom of page 36 in the FNVEdit Training Manual, is used to change the value, text, or reference in a single sub-record in a plugin.
  • "Drag and Drop" editing of individual sub-records, as described on page 35-36 in the FNVEdit Training Manual, is used to copy the contents of a single sub-record from a mod into a patch plugin. When creating a patch this is often the most commonly used action type.

When manually merging care must be taken with GMST (game setting) records as discussed in this post by EssArrBee. The procedure for merging these records is:

  • Right click Game Settings->Add->GMST - Game Setting.
  • Put in any FormID that start with the load order number (e.g., if the mod has 4D in the load order add 4DXXXXXX).
  • Go to the mod whose Game Setting is being merged, get the Editor ID (a text name) of the Game Setting, then add it to the Editor ID line for the new FormID just added to the merged plugin.
  • This should make the other records with that game setting show up in columns in TES5Edit, and the setting values can then be dragged and dropped from the mod with the desired Game Setting value.

An Example of Using Copy as Override and Drag and Drop to Change a Subrecord for a Patch Plugin

Drag and Drop editing is commonly used in creating patches. An example for using this in creating a patch is a patch for to resolve a conflict between Acquisitive Soul Gems (ASG) and Animated Weapon Enchants (AWE). AWE changes the shader sub-record for the Soul Trap spell, while ASG changes many other sub-records. The steps used to create the patch are described below.

  • Open TES5Edit
  • Select "Acquisitive Soul Gems.esp" and Animated Weapon Enhants.esp then click [OK].
  • Expand "Magic Effect" under Acquisitive Soul Gems.esp and select the "0005B452" node.
  • Right-click on Acquisitive Soul Gems.esp column and select "Copy as Override into...".
  • Select "<new file>" and click [OK]. In the resulting text box type "ASG-AWE Patch".

When you do this a new column appears on the right titled "ASG-AWE Patch". Scroll way down until you see the row heading "Enchant Shader" under "Magic Effect Data". If you look on that line you will see that three of the columns have "EnchPurpleFXShader" entries and one (AWE) has "wpnSoultrapFXS". You want to drag "wpnSoultrapFXS" entry from the AWE column to the entry on the same line in the column to the right ("ASG-AWE Patch"). When you do this it will replace the "EnchPurpleFXShader" entry with "wpnSoultrapFXS".

Guide to make a patch for solving conflicts

A small guide is available with screenshots and recommendations on patch creation. It has not been reviewed in detail yet, but it does ave some useful suggestions.

Using ModGroups to Simplify Manual Conflict Resolution

See this thread in the STEP forums and this post in the AFKmods xEdit thread. This capability is available in xEdit 3.1.0. More examples need to be added to this section about using ModGroups.

Patch Load Order

While potentially the patches created by these four methods could be merged, it is suggested that the plugins be kept separate to simplify maintenance (there will always be need to add new records and edit/remove old ones). Any merged plugins created should be loaded near where the original mod was loaded, as discussed in 1.3. The Merge Patch from 1.2 should be loaded prior to the bashed patch. Typically any manually created Patch Merge plugins from 1.4 are loaded either after the mods they reference or after the Merge Patch and before the bashed patch.

Issues in Changing FormIDs

Every different type of object in the game has an 8 digit hexadecimal number called a FormID, a unique identifier for that particular object type. As mentioned above, when merging some plugins the FormIDs for objects created by a mod (vs. the vanilla objects in the Skyrim.esm or Update.esm plugin or one of the DLC esm plugins) need to be changed. If the resulting merged mod includes all uses of these FormIDs, then there aren't any problems since the merging process changes all references to these FormIDs to use the new values. However, if there are other mods that reference one of these FormIDs that are not included in the merged plugin, there is a problem since those mods reference the old FormID value which is no longer exists. This is why the suggestion was made above to merge plugins from the same mod; this usually insures that all references to FormIDs created by the mod are included in the merged plugin. A merged plugin does not actually need to contain only plugins from the same nod, but for all the mods included in the new merged plugin all uses of any FormIDs created by these mods must be included in the merged plugin.

As mentioned above, this issue is also discussed in the "Q: My hair's gone?! My armor's gone?! My weapons are gone?! My face is gone?!?!WHAAATTTT?!!!?!" question in the FAQ for the Merge Plugins TES5Edit Script mod .

Many mods do not introduce new FormIDs in their plugins, and as long as there are no conflicts among these mods merging their plugins can be fairly straightforward (unless there are issues with merging scripts; this does not typically happen with mods that use small scripts).

Retrieved from "https://stepmodifications.org/wiki/index.php?title=Guide:Merging_Plugins&oldid=172216"