Current Skyrim Scripting Practices and Merging

[keithinhanoi]

What method do you suggest?

 

It's an interesting request, trying to kill two birds with one stone here. However, most authors who have been using Game.GetFormFromFile or Game.GetModByName to recognize and act on the presence of other mods did not expect it to become a problem. It became common practice before the merge plugin script started to exist. So it does not seem fair at all to even imply that mod authors have been doing the wrong thing by using those function calls.

 

Nevertheless, those calls do depend on other mod authors not changing the names of their mods' plugins, but we've all seen instances of this happening over and over again. So - how to deal with that, yes?

 

It has to be some kind of call that can identify a uniquely identified element in another plugin without referring to the plugin's name. I'm a Papyrus noob, but I see things like Quest.GetQuest() might work, if you don't mind making SKSE as prerequisite. I know USKP adds some new uniquely named quests, so maybe check into that?

 

Another crazy idea: Keep Game.GetFormFromFile or Game.GetModByName (whichever you're using), but have the script make use of an external text file which lists the names of other mod plugins that are checked for. Bring those names in as variable strings and then use them for the function calls. The mod user could then edit the text file's line with USLEEP.esp to change it to Unofficial Skyrim Patch.esp -- of course this would depend on the ability of scripts to read from external text files, and I don't know if that's possible.

 

Last thought on this is to just add USLEEP.esp to the script in addition to checking for Unofficial Skyrim Patch.esp so both cases are covered. This solution, of course, won't help those who are trying to use the merge plugins script.

 

If you're savvy enough to merge plugins, it should be no sweat for you to edit the scripts, replace all instances of the old esp name(s) with the new one, and recompile.

 

 

I consider myself fully proficient in TES5Edit, have done beta testing of matortheeternal's merge plugin script, and have made such changes and other changes to specific papyrus scripts.

 

Based on this experience, I would say that changes the name of a referenced plugin in a papyrus script is no trivial matter. Many mods refer to other plugins in more than one script, and you need to be aware of the dependencies of the scripts you want to recompile in order to do it correctly. It's pretty easy to screw things up in a way that won't be noticed until issues have been baked into your save file, especially if you're coming in without any background knowledge of papyrus whatsoever.

 

 

 

These 18 or so posts originally came from this topic.

The posts were move due to being off topic, sorry everyone.

-hish

[hishutup]

Its funny because Kesta, Mator and I were having a discussion on Keep Game.GetFormFromFile or Game.GetModByName in scripts.

 

Its possible to redirect the forms and name to the merged plugin and will be a feature down the line :D

He is currently working on Script Fragments and is nearly done.

 

Keith, check out mators asana, there is a bunch of newish info.

[Kesta]

Its funny because Kesta, Mator and I were having a discussion on Keep Game.GetFormFromFile or Game.GetModByName in scripts.

 

Its possible to redirect the forms and name to the merged plugin and will be a feature down the line :D

He is currently working on Script Fragments and is nearly done.

 

Keith, check out mators asana, there is a bunch of newish info.

Actually I saw this post and this is one of the reason why I put this on the table yesterday ^^

 

 

As for the debate, thought I don't like to overload mator with work, my personnal opinion is that mod author should make mods, and user have to deal with them. That's enough bother to create something that work properly and have a background compatibility integrated, users/mergers are the ones who should deal with this problem, not modders.

 

 

You can already merge "safely" dozen of mods (weather, lighting, characters parts, tweaks, items which aren't checked by CCOR, house, ...), I'm not even sure this is a major issue. AFAIK, the merging script isn't intended to collapase your 250 mod plugins into a single one.

Edited July 25, 2015 by Kesta

[kryptopyr]

Last thought on this is to just add USLEEP.esp to the script in addition to checking for Unofficial Skyrim Patch.esp so both cases are covered. This solution, of course, won't help those who are trying to use the merge plugins script.

This is my intended solution, at least where I already have USKP support in my mods.  Since the FormIDs will remain the same, it should be fairly simple to copy the existing script and change the name it's looking for (then add and adjust for any forms from the DLC patches).

 

From what I just read, I think GetQuest() would work in place of Game.GetFormFromFile or Game.GetModByName, so that's certainly something worth keeping in mind.  Thank you for making me aware of it.  Of course not all mods include new quests, but many do, even if it's just to add an MCM.

 

It's important for users to know the mods they're using, which can be rather difficult since scripts aren't the easiest thing to check.  There are a lot of mods using Game.GetFormFromFile or Game.GetModByName to identify other mods and add compatibility features.  This is why if using Frostfall or Wet & Cold, you never wanted to merge Cloaks and Winter is Coming.  Sure, you'd save yourself 2 plugin slots, but at the same time it broke compatibility with two other major mods.

 

Generally, any scripted mod that I download, I usually look for scripts named something like "maintenance," "player" or "playeralias," or "compatibility" and specifically look for mod detection/interaction within those scripts.  But there is certainly no guarantee that the mod author will follow this sort of naming convention or that they won't be checking for external FormIds elsewhere in their mod.

 

I think, even with Mator's amazing script, that users should not be attempting to merge every mod in their load order.  Mator's script may well be a game-changer, but merging large complex mods, particularly those that are still in active development, just doesn't sound like a wise idea to me.  It will be wonderful if the Merge Plugins script can merge the mods' scripts as necessary to allow this sort of merge, but I still think I'd reserve that sort of merge to specific situations and not try to simply blanket merge all scripted mods together.

[hishutup]

 

I think, even with Mator's amazing script, that users should not be attempting to merge every mod in their load order.  Mator's script may well be a game-changer, but merging large complex mods, particularly those that are still in active development, just doesn't sound like a wise idea to me.  It will be wonderful if the Merge Plugins script can merge the mods' scripts as necessary to allow this sort of merge, but I still think I'd reserve that sort of merge to specific situations and not try to simply blanket merge all scripted mods together.

If people are trying to merge every mod they can then thats their fault, Im doing it because Im testing the script.

When finding plugins to merge, you never touch mods like CCOR and ETaC Complete.esp/.esm because there are so many issues with doing so. One of the issues being that it is a master, that means that if you want to merge it, then you must use any mod that is dependent on it. Even Merge ETaC patches alone is plenty, why would one need to include the main esp.

 

The only mod that I found to be unmergable is ETaC Dragon Bridge South because it uses a ton of injected records and everything just kinda breaks.

SKSE plugins that contain a DLL are 99% unmergable because they break.

There is a known issue with noncontiguous plugins but that may be fixable at a later date.

 

Hmm, I was a little ranty and off topic but still, its the users fault, let them screw it up.

I guess authors can submit reports saying "DO NOT MERGE" but that should be reserved to plugins that are known to be unmergable because it will break everything, including other plugins. 

Now that I said that Im sure a boat load of authors are going to say that, even on merges that are perfectly fine.

[Mator]

I think this is a nice development.  I'm not sure how well I like the acronym (do I really sleep?  I'm not sure).  I hope that it will provide a degree of simplification for mod users and authors alike.

 

 

In regards to renaming the file-
I think it makes sense to rename the file because it is a different file and using the same name would lead to massive amounts of confusion regardless of how many notifications were created.  Heck, even I would be confused in certain cases.  A new, unique filename is the most natural and most logical choice when you're making a new file.

Compatibility issues are fixable.  It is also not really the STEP team's responsibility to work with those.  Here, a rant for our troubles about dependencies-

 

 

Master dependency:
The only reason you should ever have a master dependency on a mod is if you specifically require the content of that mod or specifically build off of it.  If a user were to do that with the Unofficial patches, it only makes sense that they make an updated version of their work for USLEEP, because it is in many ways a different beast from the separated Unofficial Patches.  The value a single plugin offers is great enough to make this worthwhile.  Also, how many mods out there actually require the USKP as a master?  It really shouldn't be all that many.

 

 

Script (implicit) dependencies:
The issue with this is it is a bad design pattern.  I work in Software Engineering, and the entire nature of how these dependencies work brings a bad taste to my mouth.  The number of assumptions made and the means by which these files are interfacing with each other is extremely unpleasant to me.  This is largely based on the idea of a good application programming interface (API).  I'll elaborate.

When a modder uses a Game.GetFormFromFile call, they are attempting to bring compatibility between their unit and another unit which makes similar or conflicting changes without requiring a compatibility patch.  The reasoning is that compatibility patches create load order bloat and can be difficult to maintain (which is a whole other discussion which I'll get to later).  Anywho, the compatibility that the author is trying to create is directed not at the filename, but at the changes a particular file with that name makes.  Thus, this is based on the assumption that a file with SomeModFileName.esp makes changes XYZ.  This might not be the case.  That mod could be updated, or a new mod could come around with the same filename (unlikely but possible), or mods could be merged, or... the list goes on.  The point I'm making here is there is modders are trying to interface with other mods when no interface exists, and are thus relying on completely unreliable methods to "hack" an interface together.  It is extremely delicate because it relies not just on file names, but on FormIDs and an unspoken, unagreed contract between mods that could be broken at any time.

A good interface provides an explicit contract.  The nature of what is provided and how it is provided is made clear from the get go.  Sometimes the promise of what is provided isn't fulfilled, but in that case it is extremely clear whose responsibility it is to resolve.  Either the creator of the interface changes their interface to fulfill their promises, or they change their promises to better describe what the interface provides.

Now, I don't have a perfect solution for setting up such an explicit interface.  I also don't think it is possible to make a perfect solution due to how mods function.  However, I can tell you more or less how such an interface could work.

Mod Alpha makes some changes to the game.  It wants to provide other mod authors with a means to interface with the changes it makes.  The first step of this is to allow mods to cleanly detect its presence in a way that is explicit, rather than implicit.  I look towards the work around MCMs for inspiration on this.  MCMs are implemented as a script linked to a Quest.  SKSE offers a Quest.GetQuest() method.  If Mod Alpha sets up a quest with  an EditorID that is unique and fixed, any other mod can get that quest using Quest.GetQuest().  So Mod Alpha would make a quest with the Editor ID "ModAlpha-Interface".  Then, using aliases and stages, Mod Alpha can make an interface which has various important information available.  E.g. Alias CurrentVersion = "1.0.0", Alias bSomeBoolean = "true", or whatever.  Mod Beta then wants to make some compatibility features if Mod Alpha is in the user's load order.  It uses Quest.GetQuest("ModAlpha-Interface"), reads the aliases to make sure things are as expected (raising warnings if things aren't as expected), and then proceeds or terminates the activation of compatibility features.  This does mean that in order to interface with mods, the author of that mod will have to make such an interface quest, but that's how it should be.

 

Compatibility:

Ultimately, the underlying issue here is just that multiple mods don't necessarily work together.  This is because of the nature in which changes are implemented ingame.  Most of the time, the intention of most changes isn't how they are actually implemented.  E.g. In the case of an overhaul that changes the value and weight of every vanilla armor, the goal of the overhaul isn't just to change vanilla armors, it is to change all armors to reflect a new standard or base idea.  The issue is the implementation (as an ESP that makes changes to specific armors) is unchanging.  It doesn't actually LOOK at the armors that are present in the user's game, it just hardcodes its intentions on top of the armors that already exist.  Herein lies the problem.  The majority of mods execute their intentions in a non-extendable fashion, thus requiring the creation of compatibility patches, or papyrus links, or what have you.

The solution?  Well, something automated that iterates through items is what we need.  That could be a papyrus script (which is pretty inefficient usually), or it could be an external "patching" script (e.g. xEdit script).  The issue is xEdit scripting is still greek to most people and there aren't quite enough resources yet to make it accessible to everyone (as evidenced by people not using xEdit scripting more).  I'm hoping to change this with some of projects, but I'm only one guy.  There's only so much I can do with my free time.
 

But ignoring the fact that mods have a severe disjoint between their intentions and their implementation, there's still the fact that, without automated compatibility patch creation, the standard in the community has become an unmaintainable cycle of "make new stuff", "make compatibility plugins/features so stuff works together", "make new stuff", etc. ad nauseum.  So a whole bunch of the community's time is being wasted just trying to keep things up and running.

 

 

Anywho, that's about all I have to say.  Carry on.

-Mator

Edited July 25, 2015 by Mator

[kryptopyr]

 

The only mod that I found to be unmergable is ETaC Dragon Bridge South because it uses a ton of injected records and everything just kinda breaks.

Good to know.  I haven't looked at this mod, but I like to keep track of mods that use injected records.  Do you know if these are records are all intentional?

 

 

@Mator

I don't really disagree with anything you said, and I like some of the ideas you propose.  However, I think that an ideal framework for modding is not going to be achievable.  It'd be great if the entire community could be convinced to follow a specific format and use your idea of Interface quests or something similar, but I can't see it ever happening unless Bethesda actually builds in some system for inter-mod communication so that there is a concrete, indisputable standard.  Anything else is always going to rely on the same sort of "unspoken, unagreed contract between mods that could be broken at any time" that currently exists.

 

Using Quest.GetQuest() as things currently stand would involve just as many assumptions as Game.GetFormFromFile.  And in fact most modding practices are based on similar assumed standards, not just scripted compatibility.  Equipment slots are one example.  There's simply an expectation that mod authors will follow the community standard when assigning equipment slots.  Likewise, it's standard for mods to use aliases when possible instead of directly editing the player actor, change vendor leveled lists instead of directly adding items to a merchant container, include any relevant bash tags, etc.  I've designed a communication framework based on injected records, which when used correctly can be a wonderful way for mods to communicate.  But it has it's downsides as well, and also relies on a vague agreement between mod authors to use the records as intended and for other mod authors to avoid using identical FormIDs.  All of these modding practices are nothing more than unofficial standards used by mod authors.  

 

And sometimes what seem to be fairly concrete conventions end up getting broken or overturned.  Before the UP team chose to defy it, there was an assumed standard that all mods should be loaded after the official DLCs, and mod managers were designed based on that standard.  And before MO came along, the assumption was that loose files would always take priority over those packaged in a bsa.  At one time, many modders would have said those were fairly explicit practices, but they turned out to be no more than assumptions based around old standards.

 

Plugin patches that directly edit mod records are just as easy to 'break' as scripted compatibility (the breaks are just easier to identify and correct in TES5Edit).  Automated scripts aren't going to solve compatibility because they're still based off of assumptions.  True, they allow modders to extend their intentions to other mods, which can be an awesome tool, but the script still has to make assumptions on the way the various mod authors have set up their mods.  If a mod doesn't use vanilla keywords in the standard fashion or replaces certain vanilla perks with new ones, then somehow the script needs to recognize that mod as an exception and adjust accordingly.  And as soon as a user has to start making exceptions for specific mods, then your compatibility process isn't nearly as automated as you'd like it to be.

 

Many mod authors do follow the unspoken contracts as they currently exist, because it does help the community as a whole deal with compatibility issues, but there are plenty of mods that break with these conventions either because their authors weren't aware of the standard, because they didn't care, or because they thought that their way was better.  There is very little in modding that is truly concrete or explicit, other than what is made necessary by the structure of the official files.  The modding community being what it is, I just don't see any way to realistically strengthen this system without some officially-defined structure for mod communication and compatibility.

[kryptopyr]

In regards to renaming the file-

I think it makes sense to rename the file because it is a different file and using the same name would lead to massive amounts of confusion regardless of how many notifications were created.  Heck, even I would be confused in certain cases.  A new, unique filename is the most natural and most logical choice when you're making a new file.

One could argue that any updated file is a new file.  I see this as more of an update to the USKP to include support for the DLCs than as a completely new file.  If the edits themselves or the FormIDs were changing, then I definitely agree and would feel that it should be treated as a new file with a new name.  But given that the majority of the thousands of edits and non-DLC records shared between USLEEP and USKP will be identical, I think it should be treated as an update rather than a brand new file.

 

I'm just don't see where the confusion would come from.  A quick glance at the metadata or the file's masters would determine which version you were dealing with.  Almost every mod that I know of that has started out as individual files then gone on to release legendary version, has kept its original name.  The ones that don't are generally more of a pain to deal with because anyone who is creating compatibility patches has to decides whether to update and/or support two different versions.  In many cases, existing compatibility patches remain valid and functional if the mod doesn't change it's name.  Though, again, if there is a huge restructuring or change to the record edits or FormIDs within the file, then it should probably change names since that restructuring will likely invalidate any patches regardless.

 

Master dependency:

The only reason you should ever have a master dependency on a mod is if you specifically require the content of that mod or specifically build off of it.  If a user were to do that with the Unofficial patches, it only makes sense that they make an updated version of their work for USLEEP, because it is in many ways a different beast from the separated Unofficial Patches.  The value a single plugin offers is great enough to make this worthwhile.  Also, how many mods out there actually require the USKP as a master?  It really shouldn't be all that many.

But it isn't necessarily a different beast with regards to a specific mod.  Take a house mod that alters Proudspire and has been forced to use USKP as a master only because of the single cookpot record that it adds.  Chances are that one record will remain identical between USLEEP and USKP.  If the file name wasn't changed, then that house mod would still function perfectly regardless of whether a player is using the older USKP or the newer legendary version.  However, by changing the name, the author of that house mod now has the following options:

1) do nothing and allow their mod to become outdated and unusable for the growing majority of players

2) change the mod to support the new file name and drop support for the non-legendary version

3) update the mod for the legendary version and then continue supporting two versions of their file instead of one

 

Obviously if that mod author is no longer around, then #1 is going to happen by default.  Some mod authors will choose #2 for simplicity and to ease their own work load.  Others will try to support both versions because there are still people who don't have all DLCs and, quite frankly, because there is nothing about their house mod that requires the DLCs other than the Unofficial patch.

 

I think that forcing mod authors to update their mods and to indirectly make their mods dependent on all three DLC, when the mod wouldn't otherwise need to be dependent on any of the DLCs is simply the wrong choice.  Referring back to the comments on community standards and unspoken contracts, I feel like this decision could be seen as a breach of that assumed contract. 

 

-----------------------

 

Also, just to be clear, I don't think for a second that this is going to be a devastating compatibility disaster for the community.  As a mod author, I see it as a minor annoyance more than anything. There will be a handful of mods that this have some issues due to this since their authors aren't around anymore, and a probably several dozen others that will be affected and will simply adapt to the new file.  The convenience of having a unified unofficial patch more than makes up for any small headaches created by it's implementation.  If I seem to be arguing the point more strongly than seems warranted, it's mainly because I'm finding this discussion of compatibility and community standards to be rather interesting and enjoyable. 

[DoubleYou]

From a maintenance perspective, would you rather work with one file or four files?

Oh, I do understand and support the merge of the unofficial patches. I just can't understand merging, say, Unofficial Skyrim Patch, Book Covers Skyrim, and Wet and Cold together.

[TechAngel85]

Oh, I do understand and support the merge of the unofficial patches. I just can't understand merging, say, Unofficial Skyrim Patch, Book Covers Skyrim, and Wet and Cold together.

Oh. Yeah, I don't see the point either unless users are reaching the plugin limit.

[EssArrBee]

Skyrim's limit is 255 so it isn't that big a deal. It's nice to keep that number down, but once your reaching that mark stability can be a nightmare. Fallout on the other hand is limited to about 135. The merge script and bashed patching is the only thing that saved us from having guides half their current size.

[Octopuss]

I don't even understand how can people have so many mods they are hitting the limit. I'd say my game is quite modded, and I still am at 150-160 plugins.

I bet 90% of people who are in the 250ish range don't even know what mods they have installed, and simply downloaded every single piece of junk they could find.

[TirigonX]

I don't even understand how can people have so many mods they are hitting the limit. I'd say my game is quite modded, and I still am at 150-160 plugins.

I bet 90% of people who are in the 250ish range don't even know what mods they have installed, and simply downloaded every single piece of junk they could find.

Well, the more mods you use the more compatibility patches you'll most likely need. And that's something that can push you towards the limit very quickly. I have dozens of little patches, which I merge into themed patches. E.g. one file for all ELFX patches, one file for all AOS patches and so on. Apart from that I also merge some small mods that contain only a few records. 

 

I have about 200 esp files in my load order, without merging I'd definitely have more than 255. And yes, I know all of them ;)

I never merge any complex mods though. All the mods that I merge are only a few kilobytes big.

[Mator]

Preface:
This post may seem somewhat defensive. I fully respect krypt and I am in fact currently working on updating some scripts for her. However, I feel very strongly about my position in these matters. All of my energy and words below are directed not at krypt as a person, but at the subject at hand.

 

Good to know. I haven't looked at this mod, but I like to keep track of mods that use injected records. Do you know if these are records are all intentional?

I haven't looked at it either, I've only heard about it second hand.

 

@Mator
I don't really disagree with anything you said, and I like some of the ideas you propose. However, I think that an ideal framework for modding is not going to be achievable. It'd be great if the entire community could be convinced to follow a specific format and use your idea of Interface quests or something similar, but I can't see it ever happening unless Bethesda actually builds in some system for inter-mod communication so that there is a concrete, indisputable standard. Anything else is always going to rely on the same sort of "unspoken, unagreed contract between mods that could be broken at any time" that currently exists.

Oh, I know that an "ideal" framework is not achievable. I said as much in my OP. It would be great if the community could follow a specific format, and I honestly don't think any of this discussion is worth discussing if we don't consider the community capable of following a standard. Examples of standards the community follows:

- Mod Configuration Menus
- Separation of mods that do different things (not one mod that does a bunch of random things)
- Numerous implicit standards from the Bethesda files
- others..? I'm not super well-versed in this area, to be honest

These examples (most notably the MCM example) show exactly how capable our community is at establishing and following standards. I think it is overly pessimistic to assume we wouldn't be capable of defining a new standard and documenting it sufficiently to get mod authors to buy into it. Heck, just getting a few top mods/modders to follow it would set enough of an example for the entire community to follow suit.

There are certainly people who don't follow standards, but there always will be and that's not our problem.

 

Using Quest.GetQuest() as things currently stand would involve just as many assumptions as Game.GetFormFromFile. And in fact most modding practices are based on similar assumed standards, not just scripted compatibility. Equipment slots are one example. There's simply an expectation that mod authors will follow the community standard when assigning equipment slots. Likewise, it's standard for mods to use aliases when possible instead of directly editing the player actor, change vendor leveled lists instead of directly adding items to a merchant container, include any relevant bash tags, etc. I've designed a communication framework based on injected records, which when used correctly can be a wonderful way for mods to communicate. But it has it's downsides as well, and also relies on a vague agreement between mod authors to use the records as intended and for other mod authors to avoid using identical FormIDs. All of these modding practices are nothing more than unofficial standards used by mod authors.

There is a HUGE difference between using Quest.GetQuest with a quest that exists for the intention of serving as an interface for other mods to tie into and using Game.GetFormFromFile or Game.GetFileByName to grab arbitrary information from arbitrary mods. In one hand the mod you're interfacing with with a component made for you to tie into, while in the other you're relying on arbitrary "random" things like filenames and FormIDs. Yes, an interface quest could be named anything (like a file), but an interface quest has multiple advantages:

- You can store any number of arbitrary variables (aliases) in it which can be accessed by other modders to verify the quest is what you want, and not something else
- It won't be broken by a formID renumbering, which is how things should be because FormIDs are arbitrary
- It won't be broken by merging or files being renamed, which is how things should be because filenames are arbitrary and everyone merges files these days

- It is extremely easy to re-implement/carry over between files (unlike filenames and FormIDs)

Overall, the interface quest design pattern creates a massive increase in reliability for cross-mod compatibility when compared to the current method of using Game.GetFormFromFile or Get.GetFileByName.

Agreements can always be broken. The question is how robust/reliable the agreement is. How easy is it for someone to break it, and how often is it actually broken? The current agreement (using Game.GetFormFromFile) is broken all the time. It is ridiculously delicate and anyone who has to deal with authors who use it can't help but scream (internally) "WHY DID THEY DO THIS?!". I know this because I've had people tell me how frustrating it is (not to mention the pressure that is puts on me to resolve it in Merge Plugins, which requires an insanely more complex solution than people just using a better design pattern). I feel I have the right, as a person who is burdened by this issue, to say that it really needs to change.

 

And sometimes what seem to be fairly concrete conventions end up getting broken or overturned. Before the UP team chose to defy it, there was an assumed standard that all mods should be loaded after the official DLCs, and mod managers were designed based on that standard. And before MO came along, the assumption was that loose files would always take priority over those packaged in a bsa. At one time, many modders would have said those were fairly explicit practices, but they turned out to be no more than assumptions based around old standards.

Assumptions are the problem. With Quest.GetQuest used with an interface quest there is no assumption because all parties involved have taken actions to achieve the same goal. There is no room for misunderstanding, because everyone has taken steps with clear intention to achieve the same goal. This is not the case with Game.GetFormFromFile.

In addition, Quest.GetQuest is extremely scalable and easy to change. Currently, if you want to access a record in a papyrus script from another file without requiring that file as a master (a silent dependency) you access that record by using Game.GetFormFromFile. Alternative to this, you could use Quest.GetQuest to get a quest which can then have an alias to the FormID you want to access. This way when the source mod gets updated your code doesn't have to change, because the alias on the quest gets updated. What we're looking for is to create a system where the least amount of work is necessary to make changes. I should be able to go into an ESP file and make some changes without creating hours upon hours of work for other mod authors who rely on that ESP file through some arbitrary inflexible system like Game.GetFormFromFile.

 

Plugin patches that directly edit mod records are just as easy to 'break' as scripted compatibility (the breaks are just easier to identify and correct in TES5Edit). Automated scripts aren't going to solve compatibility because they're still based off of assumptions. True, they allow modders to extend their intentions to other mods, which can be an awesome tool, but the script still has to make assumptions on the way the various mod authors have set up their mods. If a mod doesn't use vanilla keywords in the standard fashion or replaces certain vanilla perks with new ones, then somehow the script needs to recognize that mod as an exception and adjust accordingly. And as soon as a user has to start making exceptions for specific mods, then your compatibility process isn't nearly as automated as you'd like it to be.

No, script "breaks" are FAR easier to fix. You change a few lines of code and everyone re-runs the script, and everything is fixed for everyone. If we're talking about patching hundreds of files, which is easier:
- Editing hundreds of ESP files
- Editing a few lines of code in a script
?

You tell me!
As I said before, everything relies on assumptions, the question is how robust and reliable the assumptions are. Really good scripts, however, don't have to make many assumptions at all, and the ones they do make are extremely reliable. Sometimes it is not the script's job to handle more variations of input, but mod author's jobs to make mods that fit with established standards.

You can automate exceptions. You can make automated solutions that are extremely robust. It certainly takes some time, but it takes FAR less time than making ESPs by hand. How many mods does FMO work with? Every one which follows some basic assumptions about armor/weapon filenames. Here, an example:

How many ESPs patches worth of edits can FMO produce today?
Thousands.

How long would it take me to produce all of those ESPs by hand?
Thousands of hours.

How much time have I spent on FMO?
A hundred hours

Is this worth it?
- 10-factor increase in efficiency, at least
- Don't have to worry about changes/new mods, they work instantly
- Don't have to have users manage hundreds of patches
- Don't have to bloat load orders

So, yeah, automation is superior, and always will be. I don't know what kind of point you were trying to make here. The present (and future) of the world economy (let alone Skyrim modding) is centered around automation. Non-automated methods cannot compete with automated ones.

 

Many mod authors do follow the unspoken contracts as they currently exist, because it does help the community as a whole deal with compatibility issues, but there are plenty of mods that break with these conventions either because their authors weren't aware of the standard, because they didn't care, or because they thought that their way was better. There is very little in modding that is truly concrete or explicit, other than what is made necessary by the structure of the official files. The modding community being what it is, I just don't see any way to realistically strengthen this system without some officially-defined structure for mod communication and compatibility.

I addressed this earlier. This is extremely pessimistic. I strongly disagree with this paragraph and find it insulting. I have defined the standard of merging plugins. I have helped define the standard of xEdit scripts. I have built dozens of automated solutions that operate on standards. YES, standards are not "immutable", but that doesn't mean we can't operate on them. A world without standards is chaos.

 

One could argue that any updated file is a new file. I see this as more of an update to the USKP to include support for the DLCs than as a completely new file. If the edits themselves or the FormIDs were changing, then I definitely agree and would feel that it should be treated as a new file with a new name. But given that the majority of the thousands of edits and non-DLC records shared between USLEEP and USKP will be identical, I think it should be treated as an update rather than a brand new file.

"Including support for the DLCs" is far too big to be classified as an update. It changes the masters of the file. That alone is enough reason to change the filename. The number of added records exceeds 1,000. The scope of intent of the file has increased from "fix bugs in Skyrim.esm and Update.esm" to "fix bugs in Skyrim.esm, Update.esm, Dawnguard.esm, HearthFires.esm, and Dragonborn.esm". In terms of logical scope, that's more than doubling.

And, once again, the unofficial patch team can name their files whatever they want. No one has any right to tell them otherwise, and it is absurd to think that anyone should. Filenames are not made with the intention of "creating an interface".

 

I'm just don't see where the confusion would come from. A quick glance at the metadata or the file's masters would determine which version you were dealing with. Almost every mod that I know of that has started out as individual files then gone on to release legendary version, has kept its original name. The ones that don't are generally more of a pain to deal with because anyone who is creating compatibility patches has to decides whether to update and/or support two different versions. In many cases, existing compatibility patches remain valid and functional if the mod doesn't change it's name. Though, again, if there is a huge restructuring or change to the record edits or FormIDs within the file, then it should probably change names since that restructuring will likely invalidate any patches regardless.

"Metadata, masters" shouldn't be necessary. When I look at my load order (a text document) I don't want to see a file and have to ask myself "is that this version of this file or this other COMPLETELY DIFFERENT version"? If they were to use the same filename I would have to forever call into question the existence of "Unofficial Skyrim Patch.esp" in every load order I ever saw ever again (and considering how popular these patches are, that's just about every load order). I don't want that. The UP team doesn't want that. If people don't like the new filename they can just continue using the old files (which will no longer be kept up-to-date).

 

But it isn't necessarily a different beast with regards to a specific mod. Take a house mod that alters Proudspire and has been forced to use USKP as a master only because of the single cookpot record that it adds. Chances are that one record will remain identical between USLEEP and USKP. If the file name wasn't changed, then that house mod would still function perfectly regardless of whether a player is using the older USKP or the newer legendary version. However, by changing the name, the author of that house mod now has the following options:
1) do nothing and allow their mod to become outdated and unusable for the growing majority of players
2) change the mod to support the new file name and drop support for the non-legendary version
3) update the mod for the legendary version and then continue supporting two versions of their file instead of one

"forced to use USKP as a master because of a single cookpot record that it adds". Uhhh... that doesn't make any sense. You are not forced to use a mod as a master unless you want to modify content from that mod or make use of content it adds. If you want to make a mod compatible with the USKP you make sure your changes in override records don't conflict with theirs, and that's it. The USKP doesn't add many "new object references" to the game, and if your mod does happen to conflict with one of those you need a compatibility patch, not a master dependency on your main mod. Assuming you use a compatibility patch, your new work is just to make a new compatibility patch for USLEEP, which makes sense because it's a new mod...

What about mods that rely on one of the unofficial DLC patches? What do they do for users that start using USLEEP? There is no EASY solution for everyone when it comes to this change. Instead of increasing complexity by leaving the filename the same and having people pray that things still operate the same, the filename should be changed so there isn't this crazy increase in complexity.

If the filename changes there's a clean break. Asking whether or not a mod works with USKP/USLEEP is as simple as looking at the masters. You don't retroactively force all mod authors to have made their mod mastered to USLEEP, which is a massive complexity increase for them and their users. You're operating under the assumption that FormIDs won't change, that USLEEP and USKP will be close to being equivalent... but let's face the facts, things rarely work out this well and there will be differences that will compound over time. And if USLEEP gets updated to a certain point past the USKP there may come a time where a master dependency to "Unofficial Skyrim Patch.esp" will work for USKP, but not for USLEEP, or vice versa. WHAT THEN?! Users will go absolutely insane. The forums will explode! The skies will rain fire!!!! IT WOULD BE MODDING APOCALYPSE!

Seriously. Not changing the filename is an insanely BAD idea. And the UP team has the right to do whatever the flip they want, so we need to stop discussing this like we have any say in what they do (because we don't).

 

Obviously if that mod author is no longer around, then #1 is going to happen by default. Some mod authors will choose #2 for simplicity and to ease their own work load. Others will try to support both versions because there are still people who don't have all DLCs and, quite frankly, because there is nothing about their house mod that requires the DLCs other than the Unofficial patch.

Would you be happy if there was an easy two-click solution to change a mod's master from Unofficial Skyrim Patch.esp to USLEEP.esp? Because I can make that. Then mod users/authors will have the power to change that at any time, effectively providing us with the advantages of the filename staying the same while not creating the mess that would be associated with it staying the same.

 

I think that forcing mod authors to update their mods and to indirectly make their mods dependent on all three DLC, when the mod wouldn't otherwise need to be dependent on any of the DLCs is simply the wrong choice. Referring back to the comments on community standards and unspoken contracts, I feel like this decision could be seen as a breach of that assumed contract.

Being reliant on the DLCs is a reasonable point... if we had modders who didn't have all the DLCs. The whole reason this idea is being executed on is because the polls have shown that a majority have all DLCs and would prefer a single ESP file for the unofficial patches.

Also, I can't help but continue wondering how many mods are actually master-dependent on the USKP. (or otherwise dependent on a file with any filename)

 

Also, just to be clear, I don't think for a second that this is going to be a devastating compatibility disaster for the community. As a mod author, I see it as a minor annoyance more than anything. There will be a handful of mods that this have some issues due to this since their authors aren't around anymore, and a probably several dozen others that will be affected and will simply adapt to the new file. The convenience of having a unified unofficial patch more than makes up for any small headaches created by it's implementation. If I seem to be arguing the point more strongly than seems warranted, it's mainly because I'm finding this discussion of compatibility and community standards to be rather interesting and enjoyable.

I think it's a fun discussion too. Apologies if I get a bit defensive in my post. It's just how I do discussions like this.

Edited July 27, 2015 by Mator

[hishutup]

I don't even understand how can people have so many mods they are hitting the limit. I'd say my game is quite modded, and I still am at 150-160 plugins.

I bet 90% of people who are in the 250ish range don't even know what mods they have installed, and simply downloaded every single piece of junk they could find.

You start with SRLE then look at REGS and use some other mods and BOOM!, suddenly you're at 300.

Also, patching the number of patches some mods have is only fun once.

 

Skyrim's limit is 255 so it isn't that big a deal. It's nice to keep that number down, but once your reaching that mark stability can be a nightmare. Fallout on the other hand is limited to about 135. The merge script and bashed patching is the only thing that saved us from having guides half their current size.

I found that load times are not only dependent on content but also plugin count, stability not so much.

 

Oh, I do understand and support the merge of the unofficial patches. I just can't understand merging, say, Unofficial Skyrim Patch, Book Covers Skyrim, and Wet and Cold together.

 

Because that is an idiotic and ludicrous merge.

 

The only issues that I am aware of is ETaC interiors is messed up and there is a script in wao for COT compatibility, I just merged the forms because its idiotic to do it the other way.

 

 

but I digress, each to their own.