Ramifications of BSA Extraction in Mod Organizer

[z929669]

This will graduate to wiki guide format once we get sufficient community input ...

---

 
I am creating this thread to address some important information about BSA extraction and related modding concepts that come up over an over again with respect to STEP's use of Mod Organizer (MO) and MO/STEP critics that disagree with STEP's general advocacy of BSA extraction and MO's BSA-extraction functionality. In order for this to be meaningful to all users, I am providing some background information required for understanding the ramifications of BSA extraction and why STEP advocates it while other respected modders do not.
 
WARNING: Extracting BSAs deviates from the intent and expectation of the respective mod provider, so it is fair to expect that once a user extracts a BSA supplied with a given mod, that mod's author has every right to refuse support on the basis that the user implementation is no longer the same mod. Further, if a user extracts vanilla Skyrim BSAs, ALL mod authors and the modding community at large have the right to refuse ANY support to said user!
 
NOTE: The STEP community follows Wrye's cathedral model of modding, and since we advocate experimentation and an open modding approach, we also try to encourage 'creativity' (but not flagrant stupidity, TYVM) and users are welcome to support each other in this community, and we'll do our best to help as we are able, particularly with regard to what we advise in any of our guides. After all, modding is a largely creative endeavor and a learning experience!

---

 
Executive Summary
 
STEP provides instruction for a range of methods to mod Skyrim. Novice users can follow the STEP Guide, which is pretty straight forward and adheres to conventional modding methods. More advanced users or users that want to take it further can follow the additional guides and employ some more advanced and less conventional techniques (like BSA extraction and texture optimization) that we have found to be viable and optimal for the 'perfectionistic' or 'adventurous' modder. Some of these techniques are traditionally the jobs of the mod providers rather than the mod users, but we encourage and empower our community to take on these tasks if they wish, because many mod providers do not optimize their mods, and we advocate customization of mods by the user.
 
STEP advocates BSA extraction because it allows for more granular control of the modded Skyrim (but see 'cons' below). Critics object that this practice 'breaks' the fundamental standards of install-order and load-order methodology that many mods and modding utilities are built around. Nevertheless, I and many STEP staff and members have not found this to be the case and propose that these concerns likely stem as a result of Mod Organizer users having problems and complaining and blaming the mod authors on mod threads like the USkP when something goes wrong with their game. Why MO users? Because MO is the only mod manager that exposes BSA extraction to any user that installs a mod with a BSA in it (basically, all MO users).

• STEP advocates BSA extraction (given the user understands the ramifications!)
  • It is a prerequisite to texture optimization (NOTE: repackaging vanilla BSAs is possible with CK's Archive.exe but problematic with BSAopt)
  • Used properly, it does not cause any load-order issues at all
  • It allows more granular control of a modded setup
  • It can theoretically lead to excess disk fragmentation (HDDs only, not SSDs)
  • It can theoretically reduce in-game performance if most/all game assets are loose files under a heavily-modded game
• BSA extraction is not 'bad' (NOTE: it can alter the intended behavior of a mod's interaction with other mods if used improperly)
• BSAs are not 'bad'
  • simplify mod distribution
  • simplify user maintenance (NOTE: particularly for manual installation and traditional mod management ... but NOT for Mod Organizer users!)
  • simplify support by mod providers
  • BSAs are the future trend in Bethesda modding, so best to get used to them

---

 
Install Order vs Load Order
 
A word about game assets: Game assets are any files (resources) that get used by Skyrim. Plugins (*.esm & *.esp) are special assets that are used by Skyrim to call upon other assets and to provide instructions for their use. Thus, plugins are the 'brain' of the game. The only plugins needed to play Skyrim are Skyrim.esm &  Update.esm. Other plugins are from the DLC add-ons or other mods made by the modding community like the USkPs, etc.
 
Install order: The order of mod files installed onto disk. If two mod packages contain the same file names along the same file paths (e.g., textures/blah.dds), then the last installed version overwrites (i.e., overrides) any previous version and is thus the version that will be used by the game if it is called upon by the game (via a plugin). These mod files can be anything at all (e.g., text, images, ... whatever), but only certain file types are used by the game: textures, meshes, scripts, plugins, etc. Therefore, install order affects what resources the game will use.
 
(NOTE: MO does not install mods to the data directory, but rather mods are extracted into mod folders within a user-specified location. MO creates a virtual /Data/ that appears to Skyrim as the actual /Data/, and it populates this virtual directory with mod assets from the install directory as specified by the MO installation priority. Otherwise, there is effectively no difference between MO and other mod managers, but this difference is fundamental and confers a significant advantage to MO users).
 
Load Order: The order that plugins are loaded into the game. Like install order, the last plugin loaded overrides all previous plugins. Since plugins reference assets within /Data/ by file name, there is potential for two different plugins to reference the same named resource.  Additionally, since plugins provide instructions as to the use of these resources, load order can also affect game behavior. Therefore, load order affects both what the game will use and how the game will use it.
 
What are BSAs?
 
Mandatory reading: read this important background information!
 
BSA: A proprietary archive of game assets that mirrors /Data/ directory structure. Thus, a BSA file is an archive exactly like a folder that is simply packaged as a file. The same is true of any ZIP or 7z archive.
 
How do BSAs Work?
 
For Skyrim to be 'aware' of a BSA, it must either be registered in Skyrim.ini or loaded with a plugin of same name. Once recognized, the game sees any BSA as part of /Data/ itself; however, when conflicts exist between files contained within a registered BSA, a plugin-loaded BSA or within /Data/ as loose files, things are a little trickier:

• Registered BSAs: These load at Skyrim start in the order that they are listed in Skyrim.ini, last loaded BSA 'wins' in event of resource conflicts of contents within.
• Plugin-loaded BSAs: These load when a new or saved game is loaded after Skyrim starts. Each BSA is loaded at the time the plugin of same name is loaded. So any BSA with content resource conflicts corresponding to a plugin will 'win' if its plugin is loaded after the conflicting plugin. Basically, these BSAs (and all of their asset content) are referenced by their plugin and loaded according to plugin load order. Plugin-loaded BSAs always 'win' where they conflict with Registered BSAs. The only exception is with respect to resources required at Skyrim start but before savegame (or new game) load, like No Menu and Loading Smoke.
• Note about loose files: Loose files always override same files inside of registered AND plugin-loaded BSAs!

 
Summarizing in terms of prioritization and load order ...
 
Skyrim asset priority:

1. Loose assets always win
2. Plugin-loaded BSAs win all but #1 (EXCEPTION: plugins are only loaded when a new or saved game is started, so plugin-loaded BSAs have zero priority with regard to pre-game assets)
3. Registered-BSA assets lose to all #1 & #2
4. Registered Skyrim BSAs and other official content and DLCs behave no differently than "after market", mod BSAs

Plugin/BSA load order:

1. Registered BSAs load according to list order in Skyrim.ini
2. Plugin-loaded BSAa load with respect to the corresponding plugin load order
3. Plugins load according to %USERPROFILE%/Appdata/Local/Skyrim/plugins.txt, which is managed by BOSS/LOOT

BSA Pros:

• Keep the Data directory clean and uncluttered (NOTE: this does not apply to MO users though, since MO uses the virtual file system).
• Allow easy mod management, since all of a mod's files are much simpler to identify and update or remove (mitigates user error= less support burden)
• Make it easier for mod authors to distribute and maintain control over how the mod functions (mitigates user error = less support burden)
• UPDATE:
• Better performance (NOTE: a lot of loose files slows down game startup, especially when using MO)
• Less disk usage (NOTE: BSAs can be compressed; HDD fragmentation is less of an issue)

BSA Cons:

• Removes an element of user-level control ... and many mod users are control freaks (STEP especially)
• Users can no longer efficiently see contents of a mod (NOTE: although Wrye Bash does expose this information, albeit with a performance hit ... is this functionality inherent or is it off by default??)
• Incentivizes mod authors to provide BSA 'hotfixes' as loose files (NOTE: This has undesireable ramifications for MO users due to behavior of BSA extraction in MO ... BSA extracts last, so loose file hotfix is overridden by original version within the BSA! EDIT: this is fixed in the current beta and next release of MO)
• Mod authors are forced to upload all files (the entire BSA) for any updates (all files are contained within a BSA), and users are forced to either download again or deal with the issue just previous if the mod author has supplied a 'hotfix'-type update.

BSAs & Steam Workshop
Steam Workshop only allows mods that use the BSA + ESP format. STEP finds this overly restrictive and unnecessarily 'controlling'. I personally resent it and only deal with Steam because it is the wrapper for Skyrim (unfortunately, IMO). The Steam Workshop and Steam-Skyrim community are valid entities that do not deserve to be totally ignored, but STEP does not recommend that it be used as a primary source for mods or modding information. The Nexus is the STEP-preferred source for all modding needs. For information, STEP is a good primary source, and we point to the best alternative sources, but here are a few others:

• TESAlliance
• Bethsoft Forums
• Unofficial Elder Scrolls Wiki (UESP)
• SkyrimSearch.com
• Tamriel Maps 

BSAs & Mod Organizer
 
Since Mod Organizer allows users to extract BSAs during mod installation, MO potentially obviates any functionality of registered or plugin-loaded BSAs. Thus, any mod that uses a BSA is effectively constrained henceforth by rules pertaining to loose files, so its assets are no longer linked to hierarchies of BSA registration order or plugin load order. This and the fact that all or user-specified mod resources can be loose and manageable by MO confers a clear advantage to the user.
 
BSA Extraction Pros:

• MO users have a much more granular level of asset control and can prioritize BSA contents at the loose-files level
• It is a prerequisite to texture optimization (NOTE: repackaging vanilla BSAs is possible with CK's Archive.exe but problematic with BSAopt)

Other issues can arise though, so only informed users that understand the ramifications should be using this functionality (unpacking BSAs). Following are some things to be aware of when unpacking BSAs (that mod authors intended to remain packed as delivered!).
 
UPDATE: There does not seem to be any need for standard users to extract mod BSAs in MO, because once can subvert the constraints of the standard load order/asset prioritization system from the Archives Tab:

• Plugin checked, BSA checked - Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
• Plugin checked, BSA unchecked - Follows plugin load order for all unchecked BSAs. All loose file assets will "overwrite." OTHER checked BSAs will NOT overwrite, which is why unchecking BSAs can lead to unpredictable results or dificult-to-resolve conflicts (hence the warning).
• Plugin unchecked, BSA checked - Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
• Plugin unchecked, BSA unchecked - As if the plugin and BSA don't even exist in the mod setup.
• Furthermore, MO will scan all mod BSAs (aside from those in /Data/) and include these assets in the Mod > Information > Conflicts tab. So in MO, BSAs effectively behave like loose files when checked in the Archives Tab! Mod developers will still find the BSA extraction functionality handy for testing purposes during production of updates to their existing BSA-packed mods or when developing new ones dependent on assets contained within BSAs. 

BSA Extraction Cons:

• BSA assets are now given loose files priority, so this alters the mod author's original design intent and may introduce false 'bugs' that nobody on any forums will likely want to or know how to diagnose or fix ;)
• BSA extraction in MO happens after loose files are installed. This means that any loose 'hotfixes' would be overwritten by the BSA version, which is outdated. EDIT: this is fixed in the current beta and next release of MO

MO exposes BSA extraction functionality using a prompt when a mod containing a BSA is first encountered. This functionality can henceforth be "always allowed" or selective, based on user preference in response to this prompt. Users that do not fully understand the ramifications of BSA extraction on the specific mods they are using together should not use this feature. If "automatic BSA extraction" is in effect, it can be reset from Settings (click the wrench icon in the toolbar) > Reset Dialogs > click 'yes' at the prompt.
 
First, STEP recommends that users NEVER "always allow" automatic BSA extraction ... why? Because there is no need to do this at all, since the granular functionality already resides within the Archive Tab. More importantly, because many unknown or unintended prioritization issues can come into play as described previously. It is always safer to use the BSA unless it will cause a de facto undesirable result.
 
BSAs that have optimized textures and can be overridden completely by downstream mods should stay inside BSAs (or repackaged using CK's Archive.exe). If assets from inside a BSA need to overwrite some mods and be overwritten by others, then sometimes it makes sense to extract the BSA. MO has a beautiful tool accessible from within its Archive Tab. If a BSA is present in the load order, it will appear in the Archives Tab. Leave it unchecked to allow it to behave normally and be loaded by its plugin (if the plugin is active), and check it to extract the BSA to the mod folder and effectively confer loose-files prioritization.
 
Critics of Mod Organizer and STEP (for Officially Advocating BSA Extraction) 

Some within the well-respected modding community are at odds with the idea of BSA extraction advocated by STEP and facilitated by MO. The most notable contingent is the USkP team. This is relatively old news and nothing that should be shocking, so please do not treat it that way. The reasons are not unfounded and actually valid. I bring this up solely to address the idea to remove BSA-extraction from MO that Tannin suggested if MO-detected load order issues are not resolved properly (by submitting a ticket). In fact, I created this thread to address this one issue as much as to address the knowledge gap that is the real cause of any issues associated with BSA extraction.
 
Some modders are more or less happy with MO's ability to invoke BSA extraction. Generally, mod authors who have gotten a lot of grief with respect to their mods --for problems caused by the ramifications of rampant BSA extraction-- seem to have more of a problem with MO (see note below!). This has been somewhat problematic for STEP and MO with respect to outsiders privy to the argument but not privy to STEP or MO ... never mind that the 'fault' should be shouldered solely by the unwitting mod user for invoking BSA extraction without understanding the ramifications of doing so ... and ours for not properly educating our user base to that effect (hence this thread).
 
Let it be said that the STEP modding community and the vast majority of modders are the kinds of users that use PCs instead of Macs and tend to be somewhat removed from computing 'norms' imposed by Apple and Microsoft and their ilk. In general, we do not like our control restricted in favor of provider control over our resources to make their lives simpler. We are generally in favor of digital freedom, open source software, and the honor system. Big Brother and his methods are generally unwelcome. Demonizing BSA extraction in general and removing it from MO in particular in order to enhance level-of-conrtol by mod providers would be a big mistake, as STEP itself somewhat relies on this feature (and will to a larger extent in the future). However, I think that it is very important that our users understand the ramifications of using BSA extraction, and we need to address explicitly in the STEP Guide.
 
The MO user base (not its 'critic' base) should guide Tannin's direction of MO, IMHO ... 
 
I want to explore what we do (and do not) know with regard to the ramifications of BSA extraction to our MO users and how best to make MO a broadly accepted utility among all within the modding community ... not just STEP. In order to do so, we must not dredge up combative arguments. Constructive argument is good for all respective modding endeavors, so what has been said in the past is water under the bridge. So keep it lively and fact filled ... but keep it polite and considerate!
 
Please report bugs as Tannin requests using the link above. Also please post to this thread and help us to improve the breadth and accuracy of this OP!
 

[z929669]

Okay I guess this is as good a time as ever to voice this. 

 

I do honestly think you are making a much larger issue out of this then it needs to be (Or I am just failing rather badly in even understanding the issues you have)! As I tried to tell in the vampire issue thread.. I was not able to reproduce it the way I loaded my assets! 

I use all (cleaned) official .esm´s (in overwrite) before any .esp and nothing selected in archives at all. 

In mod order then only optimized HDDLC are before USKP fixes followed by everything else in profiles that have that. Other then the texture.bsa´s nothing is unpacked. 

 

I have never had any issues with this method at all, even during the giant weekend spent looking through every .esp for conflicts and fixing them as I went along did I see any issues out of what would be expected in .esp conflicts. 

 

My Data dir is as pristine as the day I installed skyrim... I could delete MO and all mods and still start the game without any issues what so ever! 

This was the main reason I even started using MO! It could do this where all other managers would require a revalidation of the data dir to make sure. 

 

The most elegant solution if any would in my opinion be to just remove the archives tab since it apparently cause a lot of confusion for certain folk. 

 

So to sum up 

1: Data dir is as it is downloaded from steam 

2: Have a cleaned version of the official .esm´s in overwrite. 

3: Load the .esm´s before ANY .esp in priority order. 

4: ALL bsa´s from mods are unpacked. Only the texture ones from official are. 

 

That is how it works for me, and I really am confused how it can confuse anyone at all. 

 

So sorry if this all sounded a bit angry or annoyed etc... but I am really getting confused as to the overall issue you seem to have. 

You and Neo are using a valid solution it would appear, but ONLY if you are also including the BSA in Overwrite, no? From previous discussion on this thread and input from DoubleYou and Tannin, I thought that the DLC BSA assets were never treated as loose files (check or unchecked ... unless unpacked), and so would thus be overridden by USkP when both are present, unless USkP BSA is UNchecked and its plugin remains checked (then that forces USkP load by BOSS/LOOT priority, which is the only behavior allowed to official DLC).

 

... so either I have it wrong (in which case, correct behavior needs to be re-explained by Tannin et al, or I need to stop smoking crack), or you (and Neo) are mistaken

Kuldebar: 

 

So the solution would appear to be as simple as create that mod and/or use overwrite.. and add that to the guide. Instead of going about requiring that MO make a workaround for something that there already exist an elegant and simple solution for.

(How much simpler can a solution get then a simple copy paste?) 

 

Heck if STEP makes their own BSA´s with following .esm´s and/or .esp´s then this solution is perfect for it anyways! 

 

Even calling it a solution seems a bit odd, since it has been an option all along to begin with. 

Applying MO-specific workarounds seems less ideal to me though if we want broader acceptance of MO as a supported alternative to other mod managers. The USP team is (has been for some time now) speaking out against MO (and STEP) for not adhering to a standard that applies to other mod management utilities and more importantly: a standard that is explicitly considered in the development of the USPs and possibly other mods that conform to and are developed around Bethsoft standards. It IS a valid argument.

 

I just think that it is worthy that Tannin consider implementing some changes that, by default, bring MO into full alignment with the Bethsoft standards, so that out of the box, mod installation instructions pertain to MO users as they do for all others. I am not saying to remove all of the advance functionality that MO has. I am only suggesting that it be optional behavior and not default behavior. Tannin can either agree or disagree, but it would be great to hear his arguments for or against that so that STEP can present messaging consistent with his own on this subject.

 

As I said on the 2.2.9 staff dev thread, STEP provision of official content has some issues relating to Steam auto-update behavior and/or, addition of custom plugin content loaders, and/or unpacking of BSAs ... or basically treatment by MO users that differs from treatment by users of other mod managers. Just head over there and comment, because nobody has yet responded to these (I think) valid considerations that need solutions.

 

No, the DLC are not treated "specially." The Data folder is though and with good reason.

 

Move the DLC out of the Data folder and into the MO mods directory where the rest of your mods are located because the DLC are mods—official mods but still mods.

 

I'm impressed that he hasn't thrown his computer off a bridge yet. Or jumped himself.

By virtue of being in /Data/, the DLC ARE treated "specially." The statement is pure and simple fact.

 

I agree that copying all assets in /Data/ as MO mods would resolve all Prioritization compat issues for MO users

 

I assume you mean Tannin? FYI, this is a civil and valid thread that is broaching a valid topic that deserves consideration on the broader context. The issue is not going away. We are seeking to finally bring it out front for discussion so that --at the very least-- we can have an official position on the topic and address to the broader modding community by consensus. Tannin can do what he wants, but I think it is fallacy to avoid a subject so as not to "rock the boat". Boats need rocking, and I believe Tannin has his own opinion, and I am eager to hear it. Whatever he decides to do, it will not detract from STEP's use of MO as the default mod management utility, because it is the best and most extensible mod manager available. We just want to acknowledge the valid issue that MO subverts Bethsoft standards and does not appear to presently have a 'mode' that conforms to them. So many people are reading this thread and learning things for the first time (many of whom have been using MO and modding for quite a long time!)

[Kuldebar]

Combative arguments? Strange way of describing an accounting of the basic facts.

 

Again, it does no one any good for people to come into the thread and act like there this problem was readily and easily solved by some well known method. It wasn't. In fact, the problem wasn't even being properly recognized for the most part.

 

So, I suggest promoting emotional sensitivity as a legitimate reaction to a simple straight-forward discussion is not helpful and only serves to draw attention away from the topic by making it personal.

 

@z929669 Thanks for replying to the DLC "specially" point, I suppose I should have been just as direct in my earlier response.

Edited May 25, 2014 by Kuldebar

[z929669]

@Kludebar

NP, I think everything is 'fine' on this thread, so no worries. People can get a little prickly and heated when something they are passionate about is being challenged.

 

@Wolverine

Thanks, and great points! Also, I intended the OP to be the condensation of this threads contents, so all moderators and site staff: PLEASE help me to keep it updated with relevant facts. I am writing updates in green text, and using strike-through to eliminate falsehood ;) Really ... whatever you guys think is important, and you don't have to agree with any of my own opinions! The thread should contain relevant facts that we (and Tannin) can reference.

 

@Aiyen and SRB and Kelmych

I think that proposed solutions about current workaround and pros/cons of those workarounds should go in there, so have at it if you please (but do look at my argument first, just in case it makes sense or is worth considering).

[Aiyen]

You and Neo are using a valid solution it would appear, but ONLY if you are also including the BSA in Overwrite, no? From previous discussion on this thread and input from DoubleYou and Tannin, I thought that the DLC BSA assets were never treated as loose files (check or unchecked ... unless unpacked), and so would thus be overridden by USkP when both are present, unless USkP BSA is UNchecked and its plugin remains checked (then that forces USkP load by BOSS/LOOT priority, which is the only behavior allowed to official DLC).

 

... so either I have it wrong (in which case, correct behavior needs to be re-explained by Tannin et al, or I need to stop smoking crack), or you (and Neo) are mistaken

Z sorry for doing this all in here.. but since you split your response over two threads I really do not feel like repeating myself in multiple locations. 

 

To clarify. 

No bsa´s are moved on my setup. Like I stated earlier only the texture .bsa is unpacked due to being optimized... the only files I have in overwrite are the cleaned .esm´s. Since ALL my assets are loose then all I have to worry about is the priority order. 

 

As for this being a work around... and somehow is forcing every mod author to make separate install instructions if you use MO makes no sense to me. 

 

You copy the official .esm´s in once as part of the MO install and are done... the data dir is left intact and unspoiled, and you can install mods without these weird issues since the priority order will make sure everything lower automatically overwrites. 

 

I have simply used what MO has written in the bottom part of the archives tab since it came out. If you only have loose files then everything will always overwrite the vanilla .bsa´s regardless if they are marked or not. 

 

Copying a few files when first installing, and setting everything following that to be unpacked does not seem like a massive workaround for me.. just normal installation procedure. The only issues that can come from this is if sorting software create their lists based on assumptions of how NMM works. Which I am still fairly certain is the core of the issue... not MO itself or how it does things. 

 

If it was up to me the way MO does it would be the standard since it is logical, simple to understand and allow for efficient modding! 

[GSDFan]

OK I have to speak up here. @Aiyen Said

 

 

To clarify. 

No bsa´s are moved on my setup. Like I stated earlier only the texture .bsa is unpacked due to being optimized... the only files I have in overwrite are the cleaned .esm´s. Since ALL my assets are loose then all I have to worry about is the priority order.

Correct me if I am wrong in interpreting your use of the word "overwrite" as meaning a mod and not the overwrite folder.

 

Keeping files in the Overwrite folder is a not a good idea, especially the DLC files. Unless something has changed, the data directory has the lowest priority. Next in line is the MO mods directory priority ordering followed by the Overwrite folder, which always wins a conflict.

 

I ran a simple test on an unclean Dawnguard.esm from the data directory. Loot reports that it needs cleaning. I next have my cleaned masters mod enabled and loot reports all is good. Next I copied the unclean Dawnguard.esm to the overwrite folder and left the cleaned masters  mod enabled and ran loot, again it reported it a unclean.

 

In conclusion the overwrite folder should NOT be used to house any files unless you want it to win a conflict.

[z929669]

@Aiyen

I see now that I did not consider that you unpacked all BSAs, including DLC? Please confirm, because this would explain it. If that is the case, then I would argue that we should NOT be unpacking BSAs except where it makes sense for conflict resolution ... and NEVER to unpack the base game BSAs, official DLCs or the USP BSAs. Even Tannin has recommended this I believe (an he goes further in stating that BSA extraction should not be default MO behavior).

 

My point is that as a general practice applicable to all user systems and all mod management utilities (and even manual installation), we should be recommending methods (in the STEP guide) that generally apply and not just to MO users (MO users can handle things in a number of viable ways, including yours and my own variant method, which differs). We may not be doing so exactly ATM, but we are learning a few things right now.

 

STEP should be recommending generalized and standard "best practice" wherever possible for purposes of compatibility and applicability. We are nailing down some of these best practices here in this thread, and my point is that your methods, while viable and efficient for you, are not "best practice." I am still trying to figure that out for myself ;)

 

oh, and I agree with GSDfan that the Overwrite folder should not be used to house mods, which is clearly not MO best practice (I would never use it that way, simply because I view that location as highly volatile and a source for 'snapshots' of things that I want to create 'mods' from).

[Aiyen]

Okay I hope this is clear to avoid all confusion then. For the sake of this discussion. 

 

Only the textures.bsa is unpacked and optimized and the result put into its own mod! The rest are in the data folder save and sound. Only the main skyrim file .bsa´s are ticked. All the DLC ones are not! 

 

In my MO overwrite folder I have the cleaned official .esm files (So Update.esm, DG.esm, DB.esm and HF.esm). I could move them to a separate mod (Like Neo does), the only reason I have never done this is simply because I am lazy and never had a reason to do it. The load priority order takes precedence anyways. The only thing you should not have in overwrite is assets you want other stuff to be able to overwrite. 

 

The priority order is the following 

Skyrim.esm, Update.esm, Dawnguard.esm, Hearthfire.esm, Dragonborn.esm, ... everything else. 

 

On the left mod list pane the priority order is 

SKSE, HDDLC textures, ... everything else. 

 

Hope that all makes sense. 

[Kuldebar]

The way I am using MO as of today is more in line with how nearly all typical users will use it.

 

All Bethesda Official files in their default location (cleaned, of course)

 

All mods installed via MO and LOOT sorted with logical left hand window sorting based on both LOOT and discretion.

 

 

 

 

 

My objective in pointing this out is not to imply Aiyen is wrong or anything just that for the sake of discussion we need to establish the baseline. I think it is fair and accurate to say my screen shots illustrate a typical MO User setup.

 

From this established baseline, what elegant solutions present themselves, or are any "solutions" even now necessary? Since BSA Extraction has been strictly limited in use and certainly not being used for the Unofficial Patch BSA's, aren't we avoiding the "vampire fix" weirdness so exhaustively discussed elsewhere.  (I use the Vampire Fix as a marker for the how the problem can manifest)

Edited May 25, 2014 by Kuldebar

[Aiyen]

Just to clarify something for me... do these issues also happen if all unofficial .esp´s are moved after all the .esm ? In which case it is not even a BSA issue but a sorting one. 

 

Getting a bit tried now so sorry for asking this here! 

 

 

Just thought it would be nice to eliminate sorting issues as potential cause once and for all. 

[Kuldebar]

Just to clarify something for me... do these issues also happen if all unofficial .esp´s are moved after all the .esm ? In which case it is not even a BSA issue but a sorting one. 

 

Getting a bit tried now so sorry for asking this here! 

 

 

Just thought it would be nice to eliminate sorting issues as potential cause once and for all. 

I don't know, I always used the "official" sorting instructions from the unofficial patch team and BOSS/LOOT.

 

For a short period of time, BOSS was messing up the USKP file ordering, but that was resolved and BOSS and LOOT now follow the same standards. I have never had a reason to deviate from it.

[Aiyen]

Well seems like it would be worth testing out before continuing. If the issue is in fact not because of bsa´s but sorting then the issue is entirely different from what the we are discussing now. Just think it is worth to eliminate it. Should not take more then a single test run. 

 

If it does resolve the issue then this discussion should more go over to how to deal with what to suggest in relation to sorting over what to suggest with relation to bsa´s. 

[Kuldebar]

I'd agree but the vampire fix problem was discovered to be caused, not by load ordering, but by the way MO treats the official DLC's when they are installed in the default universal manner and the Unofficial Patch BSA'a were extracted. Fixes that would not normally have been forwarded if the user had Dawnguard installed were indeed prioritized and caused a problem.

 

Leaving the Unofficial Skyrim Patch in BSA form resolved this issue, much to the consternation of people like me.

 

In electronics, we were taught a term "electrically the same" meaning no difference of potential.

 

Up until the last few days, I had believed that MO would allow me to have my BSA's extracted and be "electrically the same" in regards to the application of the files if the files were placed in their proper order (left-hand pane and right-hand pane with ESP's enabled).

Edited May 25, 2014 by Kuldebar

[fireundubh]

The USP team is (has been for some time now) speaking out against MO (and STEP) for not adhering to a standard that applies to other mod management utilities and more importantly: a standard that is explicitly considered in the development of the USPs and possibly other mods that conform to and are developed around Bethsoft standards. It IS a valid argument.

Arthmoor doesn't like MO because he still thinks you can drag-and-reorder BSAs in the Archives tab as you once could in older versions of MO. That's it. That's the only reason behind his hate for MO. Tannin has had lengthy arguments with Arthmoor in the Bethesda Softworks forum, as have I in various Nexus threads. He's not familiar with MO at all. And, like Tannin said, I don't think MO should be designed, or redesigned, for the people who aren't drinking the same Kool-Aid. You can feed only the hungry, so why waste your food? I just beat that metaphor to death.

 

By virtue of being in /Data/, the DLC ARE treated "specially." The statement is pure and simple fact.

No, technically, it's not the same at all. For the DLC to be treated, specifically, as excluded files (i.e., excluded from the way mods are treated), Tannin would have to specifically account for each file in the code. The fact that you can resolve Kuldebar's issue by moving, copying, or linking the DLC outside the Data folder indicates that the Data folder itself is excluded, not the files. You could place any other BSA in the Data folder and, most likely, achieve the same effect. I don't know why the Data folder was excluded like so, but I'd wager that the Data folder is therefore protected from Mod Organizer operations in order to truly keep the Data folder pristine, as Kuldebar would say.

 

We just want to acknowledge the valid issue that MO subverts Bethsoft standards and does not appear to presently have a 'mode' that conforms to them.

Bethesda's standards would have you installing mods directly into the Data folder. MO was designed to, effectively, subvert that practice because it's a bad practice. You're proposing that Tannin create an inconsistency to achieve consistency—that Tannin should commit to a not insignificant rewrite of a core system to specifically exclude the DLC from the rule that protects the Data folder, in order to conform to the way things are done as part of a bad practice. I disagree with your suggestion and its basis, but you know, at the end of day, how does agreeing or disagreeing matter? With regard to MO, Tannin is free to do what he wants; everything else is immaterial. At this point, I think more than enough has been said on both "sides" and we're just talking in circles. Edited May 25, 2014 by fireundubh

[Razorsedge877]

  I know I am not a coder or computer tech but I have been following these posts to the best of my ability. I am pretty sure I read a post from Tannin that says he is pretty much against mod extraction. I did an install of Skyrim Revisited and if I extract all of the mods I get a PMOP warning. If I don't extract the BSAs I don't get any PMOP warning. At the risk of sounding dumb because I know I'm not a computer guy I think people are trying to make this more difficult than it should be.

  If you ask me the only mods that affect the PMOP warning are game changing mods (Which have the scripts). Unless it is a game changing mod that affects textures that you need to hide for earlier texture mods I don't understand why we need to extract all of the BSAs. Now on the other hand texture mods shouldn't have scripts so they should be able to be extracted with no issues and really the only ones that would need extracted are texture mods that are later in the install order that need to have textures hidden if you want to a texture from an earlier mod.

I know you guys get tired of explaining how things work but please if my thinking is wrong explain to me why.

[Kuldebar]

  I know I am not a coder or computer tech but I have been following these posts to the best of my ability. I am pretty sure I read a post from Tannin that says he is pretty much against mod extraction. I did an install of Skyrim Revisited and if I extract all of the mods I get a PMOP warning. If I don't extract the BSAs I don't get any PMOP warning. At the risk of sounding dumb because I know I'm not a computer guy I think people are trying to make this more difficult than it should be.

  If you ask me the only mods that affect the PMOP warning are game changing mods (Which have the scripts). Unless it is a game changing mod that affects textures that you need to hide for earlier texture mods I don't understand why we need to extract all of the BSAs. Now on the other hand texture mods shouldn't have scripts so they should be able to be extracted with no issues and really the only ones that would need extracted are texture mods that are later in the install order that need to have textures hidden if you want to a texture from an earlier mod.

I know you guys get tired of explaining how things work but please if my thinking is wrong explain to me why.

Yes, he did state that, as you said, he is "against" BSA Extraction. Unfortunately the articulation of why he was against it was not so very clear and honestly seemed to be rather arbitrarily dismissive in general which for people who like to think things through is just begging the question to the nth degree.

 

After actual conversations. albeit somewhat tedious in nature, and the sharing of information and experiences, I have come to the conclusion that BSA Extractions messes up the execution of some things like the Unofficial Patches.