Ramifications of BSA Extraction in Mod Organizer

[z929669]

This will graduate to wiki guide format once we get sufficient community input ...

---

 
I am creating this thread to address some important information about BSA extraction and related modding concepts that come up over an over again with respect to STEP's use of Mod Organizer (MO) and MO/STEP critics that disagree with STEP's general advocacy of BSA extraction and MO's BSA-extraction functionality. In order for this to be meaningful to all users, I am providing some background information required for understanding the ramifications of BSA extraction and why STEP advocates it while other respected modders do not.
 
WARNING: Extracting BSAs deviates from the intent and expectation of the respective mod provider, so it is fair to expect that once a user extracts a BSA supplied with a given mod, that mod's author has every right to refuse support on the basis that the user implementation is no longer the same mod. Further, if a user extracts vanilla Skyrim BSAs, ALL mod authors and the modding community at large have the right to refuse ANY support to said user!
 
NOTE: The STEP community follows Wrye's cathedral model of modding, and since we advocate experimentation and an open modding approach, we also try to encourage 'creativity' (but not flagrant stupidity, TYVM) and users are welcome to support each other in this community, and we'll do our best to help as we are able, particularly with regard to what we advise in any of our guides. After all, modding is a largely creative endeavor and a learning experience!

---

 
Executive Summary
 
STEP provides instruction for a range of methods to mod Skyrim. Novice users can follow the STEP Guide, which is pretty straight forward and adheres to conventional modding methods. More advanced users or users that want to take it further can follow the additional guides and employ some more advanced and less conventional techniques (like BSA extraction and texture optimization) that we have found to be viable and optimal for the 'perfectionistic' or 'adventurous' modder. Some of these techniques are traditionally the jobs of the mod providers rather than the mod users, but we encourage and empower our community to take on these tasks if they wish, because many mod providers do not optimize their mods, and we advocate customization of mods by the user.
 
STEP advocates BSA extraction because it allows for more granular control of the modded Skyrim (but see 'cons' below). Critics object that this practice 'breaks' the fundamental standards of install-order and load-order methodology that many mods and modding utilities are built around. Nevertheless, I and many STEP staff and members have not found this to be the case and propose that these concerns likely stem as a result of Mod Organizer users having problems and complaining and blaming the mod authors on mod threads like the USkP when something goes wrong with their game. Why MO users? Because MO is the only mod manager that exposes BSA extraction to any user that installs a mod with a BSA in it (basically, all MO users).

• STEP advocates BSA extraction (given the user understands the ramifications!)
  • It is a prerequisite to texture optimization (NOTE: repackaging vanilla BSAs is possible with CK's Archive.exe but problematic with BSAopt)
  • Used properly, it does not cause any load-order issues at all
  • It allows more granular control of a modded setup
  • It can theoretically lead to excess disk fragmentation (HDDs only, not SSDs)
  • It can theoretically reduce in-game performance if most/all game assets are loose files under a heavily-modded game
• BSA extraction is not 'bad' (NOTE: it can alter the intended behavior of a mod's interaction with other mods if used improperly)
• BSAs are not 'bad'
  • simplify mod distribution
  • simplify user maintenance (NOTE: particularly for manual installation and traditional mod management ... but NOT for Mod Organizer users!)
  • simplify support by mod providers
  • BSAs are the future trend in Bethesda modding, so best to get used to them

---

 
Install Order vs Load Order
 
A word about game assets: Game assets are any files (resources) that get used by Skyrim. Plugins (*.esm & *.esp) are special assets that are used by Skyrim to call upon other assets and to provide instructions for their use. Thus, plugins are the 'brain' of the game. The only plugins needed to play Skyrim are Skyrim.esm &  Update.esm. Other plugins are from the DLC add-ons or other mods made by the modding community like the USkPs, etc.
 
Install order: The order of mod files installed onto disk. If two mod packages contain the same file names along the same file paths (e.g., textures/blah.dds), then the last installed version overwrites (i.e., overrides) any previous version and is thus the version that will be used by the game if it is called upon by the game (via a plugin). These mod files can be anything at all (e.g., text, images, ... whatever), but only certain file types are used by the game: textures, meshes, scripts, plugins, etc. Therefore, install order affects what resources the game will use.
 
(NOTE: MO does not install mods to the data directory, but rather mods are extracted into mod folders within a user-specified location. MO creates a virtual /Data/ that appears to Skyrim as the actual /Data/, and it populates this virtual directory with mod assets from the install directory as specified by the MO installation priority. Otherwise, there is effectively no difference between MO and other mod managers, but this difference is fundamental and confers a significant advantage to MO users).
 
Load Order: The order that plugins are loaded into the game. Like install order, the last plugin loaded overrides all previous plugins. Since plugins reference assets within /Data/ by file name, there is potential for two different plugins to reference the same named resource.  Additionally, since plugins provide instructions as to the use of these resources, load order can also affect game behavior. Therefore, load order affects both what the game will use and how the game will use it.
 
What are BSAs?
 
Mandatory reading: read this important background information!
 
BSA: A proprietary archive of game assets that mirrors /Data/ directory structure. Thus, a BSA file is an archive exactly like a folder that is simply packaged as a file. The same is true of any ZIP or 7z archive.
 
How do BSAs Work?
 
For Skyrim to be 'aware' of a BSA, it must either be registered in Skyrim.ini or loaded with a plugin of same name. Once recognized, the game sees any BSA as part of /Data/ itself; however, when conflicts exist between files contained within a registered BSA, a plugin-loaded BSA or within /Data/ as loose files, things are a little trickier:

• Registered BSAs: These load at Skyrim start in the order that they are listed in Skyrim.ini, last loaded BSA 'wins' in event of resource conflicts of contents within.
• Plugin-loaded BSAs: These load when a new or saved game is loaded after Skyrim starts. Each BSA is loaded at the time the plugin of same name is loaded. So any BSA with content resource conflicts corresponding to a plugin will 'win' if its plugin is loaded after the conflicting plugin. Basically, these BSAs (and all of their asset content) are referenced by their plugin and loaded according to plugin load order. Plugin-loaded BSAs always 'win' where they conflict with Registered BSAs. The only exception is with respect to resources required at Skyrim start but before savegame (or new game) load, like No Menu and Loading Smoke.
• Note about loose files: Loose files always override same files inside of registered AND plugin-loaded BSAs!

 
Summarizing in terms of prioritization and load order ...
 
Skyrim asset priority:

1. Loose assets always win
2. Plugin-loaded BSAs win all but #1 (EXCEPTION: plugins are only loaded when a new or saved game is started, so plugin-loaded BSAs have zero priority with regard to pre-game assets)
3. Registered-BSA assets lose to all #1 & #2
4. Registered Skyrim BSAs and other official content and DLCs behave no differently than "after market", mod BSAs

Plugin/BSA load order:

1. Registered BSAs load according to list order in Skyrim.ini
2. Plugin-loaded BSAa load with respect to the corresponding plugin load order
3. Plugins load according to %USERPROFILE%/Appdata/Local/Skyrim/plugins.txt, which is managed by BOSS/LOOT

BSA Pros:

• Keep the Data directory clean and uncluttered (NOTE: this does not apply to MO users though, since MO uses the virtual file system).
• Allow easy mod management, since all of a mod's files are much simpler to identify and update or remove (mitigates user error= less support burden)
• Make it easier for mod authors to distribute and maintain control over how the mod functions (mitigates user error = less support burden)
• UPDATE:
• Better performance (NOTE: a lot of loose files slows down game startup, especially when using MO)
• Less disk usage (NOTE: BSAs can be compressed; HDD fragmentation is less of an issue)

BSA Cons:

• Removes an element of user-level control ... and many mod users are control freaks (STEP especially)
• Users can no longer efficiently see contents of a mod (NOTE: although Wrye Bash does expose this information, albeit with a performance hit ... is this functionality inherent or is it off by default??)
• Incentivizes mod authors to provide BSA 'hotfixes' as loose files (NOTE: This has undesireable ramifications for MO users due to behavior of BSA extraction in MO ... BSA extracts last, so loose file hotfix is overridden by original version within the BSA! EDIT: this is fixed in the current beta and next release of MO)
• Mod authors are forced to upload all files (the entire BSA) for any updates (all files are contained within a BSA), and users are forced to either download again or deal with the issue just previous if the mod author has supplied a 'hotfix'-type update.

BSAs & Steam Workshop
Steam Workshop only allows mods that use the BSA + ESP format. STEP finds this overly restrictive and unnecessarily 'controlling'. I personally resent it and only deal with Steam because it is the wrapper for Skyrim (unfortunately, IMO). The Steam Workshop and Steam-Skyrim community are valid entities that do not deserve to be totally ignored, but STEP does not recommend that it be used as a primary source for mods or modding information. The Nexus is the STEP-preferred source for all modding needs. For information, STEP is a good primary source, and we point to the best alternative sources, but here are a few others:

• TESAlliance
• Bethsoft Forums
• Unofficial Elder Scrolls Wiki (UESP)
• SkyrimSearch.com
• Tamriel Maps 

BSAs & Mod Organizer
 
Since Mod Organizer allows users to extract BSAs during mod installation, MO potentially obviates any functionality of registered or plugin-loaded BSAs. Thus, any mod that uses a BSA is effectively constrained henceforth by rules pertaining to loose files, so its assets are no longer linked to hierarchies of BSA registration order or plugin load order. This and the fact that all or user-specified mod resources can be loose and manageable by MO confers a clear advantage to the user.
 
BSA Extraction Pros:

• MO users have a much more granular level of asset control and can prioritize BSA contents at the loose-files level
• It is a prerequisite to texture optimization (NOTE: repackaging vanilla BSAs is possible with CK's Archive.exe but problematic with BSAopt)

Other issues can arise though, so only informed users that understand the ramifications should be using this functionality (unpacking BSAs). Following are some things to be aware of when unpacking BSAs (that mod authors intended to remain packed as delivered!).
 
UPDATE: There does not seem to be any need for standard users to extract mod BSAs in MO, because once can subvert the constraints of the standard load order/asset prioritization system from the Archives Tab:

• Plugin checked, BSA checked - Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
• Plugin checked, BSA unchecked - Follows plugin load order for all unchecked BSAs. All loose file assets will "overwrite." OTHER checked BSAs will NOT overwrite, which is why unchecking BSAs can lead to unpredictable results or dificult-to-resolve conflicts (hence the warning).
• Plugin unchecked, BSA checked - Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
• Plugin unchecked, BSA unchecked - As if the plugin and BSA don't even exist in the mod setup.
• Furthermore, MO will scan all mod BSAs (aside from those in /Data/) and include these assets in the Mod > Information > Conflicts tab. So in MO, BSAs effectively behave like loose files when checked in the Archives Tab! Mod developers will still find the BSA extraction functionality handy for testing purposes during production of updates to their existing BSA-packed mods or when developing new ones dependent on assets contained within BSAs. 

BSA Extraction Cons:

• BSA assets are now given loose files priority, so this alters the mod author's original design intent and may introduce false 'bugs' that nobody on any forums will likely want to or know how to diagnose or fix ;)
• BSA extraction in MO happens after loose files are installed. This means that any loose 'hotfixes' would be overwritten by the BSA version, which is outdated. EDIT: this is fixed in the current beta and next release of MO

MO exposes BSA extraction functionality using a prompt when a mod containing a BSA is first encountered. This functionality can henceforth be "always allowed" or selective, based on user preference in response to this prompt. Users that do not fully understand the ramifications of BSA extraction on the specific mods they are using together should not use this feature. If "automatic BSA extraction" is in effect, it can be reset from Settings (click the wrench icon in the toolbar) > Reset Dialogs > click 'yes' at the prompt.
 
First, STEP recommends that users NEVER "always allow" automatic BSA extraction ... why? Because there is no need to do this at all, since the granular functionality already resides within the Archive Tab. More importantly, because many unknown or unintended prioritization issues can come into play as described previously. It is always safer to use the BSA unless it will cause a de facto undesirable result.
 
BSAs that have optimized textures and can be overridden completely by downstream mods should stay inside BSAs (or repackaged using CK's Archive.exe). If assets from inside a BSA need to overwrite some mods and be overwritten by others, then sometimes it makes sense to extract the BSA. MO has a beautiful tool accessible from within its Archive Tab. If a BSA is present in the load order, it will appear in the Archives Tab. Leave it unchecked to allow it to behave normally and be loaded by its plugin (if the plugin is active), and check it to extract the BSA to the mod folder and effectively confer loose-files prioritization.
 
Critics of Mod Organizer and STEP (for Officially Advocating BSA Extraction) 

Some within the well-respected modding community are at odds with the idea of BSA extraction advocated by STEP and facilitated by MO. The most notable contingent is the USkP team. This is relatively old news and nothing that should be shocking, so please do not treat it that way. The reasons are not unfounded and actually valid. I bring this up solely to address the idea to remove BSA-extraction from MO that Tannin suggested if MO-detected load order issues are not resolved properly (by submitting a ticket). In fact, I created this thread to address this one issue as much as to address the knowledge gap that is the real cause of any issues associated with BSA extraction.
 
Some modders are more or less happy with MO's ability to invoke BSA extraction. Generally, mod authors who have gotten a lot of grief with respect to their mods --for problems caused by the ramifications of rampant BSA extraction-- seem to have more of a problem with MO (see note below!). This has been somewhat problematic for STEP and MO with respect to outsiders privy to the argument but not privy to STEP or MO ... never mind that the 'fault' should be shouldered solely by the unwitting mod user for invoking BSA extraction without understanding the ramifications of doing so ... and ours for not properly educating our user base to that effect (hence this thread).
 
Let it be said that the STEP modding community and the vast majority of modders are the kinds of users that use PCs instead of Macs and tend to be somewhat removed from computing 'norms' imposed by Apple and Microsoft and their ilk. In general, we do not like our control restricted in favor of provider control over our resources to make their lives simpler. We are generally in favor of digital freedom, open source software, and the honor system. Big Brother and his methods are generally unwelcome. Demonizing BSA extraction in general and removing it from MO in particular in order to enhance level-of-conrtol by mod providers would be a big mistake, as STEP itself somewhat relies on this feature (and will to a larger extent in the future). However, I think that it is very important that our users understand the ramifications of using BSA extraction, and we need to address explicitly in the STEP Guide.
 
The MO user base (not its 'critic' base) should guide Tannin's direction of MO, IMHO ... 
 
I want to explore what we do (and do not) know with regard to the ramifications of BSA extraction to our MO users and how best to make MO a broadly accepted utility among all within the modding community ... not just STEP. In order to do so, we must not dredge up combative arguments. Constructive argument is good for all respective modding endeavors, so what has been said in the past is water under the bridge. So keep it lively and fact filled ... but keep it polite and considerate!
 
Please report bugs as Tannin requests using the link above. Also please post to this thread and help us to improve the breadth and accuracy of this OP!
 

[EssArrBee]

Um... so, just to be sure I understand correctly. What you're saying is that if we install a mod that adds files in a loose file format it won't matter that we did so because the .bsa containing the same files takes priority? Please correct me if I'm wrong (I am operating with a headache and need to go to bed but that's what I'm reading atm).

 

At ESRB I'm fairly certain it is allowed or at least it's a grey area; I doubt they could prevent us from uploading them somewhere (so long as we say where the came from) without changing their TOS and End-User agreements.

He's saying that install order matters with MO because the files in the BSA end up being the ones used by the VFS if the BSA is installed after the loose files.

[z929669]

@Tannin

Thanks for clarifying. I too am beginning to think that the functionality that you built into MO, while very clever as Kelmych states, it is also very confusing and contrary to standards. I like the idea of having this functionality as an option but maybe not as default behavior ... and even more importantly, we need to get the exact functionality documented, as I still have many questions (see my last post).

 

What seems clearer now is that MO effectively treats BSAs as loose files that are either prioritized by the plugin (Archive Tab BSA unchecked) or by the install position (Archive Tab BSA checked) ... is this correct? What about when the plugin AND the BSA are unchecked (warning triangle). Does MO read the BSA content and show the conflicts with loose files? If so, does the conflict resolution change if the BSA is checke or unchecked in the Archives Tab?

 

I think that this subversion could be very powerful, as no other tool can do this, but perhaps it should also be moved to plugin status (and definitely the documentation needs to be much clearer, which I and others can help with, given that we understand fully).

 

Please feel free to correct any inconsistencies in the OP! As a moderator of these forums, you should be able to edit that to correct any fiction that I have written and turn it into fact. Otherwise, if I am confident I understand, I will do it myself.

 

@Kalmych & SRB

Vano89 set the precedence (as well as others like alt3rn1ty's vanilla reduced textures) by altering the vanilla assets and distributing them on the Nexus. What I am proposing is no different, and after all this time, it would seem that Bethesda and Dark0ne are OK with this, probably because the textures are altered and not digitally identical. Many of the assets used by Nexus mods are alterations of the originals. As long as we only include changed textures, I think we are at least in compliance with many other Nexus mod practices. Let's do it eh?

[Garfink]

@ Tannin

 

Query:  Could you implement a button in MO, that attempts to automatically reorder the order (of the left pane) to resolve the "Potential mod order problems".  At the moment, it takes forever sometimes to rearrange the mods manually.  

[Tannin]

Um... so, just to be sure I understand correctly. What you're saying is that if we install a mod that adds files in a loose file format it won't matter that we did so because the .bsa containing the same files takes priority? Please correct me if I'm wrong (I am operating with a headache and need to go to bed but that's what I'm reading atm).

What I'm saying is:- If the mod with the bsa          is higher in the installation order than the one with loose files then the bsa          takes precedence- If the mod with the loose files is higher in the installation order than the one with bsa          then the loose files take precedence Or, to phrase it more plainly (like I did before)- If a mod has higher installation order then it takes precedence. The format just does not matter I honestly don't understand how MO is complicated when it gets rid of a whole category of complexity... @z929669: If esp and bsa are unchecked then the bsa is not loaded and can thus not conflict

 

and definitely the documentation needs to be much clearer, which I and others can help with, given that we understand fully).

 

WHAT do I document? That a certain problem people may have read on the internets does not actually affect them? I can hardly start documenting unusual things MO does not do.

Also, sorry to say this: documentation is worthless. People don't read it (well, there may be a few but certainly the majority of my users don't, this very discussion is proof of that).

 

@Garfink: I had such a button in the very first version that had the potential mod order warning but doubleyou convinced me to remove it. Don't know which thread that was in.

[Razorsedge877]

Question for Tannin.

I am unclear on one thing. If I do not extract a BSA on a mod am I still able to Hide texture files in that mods file tree so that a lower mods file can be used instead. I keep seeing on here you can only hide files if you extract BSAs but I'm not sure I agree with that

[Aiyen]

As tannin has said 

 

The only defining factor is the install priority... It does not matter if one use BSA or loose. It is logical, and I fail to see where the issue is. 

 

I can unpack my bsa´s get the textures, meshes whatever and create my "custom test mod" just before overwrite and make sure that the assets I work on are loaded in the game, and I can remove them afterwards and make sure that the vanilla contents take over again. 

 

As it is now it is a convenience that MO handles the unpacking of bsa´s... I hope it stays that way, even if it is hidden and requires activation. 

 

 

I must admit that as long as only the install priority is what you should sort after I fail to see how any issues can ever arise unless people sort by name and have their list entirely weird. Even custom assets would be loaded to the correct file path. 

[z929669]

 

@DoubleYou (and Tannin or whomever)

I am afraid that I am still not clear on how MO handles BSAs. If I take the explanation about BSA priorities in the MO guide, then

• It would seem that MO can prioritize BSA content (as a solid block of assets) according to the install prioritization rather than the plugin prioritization, thus, decoupling a BSA from its plugin.
• When a BSA in the Archives Tab is checked, it allegedly loads according to install prioritization, but if it is unchecked, it is loaded according to its plugin prioritization ... this does not jive at all with the caption on the bottom of the Archives Tab window ("Marked archives ( ) are still loaded on Skyrim but the regular file override mechanism will apply: Loose files override BSAs, no matter the mod/plugin priority"); I only get the symbol if the BSA is unchecked and has a plugin that is checked in the Plugins Tab. Otherwise, the BSA is either checked or unchecked. Furthermore, right clicking the BSA provides an option to extract. Finally, dummy plugins used only to load a BSA are potentially redundant, but otherwise not. This is all very confusing, so we need a very clear explanation of the four scenarios and if any of the four scenarios following do or do not apply if extract is/is not used AND/OR if the plugin is a dummy or not:
  • Plugin checked, BSA checked - ?
  • Plugin checked, BSA unchecked -   ... ?
  • Plugin unchecked, BSA checked - ?
  • Plugin unchecked, BSA unchecked - ?
• Is checking/unchecking in the Archives Tab an alternative to BSA registration or is this effectively accomplished by registering the BSA in Skyrim.ini?
• Does MO scan BSA archives and include the asset conflict information in Mod-Right-Click > Information > Conflicts?

 

Could someone please answer each of these bullets explicitly?

 

I am growing convinced that nobody other than Tannin CAN, and he is likely too frustrated with the whole issue to do so I suspect ...

 

What I'm saying is:

- If the mod with the bsa          is higher in the installation order than the one with loose files then the bsa          takes precedence

- If the mod with the loose files is higher in the installation order than the one with bsa          then the loose files take precedence

 

Or, to phrase it more plainly (like I did before)

- If a mod has higher installation order then it takes precedence. The format just does not matter

 

I honestly don't understand how MO is complicated when it gets rid of a whole category of complexity...

 

@z929669: If esp and bsa are unchecked then the bsa is not loaded and can thus not conflict

 

WHAT do I document? That a certain problem people may have read on the internets does not actually affect them? I can hardly start documenting unusual things MO does not do.

Also, sorry to say this: documentation is worthless. People don't read it (well, there may be a few but certainly the majority of my users don't, this very discussion is proof of that).

 

@Garfink: I had such a button in the very first version that had the potential mod order warning but doubleyou convinced me to remove it. Don't know which thread that was in.

Documentation is VITALLY IMPORTANT. Why do you think I spent hours working on this OP and this issue? How else is anyone to know exactly how things work? See my posts quoted just previous. I STILL cannot seem to wrest the answers I need to properly communicate exactly what the bahavior is. I understand that BSAs have same priority as loose files if checked ... what about the other three cases?

 

I understand that soooo many people don't read through documentation, and it IS frustrating, but there simply MUST be a resource for the 10% of people that will actually reference it. then they can spread the information rather than the misinformation. It only helps!

 

As tannin has said 

 

The only defining factor is the install priority... It does not matter if one use BSA or loose. It is logical, and I fail to see where the issue is. 

 

I can unpack my bsa´s get the textures, meshes whatever and create my "custom test mod" just before overwrite and make sure that the assets I work on are loaded in the game, and I can remove them afterwards and make sure that the vanilla contents take over again. 

 

As it is now it is a convenience that MO handles the unpacking of bsa´s... I hope it stays that way, even if it is hidden and requires activation. 

 

 

I must admit that as long as only the install priority is what you should sort after I fail to see how any issues can ever arise unless people sort by name and have their list entirely weird. Even custom assets would be loaded to the correct file path. 

It is not at all clear HOW things function with regard to the questions I pose above. There are mixed messages, and if you and others read my post with a bit of empathy from the perspective of a person that was not involved with MO since its inception, you might understand my perspective. This is why there is controversy about the BSA functionality and subversion of the standard load ordering system. Only MO does this, and I think it is very powerful, but it is junk if we cannot (or do not) know how to explain the functionality in simple terms.

 

The *supposed* effective result is that the install order prioritization is the ONLY prioritization from the MO perspective, but I really don't believe that after reading the documentation. Plugin-loaded BSAs would seem to have prioritization in some cases (when the BSA is unchecked in Archives Tab??).

 

Could someone PLEASE PLEASE PLEASE! thoughtfully and explicitly address the questions I am asking? I will help to disseminate the CORRECT information once I know.

 

I challenge anyone other than Tannin himself to deliver a simple and complete explanation of MO-Skyrim asset prioritization and loading 'rules' (these are two different things BTW).

[Tannin]

Could someone please answer each of these bullets explicitly?

Sorry, hat a crappy day at work today.re 1: yes, if you check a bsa in the archives tab it is decoupled from its plugin. if it's not checked the bsa will work like it would with any other mod manager (or without). The MO conflict information will not be correct in this case.re 2: Why is this confusing? This is a list of archives -> check one to have it loaded in the expected/intuitive way.If the bsa is checked it will be loaded. the plugin does not matter then. it is loaded in the mod order (aka installation order aka asset order) -> no warningIf the bsa is unchecked and the plugin is unchecked then the bsa doesn't get loaded. It is to be assumed that is what the user wants (why would he uncheck everything?) -> no warningif the plugin is checked and the bsa is not then the bsa is loaded although it's not checked and its load order is different from the installation order -> THIS is the confusing case. This is the case that is counter-intuitive and thus the case that needs to be explained -> hence the mark. And this is exactly what the text says: "Contrary to what you might expect when you see an unchecked option this bsa is still active and may behave in an unexpected way"re 3: Yes, on the technical level checked files are loaded through the ini file but they don't behave like "registered bsa" as described by Lojack so don't call them that! They can override loose files.re 4: yes, except for the vanilla bsas (or better: all bsas in the data directory) 

Documentation is VITALLY IMPORTANT. Why do you think I spent hours working on this OP and this issue? How else is anyone to know exactly how things work? See my posts quoted just previous.

The problem is: all of this, every single thing I posted here I have explained before.The following is documented directly inside MO:

BSA files are archives (comparable to .zip files) that contain data assets (meshes, textures, ...) to be used by the game. As such they "compete" with loose files in your data directory over which is loaded.By default, BSAs that share their base name with an enabled ESP (i.e. plugin.esp and plugin.bsa) are automatically loaded and will have precedence over all loose files, the installation order you set up to the left is then ignored!BSAs checked here are loaded in such a way that your installation order is obeyed properly.

and 

This archive will still be loaded since there is a plugin of the same name but its files will not follow installation order!

and 

Another special type of files are BSAs. These are bundles of game resources.These archives can be a real headache because the way bsas interact with non-bundled resources is complicated. The game can even crash if required archives are not loaded or ordered incorrectly. MO applies some "magic" to make all BSAs that are checked in this list load in the correct order interleaved with the non-bundled resources. Usually it's best to check all bsas that have an exclamation mark at the side.

And in the wiki we also have it explained: 

If a mod contains assets in a BSA file, it will appear in the Archives tab under the mod's name. Checking a BSA will ensure that its contents make it in-game in the order that you have set your priorities in the left pane. If an unchecked BSA has a corresponding esp or esm in the load order, it will display a warning icon beside it. This indicates that the BSA will be loaded despite not being checked. Not only will such a BSA be loaded, but also it will ignore the priorities set in the left pane and load according to its position in your load order. BSAs may also be unpacked in this tab by right-clicking the BSA and selecting Extract... This will extract the BSA's contents to any folder you choose.An accompanying ESP file to extract the BSA is not necessary in Mod Organizer. The only requirement is for the BSA to be checked here.

Shall we make a poll how many people have actually read all that? How many more ways should I explain the same thing before it's documented enough? 

It is not at all clear HOW things function with regard to the questions I pose above. There are mixed messages, and if you and others read my post with a bit of empathy from the perspective of a person that was not involved with MO since its inception, you might understand my perspective. This is why there is controversy about the BSA functionality and subversion of the standard load ordering system.

BSA extraction is already a subversion of the standard load ordering. The controversy is because (some) people use advanced features they don't need or understand and don't read warnings and don't read documentation and then bother the mod authors. The 10% of users that read documentation aren't the problem, they already have the information they need available. The problem are the users that don't and never will read documentation. These people can only be helped by taking control away from them.MO gives you a mod order suggestion. If you just follow it you don't have to know ANYTHING about BSAs and loose file. You just click on the warning icons, do what it says and the whole thing is done. finished. No need to read, no need to think. 

The *supposed* effective result is that the install order prioritization is the ONLY prioritization from the MO perspective, but I really don't believe that after reading the documentation. Plugin-loaded BSAs would seem to have prioritization in some cases (when the BSA is unchecked in Archives Tab??).

Which is why you get a warning that tells you exactly that. There is the documentation. It's one sentence. You even quoted it but you still didn't read it...

[fireundubh]

if the plugin is checked and the bsa is not then the bsa is loaded although it's not checked and its load order is different from the installation order -> THIS is the confusing case. This is the case that is counter-intuitive and thus the case that needs to be explained -> hence the mark. And this is exactly what the text says: "Contrary to what you might expect when you see an unchecked option this bsa is still active and may behave in an unexpected way"

Here are my suggestions:

[*]Remove the Archives tab. Because BSAs are loaded regardless of whether they're checked, the Archives tab presents unnecessary information to the user and only serves as a point of failure.

[*]Tie plugins and their BSAs together. When a plugin is activated, all relevant files in the plugin's directory should be activated.

[*]However, if loose files and a BSA share the same plugin's directory, loose files should override the respective files, if any, in the BSA both at load time and in the VFS.

Edited May 22, 2014 by fireundubh

[z929669]

Sorry, hat a crappy day at work today.

I understand, believe me! No offense taken. 

re 1: yes, if you check a bsa in the archives tab it is decoupled from its plugin. if it's not checked the bsa will work like it would with any other mod manager (or without). The MO conflict information will not be correct in this case.

Thanks for this confirmation. Unchecked BSA in Archives Tab = standard, checked = MO rules.One point though is that all of the Skyrim - * BSAs are greyed out (untoggleable) and checked (these are registered in the Skyrim.ini), but official DLC BSAs are not grayed out and behave like any other add-on BSA (these have plugin loaders and are NOT registered in Skyrim.ini, nor should they be). A brief word about the greyed out registered Skyrim - * BSAs would be helpful. 

re 2: Why is this confusing? This is a list of archives -> check one to have it loaded in the expected/intuitive way.

The "expected/intuitive way" depends on the perspective of the user. That may be why I have been getting confused and seeing mixed messages. I expect the standareds of Skyrim asset prioritization. Whether that is intuitive or not depends on me. I get that you have tweaked the asset prioritization and loading mechanisms to work in a more sensible fashion that is made possible by the VFS, so messaging around this is very important (to people like me anyway). The Skyrim defaults of asset and load prioritization are only well understood by relatively few (of mod *users* anyway), so when MO's unique behavior enters into the mix it can challenge an already tenuous understanding --even if it is inherently more intuitive. We should probably explain the "MO standard behavior" as opposed to the "standard behavior".

 

If the bsa is checked it will be loaded. the plugin does not matter then. it is loaded in the mod order (aka installation order aka asset order) -> no warning

So if the plugin is only a loader, it is 100% redundant and should be tossed, but if it confers function, then there is another level of control allowed now: the plugin can invoke assets provided from any mod source, not just its own expected BSA assets (noting this as an interesting fact).

 

If the bsa is unchecked and the plugin is unchecked then the bsa doesn't get loaded. It is to be assumed that is what the user wants (why would he uncheck everything?) -> no warning

This is effectively like not installing the mod; however, the BSA could still be registered in Skyrim.ini, but as I understand it, MO does not effectively register BSAs via INI tweaks, right ... ?

 

if the plugin is checked and the bsa is not then the bsa is loaded although it's not checked and its load order is different from the installation order -> THIS is the confusing case. This is the case that is counter-intuitive and thus the case that needs to be explained -> hence the mark. And this is exactly what the text says: "Contrary to what you might expect when you see an unchecked option this bsa is still active and may behave in an unexpected way"

Agree that this is a confusing case, but the actual wording for the error is:"Marked archives () are still loaded on Skyrim but the regular file override mechanism will apply: Loose files override BSAs, no matter the mod/plugin priority"... which is confusing (the link to Lojack's post is not very helpful, as that addresses several additional features. We could write up a clear summary of standards on the siki that you can link to.If I understand correctly from your explanation above, the standard load mechanism and prioritization are in effect when the plugin is checked but the BSA is not: The BSA is loaded with the plugin, and the BSA assets override other plugin-loaded BSAs loaded previously (as well as registered BSAs), but loose assets still override the assets within this BSA, regardless of the install prioritization of the loose asets (this is the standard behavior outside of MO). Is that correct (if it is, I will propose a better info text to avert confusion). 

re 3: Yes, on the technical level checked files are loaded through the ini file but they don't behave like "registered bsa" as described by Lojack so don't call them that! They can override loose files.

This is another big point of confusion for me. Are you saying that MO adds an INI tweak to Skyrim.ini to register the BSA? I assume NO. I assume that MO loads this BSA according to the install prioritization as described above and that BSA registration is not even involved ... ? 

re 4: yes, except for the vanilla bsas (or better: all bsas in the data directory)

OK, so NONE of the Skyrim BSAs' assets (including the official DLC) are exposed within the conflict resolution of the mod Information dialog box. 

The problem is: all of this, every single thing I posted here I have explained before.

I assume that you have ... likely many times, but the information is not as clear or "up front" as it needs to be (given, the controversy of some of MO's inoovative functionality). This thread seeks to help with that  

The following is documented directly inside MO:andandAnd in the wiki we also have it explained:Shall we make a poll how many people have actually read all that? How many more ways should I explain the same thing before it's documented enough?

EX. The following is taken from the documentation:"BSAs may also be unpacked in this tab by right-clicking the BSA and selecting Extract... This will extract the BSA's contents to any folder you choose.An accompanying ESP file to extract the BSA is not necessary in Mod Organizer. The only requirement is for the BSA to be checked here. "

 

This is not very clear, since the ESP does not extract the BSA ... or is the BSA effectively extracted when the BSA is checked? .... I hope you see what I mean: there are ways we can be stating things that are more explicit and clear. The above leaves questions and even invokes new ones and is not isolated among the doc. 

BSA extraction is already a subversion of the standard load ordering. The controversy is because (some) people use advanced features they don't need or understand and don't read warnings and don't read documentation and then bother the mod authors. The 10% of users that read documentation aren't the problem, they already have the information they need available. The problem are the users that don't and never will read documentation. These people can only be helped by taking control away from them.MO gives you a mod order suggestion. If you just follow it you don't have to know ANYTHING about BSAs and loose file. You just click on the warning icons, do what it says and the whole thing is done. finished. No need to read, no need to think.Which is why you get a warning that tells you exactly that. There is the documentation. It's one sentence. You even quoted it but you still didn't read it...

But removing elements of control punnishes those that understand the functionality (or those that want to, are trying to, and are wise enough not to bother mod authors for their own lack of understanding). I don't think the solution lies in hobbling the software. A better solution is to make the most important information very relevant and right up front for all users to see (even accidentally). This still will not suffice though, because many people don't want to simply 'trust' that everything is happening the way it should be. Certainly, your testers (the MO and STEP staff among them) need to remain wary of what happens if things are not working correctly and to recognize any signs of problems. And we need to boldly emphasize certain talking points that address anything controvercial or not well understood.I am trying to condense that information here ;)If people want simplicity and opacity, Then they can use tools like NMM. MO and Wrye Bash both are more advanced tools that require more understanding to use effectively, and both expose things that arguably should not be exposed to standard users (I disagree though). MO has a more intuitive and less *scary* interface though, so it is attractive to novice users.I see MO taking over as the primary mod management utility as long as we keep things simple and clear as possible and address the concerns and controversy ... and debunk the false critiques. 

[DoubleYou]

Okay, this isn't really as hard as ya'll are making it. I thought the wiki made MO BSA understanding clear, from the Priorities tab:
 

BSA files are archives that contain assets including textures, meshes, sounds, and scripts. All vanilla data resides in BSAs and by default, the Skyrim Creation Kit packages assets for mods in BSAs. Traditionally you can not control priority between BSAs and loose files (that is: assets that came as individual files) as BSAs were always overridden. Therefore, if you wanted to use files from a BSA to override files from a mod that came as loose files, you would have to delete the conflicting loose files manually.

With MO, BSAs checked in the BSA tab are treated exactly as if they were loose files. That means their priority depends solely on the mod priority order and not on the plugin load order. If a BSA is not checked in the BSA tab but its corresponding plugin is active, the BSA shall be loaded, but in default plugin load order conflict resolution. Any checked BSAs will override any unchecked BSAs. Checking a BSA in the BSA tab turns the conflict resolution for that BSA to act like loose files.

 

Just to make everything perfectly clear:

 

• If BSA is checked in the Archives tab, forget that the files are inside a BSA. Totally. It is as if it were extracted. The only thing you can't do is hide any files within the BSA. Typical Mod Organizer black magic. Don't worry about the how. Only Tannin and a few of us enlightened ones know.
• If BSA is NOT checked in Archives tab, it will need its plugin to be loaded, unless you register it. And I wouldn't register it if I were you. You might as well check it in the Archives tab and load it at the top of your mod list priorities. Easier, safer, faster.
• If an asset is in a BSA and that BSA is checked in the Archives tab and another mod with a lower priority contains a loose version of the same asset, the asset from the BSA in higher priority will indeed be used by Mod Organizer in place of the loose asset.
• If an asset is in a BSA and that BSA is not checked in the Archives tab and another mod with a lower priority contains a loose version of the same asset, the asset from the mod with the loose version in lower priority will indeed be used by Mod Organizer in place of the BSA asset.

[Razorsedge877]

So in other words DoubleYou the only mods we need to extract are texture mods that are higher in priority that we want to hide files in so that lower priority files can take priority?

[DoubleYou]

So in other words DoubleYou the only mods we need to extract are texture mods that are higher in priority that we want to hide files in so that lower priority files can take priority?

Currently I believe there is no STEP mod that requires such treatment, but that could be a potential case, but it wouldn't necessarily need to be a texture. It could be a mesh or other asset as well.

[EssArrBee]

The only time we extract a BSA is if we need a lower priority BSA. Then we could hide the extracted file(s) as needed. Also, texture optimization, but we can get around that by uploading pre-opted BSAs.

[Kelmych]

It might be relevant to some of the mods in the

" A Real Explorer's Guide to Skyrim" pack. Some of these mods have BSAs with resources that overwrite vanilla assets and some STEP mods that have low priority values, and I doubt we always want the versions of these assets in the BSAs.