Ramifications of BSA Extraction in Mod Organizer

[z929669]

This will graduate to wiki guide format once we get sufficient community input ...

---

 
I am creating this thread to address some important information about BSA extraction and related modding concepts that come up over an over again with respect to STEP's use of Mod Organizer (MO) and MO/STEP critics that disagree with STEP's general advocacy of BSA extraction and MO's BSA-extraction functionality. In order for this to be meaningful to all users, I am providing some background information required for understanding the ramifications of BSA extraction and why STEP advocates it while other respected modders do not.
 
WARNING: Extracting BSAs deviates from the intent and expectation of the respective mod provider, so it is fair to expect that once a user extracts a BSA supplied with a given mod, that mod's author has every right to refuse support on the basis that the user implementation is no longer the same mod. Further, if a user extracts vanilla Skyrim BSAs, ALL mod authors and the modding community at large have the right to refuse ANY support to said user!
 
NOTE: The STEP community follows Wrye's cathedral model of modding, and since we advocate experimentation and an open modding approach, we also try to encourage 'creativity' (but not flagrant stupidity, TYVM) and users are welcome to support each other in this community, and we'll do our best to help as we are able, particularly with regard to what we advise in any of our guides. After all, modding is a largely creative endeavor and a learning experience!

---

 
Executive Summary
 
STEP provides instruction for a range of methods to mod Skyrim. Novice users can follow the STEP Guide, which is pretty straight forward and adheres to conventional modding methods. More advanced users or users that want to take it further can follow the additional guides and employ some more advanced and less conventional techniques (like BSA extraction and texture optimization) that we have found to be viable and optimal for the 'perfectionistic' or 'adventurous' modder. Some of these techniques are traditionally the jobs of the mod providers rather than the mod users, but we encourage and empower our community to take on these tasks if they wish, because many mod providers do not optimize their mods, and we advocate customization of mods by the user.
 
STEP advocates BSA extraction because it allows for more granular control of the modded Skyrim (but see 'cons' below). Critics object that this practice 'breaks' the fundamental standards of install-order and load-order methodology that many mods and modding utilities are built around. Nevertheless, I and many STEP staff and members have not found this to be the case and propose that these concerns likely stem as a result of Mod Organizer users having problems and complaining and blaming the mod authors on mod threads like the USkP when something goes wrong with their game. Why MO users? Because MO is the only mod manager that exposes BSA extraction to any user that installs a mod with a BSA in it (basically, all MO users).

• STEP advocates BSA extraction (given the user understands the ramifications!)
  • It is a prerequisite to texture optimization (NOTE: repackaging vanilla BSAs is possible with CK's Archive.exe but problematic with BSAopt)
  • Used properly, it does not cause any load-order issues at all
  • It allows more granular control of a modded setup
  • It can theoretically lead to excess disk fragmentation (HDDs only, not SSDs)
  • It can theoretically reduce in-game performance if most/all game assets are loose files under a heavily-modded game
• BSA extraction is not 'bad' (NOTE: it can alter the intended behavior of a mod's interaction with other mods if used improperly)
• BSAs are not 'bad'
  • simplify mod distribution
  • simplify user maintenance (NOTE: particularly for manual installation and traditional mod management ... but NOT for Mod Organizer users!)
  • simplify support by mod providers
  • BSAs are the future trend in Bethesda modding, so best to get used to them

---

 
Install Order vs Load Order
 
A word about game assets: Game assets are any files (resources) that get used by Skyrim. Plugins (*.esm & *.esp) are special assets that are used by Skyrim to call upon other assets and to provide instructions for their use. Thus, plugins are the 'brain' of the game. The only plugins needed to play Skyrim are Skyrim.esm &  Update.esm. Other plugins are from the DLC add-ons or other mods made by the modding community like the USkPs, etc.
 
Install order: The order of mod files installed onto disk. If two mod packages contain the same file names along the same file paths (e.g., textures/blah.dds), then the last installed version overwrites (i.e., overrides) any previous version and is thus the version that will be used by the game if it is called upon by the game (via a plugin). These mod files can be anything at all (e.g., text, images, ... whatever), but only certain file types are used by the game: textures, meshes, scripts, plugins, etc. Therefore, install order affects what resources the game will use.
 
(NOTE: MO does not install mods to the data directory, but rather mods are extracted into mod folders within a user-specified location. MO creates a virtual /Data/ that appears to Skyrim as the actual /Data/, and it populates this virtual directory with mod assets from the install directory as specified by the MO installation priority. Otherwise, there is effectively no difference between MO and other mod managers, but this difference is fundamental and confers a significant advantage to MO users).
 
Load Order: The order that plugins are loaded into the game. Like install order, the last plugin loaded overrides all previous plugins. Since plugins reference assets within /Data/ by file name, there is potential for two different plugins to reference the same named resource.  Additionally, since plugins provide instructions as to the use of these resources, load order can also affect game behavior. Therefore, load order affects both what the game will use and how the game will use it.
 
What are BSAs?
 
Mandatory reading: read this important background information!
 
BSA: A proprietary archive of game assets that mirrors /Data/ directory structure. Thus, a BSA file is an archive exactly like a folder that is simply packaged as a file. The same is true of any ZIP or 7z archive.
 
How do BSAs Work?
 
For Skyrim to be 'aware' of a BSA, it must either be registered in Skyrim.ini or loaded with a plugin of same name. Once recognized, the game sees any BSA as part of /Data/ itself; however, when conflicts exist between files contained within a registered BSA, a plugin-loaded BSA or within /Data/ as loose files, things are a little trickier:

• Registered BSAs: These load at Skyrim start in the order that they are listed in Skyrim.ini, last loaded BSA 'wins' in event of resource conflicts of contents within.
• Plugin-loaded BSAs: These load when a new or saved game is loaded after Skyrim starts. Each BSA is loaded at the time the plugin of same name is loaded. So any BSA with content resource conflicts corresponding to a plugin will 'win' if its plugin is loaded after the conflicting plugin. Basically, these BSAs (and all of their asset content) are referenced by their plugin and loaded according to plugin load order. Plugin-loaded BSAs always 'win' where they conflict with Registered BSAs. The only exception is with respect to resources required at Skyrim start but before savegame (or new game) load, like No Menu and Loading Smoke.
• Note about loose files: Loose files always override same files inside of registered AND plugin-loaded BSAs!

 
Summarizing in terms of prioritization and load order ...
 
Skyrim asset priority:

1. Loose assets always win
2. Plugin-loaded BSAs win all but #1 (EXCEPTION: plugins are only loaded when a new or saved game is started, so plugin-loaded BSAs have zero priority with regard to pre-game assets)
3. Registered-BSA assets lose to all #1 & #2
4. Registered Skyrim BSAs and other official content and DLCs behave no differently than "after market", mod BSAs

Plugin/BSA load order:

1. Registered BSAs load according to list order in Skyrim.ini
2. Plugin-loaded BSAa load with respect to the corresponding plugin load order
3. Plugins load according to %USERPROFILE%/Appdata/Local/Skyrim/plugins.txt, which is managed by BOSS/LOOT

BSA Pros:

• Keep the Data directory clean and uncluttered (NOTE: this does not apply to MO users though, since MO uses the virtual file system).
• Allow easy mod management, since all of a mod's files are much simpler to identify and update or remove (mitigates user error= less support burden)
• Make it easier for mod authors to distribute and maintain control over how the mod functions (mitigates user error = less support burden)
• UPDATE:
• Better performance (NOTE: a lot of loose files slows down game startup, especially when using MO)
• Less disk usage (NOTE: BSAs can be compressed; HDD fragmentation is less of an issue)

BSA Cons:

• Removes an element of user-level control ... and many mod users are control freaks (STEP especially)
• Users can no longer efficiently see contents of a mod (NOTE: although Wrye Bash does expose this information, albeit with a performance hit ... is this functionality inherent or is it off by default??)
• Incentivizes mod authors to provide BSA 'hotfixes' as loose files (NOTE: This has undesireable ramifications for MO users due to behavior of BSA extraction in MO ... BSA extracts last, so loose file hotfix is overridden by original version within the BSA! EDIT: this is fixed in the current beta and next release of MO)
• Mod authors are forced to upload all files (the entire BSA) for any updates (all files are contained within a BSA), and users are forced to either download again or deal with the issue just previous if the mod author has supplied a 'hotfix'-type update.

BSAs & Steam Workshop
Steam Workshop only allows mods that use the BSA + ESP format. STEP finds this overly restrictive and unnecessarily 'controlling'. I personally resent it and only deal with Steam because it is the wrapper for Skyrim (unfortunately, IMO). The Steam Workshop and Steam-Skyrim community are valid entities that do not deserve to be totally ignored, but STEP does not recommend that it be used as a primary source for mods or modding information. The Nexus is the STEP-preferred source for all modding needs. For information, STEP is a good primary source, and we point to the best alternative sources, but here are a few others:

• TESAlliance
• Bethsoft Forums
• Unofficial Elder Scrolls Wiki (UESP)
• SkyrimSearch.com
• Tamriel Maps 

BSAs & Mod Organizer
 
Since Mod Organizer allows users to extract BSAs during mod installation, MO potentially obviates any functionality of registered or plugin-loaded BSAs. Thus, any mod that uses a BSA is effectively constrained henceforth by rules pertaining to loose files, so its assets are no longer linked to hierarchies of BSA registration order or plugin load order. This and the fact that all or user-specified mod resources can be loose and manageable by MO confers a clear advantage to the user.
 
BSA Extraction Pros:

• MO users have a much more granular level of asset control and can prioritize BSA contents at the loose-files level
• It is a prerequisite to texture optimization (NOTE: repackaging vanilla BSAs is possible with CK's Archive.exe but problematic with BSAopt)

Other issues can arise though, so only informed users that understand the ramifications should be using this functionality (unpacking BSAs). Following are some things to be aware of when unpacking BSAs (that mod authors intended to remain packed as delivered!).
 
UPDATE: There does not seem to be any need for standard users to extract mod BSAs in MO, because once can subvert the constraints of the standard load order/asset prioritization system from the Archives Tab:

• Plugin checked, BSA checked - Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
• Plugin checked, BSA unchecked - Follows plugin load order for all unchecked BSAs. All loose file assets will "overwrite." OTHER checked BSAs will NOT overwrite, which is why unchecking BSAs can lead to unpredictable results or dificult-to-resolve conflicts (hence the warning).
• Plugin unchecked, BSA checked - Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
• Plugin unchecked, BSA unchecked - As if the plugin and BSA don't even exist in the mod setup.
• Furthermore, MO will scan all mod BSAs (aside from those in /Data/) and include these assets in the Mod > Information > Conflicts tab. So in MO, BSAs effectively behave like loose files when checked in the Archives Tab! Mod developers will still find the BSA extraction functionality handy for testing purposes during production of updates to their existing BSA-packed mods or when developing new ones dependent on assets contained within BSAs. 

BSA Extraction Cons:

• BSA assets are now given loose files priority, so this alters the mod author's original design intent and may introduce false 'bugs' that nobody on any forums will likely want to or know how to diagnose or fix ;)
• BSA extraction in MO happens after loose files are installed. This means that any loose 'hotfixes' would be overwritten by the BSA version, which is outdated. EDIT: this is fixed in the current beta and next release of MO

MO exposes BSA extraction functionality using a prompt when a mod containing a BSA is first encountered. This functionality can henceforth be "always allowed" or selective, based on user preference in response to this prompt. Users that do not fully understand the ramifications of BSA extraction on the specific mods they are using together should not use this feature. If "automatic BSA extraction" is in effect, it can be reset from Settings (click the wrench icon in the toolbar) > Reset Dialogs > click 'yes' at the prompt.
 
First, STEP recommends that users NEVER "always allow" automatic BSA extraction ... why? Because there is no need to do this at all, since the granular functionality already resides within the Archive Tab. More importantly, because many unknown or unintended prioritization issues can come into play as described previously. It is always safer to use the BSA unless it will cause a de facto undesirable result.
 
BSAs that have optimized textures and can be overridden completely by downstream mods should stay inside BSAs (or repackaged using CK's Archive.exe). If assets from inside a BSA need to overwrite some mods and be overwritten by others, then sometimes it makes sense to extract the BSA. MO has a beautiful tool accessible from within its Archive Tab. If a BSA is present in the load order, it will appear in the Archives Tab. Leave it unchecked to allow it to behave normally and be loaded by its plugin (if the plugin is active), and check it to extract the BSA to the mod folder and effectively confer loose-files prioritization.
 
Critics of Mod Organizer and STEP (for Officially Advocating BSA Extraction) 

Some within the well-respected modding community are at odds with the idea of BSA extraction advocated by STEP and facilitated by MO. The most notable contingent is the USkP team. This is relatively old news and nothing that should be shocking, so please do not treat it that way. The reasons are not unfounded and actually valid. I bring this up solely to address the idea to remove BSA-extraction from MO that Tannin suggested if MO-detected load order issues are not resolved properly (by submitting a ticket). In fact, I created this thread to address this one issue as much as to address the knowledge gap that is the real cause of any issues associated with BSA extraction.
 
Some modders are more or less happy with MO's ability to invoke BSA extraction. Generally, mod authors who have gotten a lot of grief with respect to their mods --for problems caused by the ramifications of rampant BSA extraction-- seem to have more of a problem with MO (see note below!). This has been somewhat problematic for STEP and MO with respect to outsiders privy to the argument but not privy to STEP or MO ... never mind that the 'fault' should be shouldered solely by the unwitting mod user for invoking BSA extraction without understanding the ramifications of doing so ... and ours for not properly educating our user base to that effect (hence this thread).
 
Let it be said that the STEP modding community and the vast majority of modders are the kinds of users that use PCs instead of Macs and tend to be somewhat removed from computing 'norms' imposed by Apple and Microsoft and their ilk. In general, we do not like our control restricted in favor of provider control over our resources to make their lives simpler. We are generally in favor of digital freedom, open source software, and the honor system. Big Brother and his methods are generally unwelcome. Demonizing BSA extraction in general and removing it from MO in particular in order to enhance level-of-conrtol by mod providers would be a big mistake, as STEP itself somewhat relies on this feature (and will to a larger extent in the future). However, I think that it is very important that our users understand the ramifications of using BSA extraction, and we need to address explicitly in the STEP Guide.
 
The MO user base (not its 'critic' base) should guide Tannin's direction of MO, IMHO ... 
 
I want to explore what we do (and do not) know with regard to the ramifications of BSA extraction to our MO users and how best to make MO a broadly accepted utility among all within the modding community ... not just STEP. In order to do so, we must not dredge up combative arguments. Constructive argument is good for all respective modding endeavors, so what has been said in the past is water under the bridge. So keep it lively and fact filled ... but keep it polite and considerate!
 
Please report bugs as Tannin requests using the link above. Also please post to this thread and help us to improve the breadth and accuracy of this OP!
 

[z929669]

WOW ... we are getting somewhere with this thread now!

 

 

The only time we extract a BSA is if we need a lower priority BSA. Then we could hide the extracted file(s) as needed. Also, texture optimization, but we can get around that by uploading pre-opted BSAs.

Don't you mean 'higher' order? This statement is confusing to me (I am easily confused I guess :P )

 

For the record: The 'best' mod/load order prioritization is that which results in the fewest number of conflicts needed to achieve the desired result with minimal asset-level manipulation (i.e., hiding/deleting files, extracting BSAs, repackaging archives, etc.).

 

We need to say something about scenarios when extracting the contents of a BSA is valid and beneficial:

• When assets need to be modified and tested (mod creators only)
• When assets need to be exposed to Windows file system for search and to expose file-level metadata (mod creators only)
• When a BSA has lots of assets that are wanted, but has others that are not wanted but exist in lower-prioritized mods that we want to be manifest, and re-prioritizing the lower-priority mods might add even more complexity to the conflict resolution. This enables the MO 'hide asset' functionality otherwise not possible on assets within the BSA.
• Any others??

 

 

 

@DoubleYou (and Tannin or whomever)

I am afraid that I am still not clear on how MO handles BSAs. If I take the explanation about BSA priorities in the MO guide, then

• It would seem that MO can prioritize BSA content (as a solid block of assets) according to the install prioritization rather than the plugin prioritization, thus, decoupling a BSA from its plugin. Yes, entirely, if the BSA is checked in the Archives tab. I would not say that it treats it as a solid block of assets, however, as a Mod Organizer proactively scans BSA/loose file conflicts and doesn't load loose files that would provide asset management contrary to mod priority order as perceived via the data tab.
• When a BSA in the Archives Tab is checked, it allegedly loads according to install prioritization, but if it is unchecked, it is loaded according to its plugin prioritization Yes ... this does not jive ??? at all with the caption on the bottom of the Archives Tab window ("Marked archives ( ) are still loaded on Skyrim but the regular file override mechanism will apply: Loose files override BSAs, no matter the mod/plugin priority") I think you are confusing MARKED-with-warning-icon BSAs with CHECKED BSAs; I only get the symbol if the BSA is unchecked and has a plugin that is checked in the Plugins Tab. Correct. Otherwise, the BSA is either checked or unchecked with a deactivated plugin. Furthermore, right clicking the BSA provides an option to extract. Very useful feature. Finally, dummy plugins used only to load a BSA are potentially redundant entirely unnecessary, but otherwise not if the BSA is unchecked in the Archives tab. This is all very confusing, so we need a very clear explanation of the four scenarios and if any of the four scenarios following do or do not apply if extract is/is not used AND/OR if the plugin is a dummy or not:
  • Plugin checked, BSA checked - ? Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
  • Plugin checked, BSA unchecked -   ... ? Follows plugin load order for all unchecked BSAs. All loose file assets will "overwrite." Checked BSAs will NOT overwrite if I'm not mistaken, which is why unchecking BSAs can be dangerous.
  • Plugin unchecked, BSA checked - ? Follows mod priority order for conflict resolution. Plugin does not affect the situation at all.
  • Plugin unchecked, BSA unchecked - ? As if the plugin and BSA never existed.
• Is checking/unchecking in the Archives Tab an alternative to BSA registration or is this effectively accomplished by registering the BSA in Skyrim.ini? Yes and no. This goes back to the the Nitpick fix I think, and is a hack that acts like injection into the ini but is not actually doing so, if my memory is serving me correctly. BSA registration is definitely not an alternative to checking it in the Archives tab.
• Does MO scan BSA archives and include the asset conflict information in Mod-Right-Click > Information > Conflicts? Yes, except archives located in the actual data directory.

 

 

This is wonderful ... thank you THANK YOU! (isn't DY a nice person? no flack at all, just information all the time )

 

I will attempt to merge this and other info into the OP, but you should also look carefully at the OP and make edits where necessary.

 

... snip / This is mainly a habit from Wrye Bash days where if you didn't extract them, you wouldn't be able to see the conflicts which is needed for testing/troubleshooting purposes; ... snip/

FYI, I recently learned that Wrye Bash indeed does allow conflict resolution to include BSA contents (I believe it is a toggleble feature though and that it is not very efficient). Ask the WB devs for more info.

 

I don't like archives in my mod folders because I want them readily searchable via a simple file name search and laid out in such a way that if I want to hide one file, or replace it even; I can do so.

 

I have had several problems with USKP "fixes" that broke something in my game. If I hadn't known about BSA extraction, I'd never resolved the issue by myself. Curiously, Arthmoor wanted to blame MO's BSA handling for the problem which it was certainly not the case! And here was the fix:

 

 

Just to be clear: these files weren't overwriting anything, they just were causing a problem from their very existence in my mod order.

 

This issue has persisted through the last two USKP releases, each time I have had to hide those two files or an NPC in Half-Moon Mill gets a bad face lift.

 

Without the USKP Fix:

 

 

With the USKP "Fix"

I am a bit skeptical that this is an issue with the USkP though and not the result of the MO-flavored asset prioritization interaction with USkP and another mod(s). If you can provide detail as to exactly how to reproduce this, I might be convinced. Isn't it true that this could be cause by a higher priority body mod or EOO or any number of other things? I have heard about this specific issue and there has been no definitive explanation as to why this is or is not an issue with MO prioritization or the USkP or both.

 

The "argument" still has merit, compression adds another layer of work for the CPU to deal with, even if it is a relatively small work load by modern standards. The same principle is involved with the much discussed EnableCompression setting in the ENB config file; the consensus is that compression can cause stuttering/slowdown and performance penalties for some computer systems.

As Tannin and Aiyen allude, the consensus is that BSA seek time (with or without compression ... not all BSAs are compressed) is negligible and that loose-asset read time is more significant (on HDDs). Also, factor in that HDD fragmentation is a very real issue with a significant effect size.

 

@z929669

 

This is controlled in the settings under Wordarounds, Force-enable game files. Removing the check will set them be able to be toggled in the data tab.

Awesome info ... if this is in the MO guide, I admit that I missed it!

 

.... snip/ I can understand that but I develop MO for anyone who wants to do modding and needs to have more control. What we actually need is several documentations:
- If you're new to modding, here is what you have to know
- If you're an experienced modder who hasn't used a manager before, here is what you have to know
- If you're a converting NMM user, here is what you have to know
- If you're a converting Wrye Bash user, here is what you have to know
 
Sure, but my expectation is that someone who is into modding and spends a considerable amount of his time on the topic is going to read the documentation available whereas someone who only cares about higher-res textures and more bad-ass dragons is not going to read a lot because his prime interest is playing the game not tinkering with it.
.... snip/

This is a fantastic idea, and I think that we need to do exactly this:

• the novice modder - Core Guide (relatively new to modding and only wants to run a modded setup as quickly and efficiently as possible. Playing the game is most important.)
• the mod creator - Mod Author Guide (probably does not know much about mod managers or their differences but knows about Bethesda prioritization standards and such. Creating and testing mods and possibly mod interactions is most important.)
• Coming from NMM - NMM User Guide (this is likely a novice modder that has been modding for a while now [why else would they be using NMM?]. Playing the game is most important.)
• Coming from Wrye Bash - WB User Guide (this is likely an old-school modder or advanced and technically-minded modder. Tinkering is most important!)
• The advance modder - Advanced MO Guide (very likely an influential, old-school modder that is deeply involved in the modding community. Modding strategy and solutions are most important. Standardization and order are high priorities.)

In all cases, exhaustive documentation is key! We need more and it needs to be more searchable. I think we need to get rid of Headertabs for our larger guides. TOC is very helpful for drilling down on relevant info as is careful article Heading organization.

 

@Tech

Thank you. You hit the nail on the head with regard to "need to know". Some of us just NEED to know WHY and HOW ... damn the CS degree! We just never had the time ... we want to learn and know EVERYTHING in the universe. We LOVE the Matrix movies and the concept of inserting an information-interface-jack directly into our brains and learning all there is to know about a thing at at the speed of light! Knowledge for the sake of knowledge. Obsessively. Ruthlessly.

[Kuldebar]

z929669, on 23 May 2014 - 09:18 AM, said:

 

I am a bit skeptical that this is an issue with the USkP though and not the result of the MO-flavored asset prioritization interaction with USkP and another mod(s). If you can provide detail as to exactly how to reproduce this, I might be convinced. Isn't it true that this could be cause by a higher priority body mod or EOO or any number of other things? I have heard about this specific issue and there has been no definitive explanation as to why this is or is not an issue with MO prioritization or the USkP or both.

 

As Tannin and Aiyen allude, the consensus is that BSA seek time (with or without compression ... not all BSAs are compressed) is negligible and that loose-asset read time is more significant (on HDDs). Also, factor in that HDD fragmentation is a very real issue with a significant effect size.

 

 

It was 100% reproducible on my system despite multiple deletions and reinstallations and clean game loads USKP versus non-USKP. Sure, it could be: MO-flavored asset prioritization interaction with USkP But that's not a meaningful statement to me because it appears rather straightforward and basic file logic. While the mesh and texture was present in my active load order, I had the glitch; without those two files present, the glitch was gone.

 

If MO caused the problem just by being MO, I'd like to know how; I do know MO allowed me to fix the issue with very little help from anybody else. (Arthmoor at least relayed the file names that impacted the NPC in question) Seriously if there's some hidden behavior in how MO works specifically related to the well documented problem I had, it would be great to know.

 

My whole point is that the two files weren't a recognizable conflict, nothing was being flagged as being overridden.

 

No one should be making an argument solely based on "performance" reasons for not finding BSA's desirable, it's not the crux. The heart of the issue is file management by the user. Archives are NOT conducive to having a full tactical awareness when you need it the most.

 

What's convenient and expedient for Mod Authors and MO Devs is not necessarily good for the users; one needs only to look at closed, proprietary systems to see this point illustrated.

[EssArrBee]

It isn't USKP either it is the CK that randomly changes tint layers when they compile the plugin.

[Kuldebar]

EssArrBee, on 23 May 2014 - 11:59 AM, said:EssArrBee, on 23 May 2014 - 11:59 AM, said:EssArrBee, on 23 May 2014 - 11:59 AM, said:EssArrBee, on 23 May 2014 - 11:59 AM, said:

It isn't USKP either it is the CK that randomly changes tint layers when they compile the plugin.

Arthmoor 

QuoteQuoteQuote

The data changed in the tint layers is irrelevant. That's the CK being the CK, and no amount of us removing those will prevent it from returning again the very next time it's saved. It goes with the territory so we don't bother trying to remove it.

 

QuoteQuoteThe USKP has a facegen file for her, which is intentional to fix bugs on the vanilla side. Dawnguard.bsa contains another, which will be loading after the USKP when your files are in the correct order. The UDGP *DOES NOT* contain a copy of her facegen data. We took that out after the reorganization changed the load ordering on the patches.You are going to need to go through your Data folder very carefully (your real one btw, not the voodoo one MO creates) and make sure you don't have a lurking copy of meshesactorscharacterFaceGenDataFaceGeomSkyrim.esm001367C.NIF or texturesactorscharacterFaceGenDataFaceTintSkyrim.esm001367C.dds sitting in there.Barring that, if this ends up being some kind of crazy thing MO did, I'm not gonna be a happy lizard.

 

 

 

 

 

Yes, BOTH UDGP and USkP (via the CK proclivities, sure but still) introduce changes of Tints (same values) ...removing them from their respective ESP didn't fix the problem either...it appears there is FaceGen data in the folders for the patches as well but I didn't know which were relevant until Arthmoor named them for me.

 

When I disabled the Unofficial ESP's for the patches the problem persist because the actual patch files folders were still checked in the left MO pane.

Edited May 23, 2014 by Kuldebar

[z929669]

I will comment about the facegen issue on that thread you linked. (let's keep that there for now, unless it proves to be related to this thread)

 

RE BSAs being good or bad:

I have not like them at all in the past, because I modded Oblivion and Skyrim without MO once. I want full control and ability to see conflicts, regardless of BSA. (Windows search functionality never really mattered much to me though). Consequently, I always hated BSAs and the entire practice. It only made me feel like mod authors were shoving their meds down my throat without letting me look at the active ingredients first.

 

With MO, I no longer really find any reason to dislike BSAs, they are actually looking pretty good now from a management and a performance standpoint, especially for HDD users. Even without MO, I see the advantages that apply to Skyrim where they never applied to Oblivion (since now the Bethesda BSA standard has been fixed). Regardless, BSA extraction and other higher functionality should remain in MO, IMO, and I do not subscribe to the idea of black-boxing information and access to useful utilities based on the reasoning that "I don't need to know".

 

Tannin's software, Tannin's call ... users will always have choice regardless. If BSA extraction were to be removed, then anyone could use MO extensibility to rebuild the lost functionality as an MO add-on. Same goes for any feature that he does not want to be responsible for maintaining (as long as he maintains his license as it currently stands ... I think).

 

The main point is that we understand how MO functions EXACTLY. Otherwise, we can't develop STEP responsibly. I don't care (much) about the code, but I do care about all of the functionality that the code manifests and the implications/ramifications.

[Kelmych]

WOW ... we are getting somewhere with this thread now!

 

 

Don't you mean 'higher' order? This statement is confusing to me (I am easily confused I guess :P )

 

 

No, lower priority is correct in SRB's post. The best way to show this is with an example; hopefully this example isn't too difficult to understand.

 

The Conflicting Graphics portion of STEP has mods that add high quality textures, and we usually do not want another mod that installs later (higher priority value) overwriting some of the these textures that were chosen by STEP participants. Suppose we have a quest mod that uses a BSA which includes modified scripts that need to overwrite the vanilla ones, some modified navmeshes, and this BSA also includes some replacements for vanilla textures. Quest mods are typically installed fairly late (higher priority) because of the modified scripts and navmeshes. If the usual MO approach is used the BSA box associated with the esp plugin is checked. MO will provide the full BSA into Skyrim. When it provides individual (loose file) resources to Skyrim using the virtual file system, it will not include any conflicting resources from mods with lower priority. The scripts and navmeshes from the quest mod will install in their proper place (since none of the lower priority loose file conflicting scripts and navmeshes will be provided to Skyrim), but the loose file textures from the Conflicting Graphics mods that conflict with those from the quest mod will also not be provided to Skyrim.

 

For this example situation, the only way to get all the resources from the quest mod installed correctly is to extract the BSA from the higher priority (later in installation order) mod. The scripts and navmeshes (which are now loose files) will be provided by MO to Skyrim as they should be. The set of textures from the quest mod that we don't want to use need to be hidden, because if they are not hidden they will be provided to Skyrim instead of the ones from Conflicting Graphics.

 

Note that this is an unusual case; there are very few instances where this is needed. When I looked at all the mods I've installed the only ones that need to use BSA extraction with MO were quest/location mods, and even then it was only a few of these mods.

 

 

By the way, now that I understand the way MO provides resources (both loose files and BSAs) to Skyrim I don't think that using optimized textures provides any significant problems. The optimized vanilla textures can be stored as archives of loose files as we are already doing; I don't see any major advantage to storing them as BSAs. These archives are installed at very low priority (very early in installation order). Even though the USP mods (as an example) use BSAs for resources MO will provide the correct resources to Skyrim. If, for example, a UHRP BSA has an improved texture then the conflicting loose file texture from the optimized vanilla textures will not be provided by MO to Skyrim.

[EssArrBee]

By the way, now that I understand the way MO provides resources (both loose files and BSAs) to Skyrim I don't think that using optimized textures provides any significant problems. The optimized vanilla textures can be stored as archives of loose files as we are already doing; I don't see any major advantage to storing them as BSAs. These archives are installed at very low priority (very early in installation order). Even though the USP mods (as an example) use BSAs for resources MO will provide the correct resources to Skyrim. If, for example, a UHRP BSA has an improved texture then the conflicting loose file texture from the optimized vanilla textures will not be provided by MO to Skyrim.

And UHRP also has a dummy plugin which means that the BSA can load and still overwrite lower priority mods and we do not lose a plugin slot.

 

Also, Klemych brought up a good point about the DATA folder BSAs and the UPP BSAs priority. That needs to be sorted out or it could be an issue.

[z929669]

@Kelmych

Yeah, I get it, but was pointing out the need to be semantically explicit in our postings. SRB's post could be taken a couple of ways, and it was confusing how he worded it.

 

On another note and related to SRBs post just before this one, there is a significant issue with how MO and BSA extraction can subvert basic fixes provided by the USPs, which means that this can and likely IS happening elsewhere. This needs to be addressed and acknowledged by all that use MO. It is not really an inherent problem, but it is a reality that would not otherwise exist were it not for MO's ability to extract BSAs AND to treat DLC BSAs differently than others.

 

I have verified the issue, and Kludebar should chime in at this point to confirm so that we can amend the misinformation floating around.

 

EDIT: posts between Kludebar and me were moved to the thread linked above so as not to detract from the main agenda here.

[Kuldebar]

Hmm, all my BSA's for everything are unpacked though properly ordered...so:

 

Otherwise, you could also unpack the DG BSA as long as you place it after the USkP in the Mod priority...

 

I was already doing this when the problem occurred, to resolve it (other than hiding the two files) I would presumably leave the Unofficial Skyrim Patch in it's BSA...still seems screwy logic to me. The ESP refers to files, why the hell does it care if they are compressed in an archive or not IF they are in proper order?

 

Edited May 24, 2014 by Kuldebar

[namtrahj]

Hi guys. This is my first post, so I want to thank everyone at STEP for their work and dedication. I just dipped into the modding world for the first time in April and STEP has been invaluable. I've played Skyrim for about 35 hours but spent many more than that on modding.

 

I also want to thank Tannin and everyone else involved in MO. I don't know much of anything about programming but the VFS seems like genius to me!

 

I do have a question regarding BSAs. I have 300+ mods installed and have been extracting all BSAs since the beginning, which I thought was the best way to do things. Is there any reason for me to go back and redownload everything, leaving the BSAs packed up? Or, since MO treats BSAs and loose files the same, can I just leave BSAs intact going forward as I download new mods and update existing ones? From what Tannin has said it sounds like that would be ok, but since I'm new to all this I wanted to ask before I start down either path.

 

Edit: I'm still using MO 1.1.2. I haven't upgraded yet because I was terrified of breaking something. Tannin's message on the Nexus about waiting for an official release if you're not sure what you're doing made me nervous.

 

One other related topic that I haven't seen mentioned (although I could have missed it) is Mator's TES5Edit merge plugins script. I've just started working with that (pfft, 255) and he very, very strongly recommends unpacking any BSAs associated with the plugins to be merged. If a mix of loose files and BSAs isn't a problem for MO then that's fine, but if not...I wouldn't be sure how to approach the situation. In any case, merging plugins is at least another valid reason that users would need to extract BSAs. Just wanted to throw that out there since I hadn't seen it mentioned.

 

Thanks again for everything, guys!

Edited May 24, 2014 by namtrahj

[z929669]

Hmm, all my BSA's for everything are unpacked though properly ordered...so:

 

Otherwise, you could also unpack the DG BSA as long as you place it after the USkP in the Mod priority...

 

I was already doing this when the problem occurred, to resolve it (other than hiding the two files) I would presumably leave the Unofficial Skyrim Patch in it's BSA...still seems screwy logic to me. The ESP refers to files, why the hell does it care if they are compressed in an archive or not IF they are in proper order?

 

If you unpack both DG DLC BSA as well as USkP BSA and prioritize both using MO (DG highest priority), then this would fix the issue. USkP could be either Packed or unpacked, but DG BSA must me unpacked. That is the key point.

 

... and BSAs are not always compressed. (They can be like a TAR archive).

[Kuldebar]

@namtrahj

 

For me, I only reverted to the unadulterated BSA's for the unofficial patches, and of course, I always had the Bethesda files in their native forms, un-extracted. I'll be leaving the regular mods extracted since generally most of them gave that as an alternative anyway in their download section.

 

 

So, in my opinion, don't go crazy, if it ain't broke don't fix it...but regarding the USKP patch series in particular; it appears there can be certain drawbacks to BSA extraction that aren't quite obvious or easy to pin point.

 

I ran into one (possibly two) issues with extracted unofficial patch BSA's...there could easily be more.

Edited May 24, 2014 by Kuldebar

[EssArrBee]

I think that when the UPP team decided to false flag the patches it threw the MO load order a curveball. Maybe package the DLC as mods in MO and load them in the correct order.

 

Install order with Opted Textures and all BSAs checked in the archive tab:

• Skyrim Opted Textures
• USKP
• Dawnguard
• Dawguard Opted Textures
• UDGP
• Hearthfire
• Hearthfire Opted Textures
• UHFP
• Dragonborn
• Dragonborn Opted Textures
• UDBP
• HRDLC1 (these three could be the optimized versions repacked without plugins)
• HRDLC2
• HRDLC3
• UHRP (untick the plugin, tick the BSA)
• All other mods...

This solves the BSA load order problem of one BSA loading before or after another. We would have to do this as the very first section of the guide after 2.C. This would mean 2.D would have to be set up for this install order to get MO to do it properly.

 

EDIT: I'm a genius.

[Kuldebar]

EssArrBee, on 24 May 2014 - 11:52 AM, said:

I think that when the UPP team decided to false flag the patches it threw the MO load order a curveball. Maybe package the DLC as mods in MO and load them in the correct order.

 

Install order with Opted Textures and all BSAs checked in the archive tab:

• Skyrim Opted Textures
• USKP
• Dawnguard
• Dawguard Opted Textures
• UDGP
• Hearthfire
• Hearthfire Opted Textures
• UHFP
• Dragonborn
• Dragonborn Opted Textures
• UDBP
• All other mods...

This solves the BSA load order problem of one BSA loading before or after another. We would have to do this as the very first section of the guide after 2.C. This would mean 2.D would have to be set up for this install order to get MO to do it properly.

 

EDIT: I'm a genius.

I think you are right, but that sounds like a pain to do. I think I'll just surrender and stop extracting unofficial patches. I had originally tried to do that when I first got MO to have all the official BSA/DLC's extracted, but I found it somewhat confusing at the time and decided against it.

 

I do see the false flag development as a "wrench" thrown into the works because it ruined the logic that had I relied upon and replaced it with something I can't actually "see".

Edited May 24, 2014 by Kuldebar

[Kelmych]

That's essentially what I proposed in another thread; we're both geniuses  . The new DLC mods should use the cleaned DLC ESMs, which allows the Data directory to contain the original uncleaned versions of the ESMs. That change was suggested a long time ago before the issue with the false-flagged USP plugins came up. This is just a small change so should be fairly easy for most users.

 

EDITed to fix the comment about the location of the DLC ESMs