TheBloke

Change request regarding formatting/highlighting of new "pseudo-mods": Tracker link: https://issue.tannin.eu/tbg/modorganizer/issues/661 Description: I would like to request the following changes to the highlighting of the new "psuedo mods": [*]Red Blue or Green text for all fields, to extra-highlight these special mods [*]EDIT: DoubleYou pointed out in the Tracker comments that red is bad because it indicates 'problem'. I quite agree, so am suggesting blue or green instead now [*]Add a new special category for pseudo-mods [*]I would suggest that vanilla/DLC content is labelled with category "Vanilla/DLC" [*]There could then be a separate, "Outside MO" (or similarly named) category for content which is not vanilla/official/DLC, but is still a pseudo-mod. [*]Remove the conflict icon [*]I can't double-click on these mods and view the Conflicts tab. Therefore I find the presence of a conflict icon distracting and misleading. [*]Furthermore, removing this conflict icon also serves to better visually differentiate pseudo-mods from normal mods. Screenshot mock-up of my requested changes: (click image to view it properly!)

Yeah that's the first one I tried ;)

Ah ha! How strange! Thanks. OK so I have run a comparison of the Exe versus the 7z, and found a few more files that are missing. Specifically: [*]dllsboss.dll [*]NCCGamebryo Base.dll [*]pluginsdatapyCfgDialog.pyo [*]pluginsbsaExtractor.dll [*]pluginshelloWorld.dll [*]pluginstestnexus.py The last two definitely look unimportant, but the others may be important; the bsaExtract.dll certainly is. I've raised this for Tannin here: https://issue.tannin.eu/tbg/modorganizer/issues/659 In the meantime, for other beta testers - it appears that the Exe installer version is incomplete, and that the 7z should be used in preference for 1.2.4. What Refresh button are you referring to? I just tested it again. I installed a new mod with a 350mb BSA file. I confirmed that the Conflicts page just showed '2 files' (the ESP and the BSA), and this persisted for at least one minute. Then I went to the Data pane (right hand side), and clicked Refresh. No change in Conflicts. Then I clicked the Refresh at the top of the mod list (left hand pane). No change in conflicts. Then I tried clicking back and forth between all the tabs of the right pane (Plugins, Archives, etc). No change in Conflicts. A further 2-3 mins later, the time it took me to come here and type that paragraph, and Conflicts is still not updated; still shows two files. I've tried clicking both Refresh buttons again, no difference. After 8 minutes total, it still had not updated. At this point, I re-started MO. As soon as it re-opened, conflicts were showing for that mod. So am I missing a Refresh button somewhere? Otherwise, I can't seem to get it to trigger a refresh like you described? Either way, I'm raising this one as a bug - I think it should just automatically refresh the Conflicts as soon as it can, without manual intervention ever required. Bug report here: https://issue.tannin.eu/tbg/modorganizer/issues/660 It only takes a few seconds to extract a 350mb BSA file on my system, so it shouldn't take more than a few seconds to get the file list (if anything, less.) Based on what you said, it seems like it's maybe on a delayed refresh cycle or something? Perhaps that's for performance reasons, but I find it really annoying and confusing. If it's really needed for performance for some people, it should be an option. So I hope this can be changed so it automatically updates conflicts for BSAs (or within a few seconds at least), like it does for loose files. Update: It's more complex even than the above. Restarting MO causes a BSA mod to show conflicts. But so does installing a new mod. So if you install mod#1, and only that mod, then its conflicts will never update unless you restart MO. But as soon as you install mod#2, mod#1's conflicts will update - but then mod#2's won't be updated! It gets even stranger; if you restart MO to get mod#1's conflicts to show, then you untick & retick it in the mod list, it will again not show any conflicts. However once you've installed another mod, mod#2, you can untick & retick mod#1 (or any pre-existing mods) as many times as you like, and their conflicts will always show. So it seems like restarting MO causes some kind of refresh of conflicts, but it's not a permanently saved one - but installing a new mod seems to make all previous mods calculate, and store/cache their conflicts, so that they never disappear again. Weird in operation, but hopefully some obvious/easy fix for Tannin :)

Started testing with 1.2.4 today. The new pseudo-mod handling seems to work very well and just as expected. Not found any issues at all yet. Great job! One thing I'm not so keen on - it seems like the mod installation "Extract BSA" dialogue option has been removed completely. It's no longer asking me if I want to extract BSA files when I install a mod. I reset my dialogues to be sure it wasn't a stuck "Always" option. I'm guessing this is deliberate, because of all the kerfuffle with extracting BSAs? EDIT: Actually it's not been removed! Well hopefully :) See next posts from DoubleYou and myself; the Exe installer is missing a couple of files compared to the 7z archive, one of which is the BSA extraction plugin that handles this. Presumably that was unintended. If it wasn't, then Tannin can confirm - and I'll ask that if he really doesn't want BSA Extraction to be obvious, that he includes the plugin but has it disabled in the Plugins menu by default. But for the moment I'm assuming it was just a mistake. Another issue - which may be a bug - the Conflicts tab is really slow to update with BSAs. I actually nearly raised a bug just now, because I re-installed all the Unofficial Patches, such that they would change from Loose to BSAs. After doing that, every one of them showed 0 Conflicts, and the "Unconflicted Files" section showed - for example for USKP - '5 files'. Where it should be thousands. Eventually, after I'd clicked around in the interface a few times, taken a couple of screenshots, and come here to write the top half of this post, I then noticed that it was correctly showing Conflicts. It felt like it took several minutes until they were updated. So maybe that's a bug. Or maybe it's an inevitable result of the time it takes MO to process a 125MB BSA file to get a list of files to compare for Conflicts? I'm concerned it might be the latter, and it's therefore another reason why someone might (I emphasise might!) want to continue extracting BSAs. EDIT: See later posts from DY and myself; it's definitely not a performance issue, and does seem like a bug. I also have some other feature request changes I'm going to raise in this area - such as improving the Filetree/Conflicts so that, for extracted BSAs, it lists all files, including those that don't conflict - thus further mitigating one of the last benefits of extracting all BSAs. But the great news is that 1.2.4 seems very stable to me; certainly it has been a drop-in replacement for 1.2.1 for me today. Awesome work Tannin, thank you as always! PS. Love the new AND/OR option in the filter, thank you! I had been meaning to request that, and there it is anyway! Makes filtering properly useful now. Much appreciated!

I've just run a detailed test of this, using Mod Organizer version 1.2.1 beta (the latest beta available on the Nexus mod page): Using MO 1.2.1 beta, I cannot re-create the issue. I have verified that the HD DLC loads without the ESPs ticked, as expected. Details of testing: [*]I went to Honningbrew Meadery, near Whiterun. [*]I stood up close to one of its walls, facing west, and saved the game. [*]I loaded the game from this point and took a screenshot, with each of the following three configurations: [*]HD DLC BSAs not available - I moved them out of /Data, to confirm the appearance when the HD DLC was definitely not loaded. [*]Note that I found that I had to move the BSAs out of /Data (or rename them) - unticking them in Archives tab did not work. See note below. [*]HD DLC BSAs available in /Data directory; ESPs not ticked (my standard configuration, as per MO's recommendations) [*]HD DLC BSAs available in /Data directory; ESPs ticked [*]The screenshots in tests 2 & 3 were identical, and noticeably visually improved over when HD DLC was not loaded at all (test 1) Image library showing screenshots from tests 1, 2 and 3: https://imgur.com/a/C2Oll @DrGamut: Try upgrading to Mod Organizer version 1.2.1 beta, which is the most recent stable beta, and is available on MO's Nexus mod page. Although 1.1.2 should work fine too, it's worth testing in the latest (stable) beta, which is the version I verified above. If you can confirm the issue still exists in 1.2.1 beta, then we'll need to examine your setup more carefully because there must be some local issue that's causing yours to not function correctly. Might need to be raised as a bug for Tannin, once we've looked at what might be different for you. Regarding disabling the HD DLC: As noted above, the only way I found to test disabling the HD DLC completely was moving its three BSA files out of /Data - or renaming them so they didn't have a .bsa extension. I first tried unticking them in the Archives tab, but I found that MO simply re-ticked them as soon as I moved out and back into that tab. Or to be more precise, it first showed the warning triangle indicating "Marked Archives are still loaded in Skyrim". Then when you move out of Archives, it re-ticks them for you automatically. Disabling vanilla content via the Archives tab is not supported in the current MO version, for the reason the warning triangle indicates (even if you untick them, the game will load them anyway because they're physically there in /Data.) It should be supported in the next version (released by Tannin for testing last night, as MO beta 1.2.2), which changes how vanilla content is handled, making it more like normal mods.

Thank you so much Tannin for implementing this! I shall now test it as hard as I can. I presume we should raise bugs both in this thread (to keep them centralised for other testers here) and on the official tracker (for Tannin's master reference) ? Cheers!

Hey guys, Regarding the following edit on z9's OP: [*]Incentivizes mod authors to provide BSA 'hotfixes' as loose files (NOTE: This has undesireable ramifications for MO users due to behavior of BSA extraction in MO ... BSA extracts last, so loose file hotfix is overridden by original version within the BSA! EDIT: this is fixed in the current beta and next release of MO) FYI, we've been discussing this over in the REGS thread, and I confirmed that the problem is fixed in 1.2.1beta whenever one installs a new mod (or Replaces an existing mod). Tannin has changed the BSA extraction such that it won't overwrite any existing loose files. But I've found that the fix for this actually causes a new, more serious issue: whenever an update patch is applied (example: upgrading Interesting NPCs from 3.05 to 3.06 using the 3.05->3.06 incremental patch archive), and that patch contains a BSA, any files in the patch BSA will not be extracted if they already existed in the original mod. So patch updates will be broken for any mods that use incremental patches, where the patch contains a BSA file that is meant to update files in the original release, and where the user has chosen to Extract all BSAs. The "don't extract from BSA when existing file is found" rule is applying always, and that is not desirable in the case where the user is choosing to Merge an update into an existing mod, and is extracting all BSAs. More information including steps to avoid, in my post on the REGS forum here. I've raised a bug report for Tannin, including a suggested way to handle the problem that solves both the original problem (dupe files between BSA vs loose, in a single archive) and the new problem: https://issue.tannin.eu/tbg/modorganizer/issues/644 I thought I should mention it here so z9 can update the OP, and so people know that upgrading to 1.2.1 beta to fix the occasional problem of dupes between BSA and loose could lead to a more serious issue, for the time being until it's fixed in a new beta or full release.

Actually there is still a problem with the change in 1.2.1 beta regarding BSA/loose file overwriting: BSAs never overwrite loose files. So if one is applying an update, and therefore choosing Merge, not Replace, and one always unpacks, this will mean that any mod that both uses BSAs and provides patches that include replacement BSAs, will not properly update existing files! :( So Interesting NPCs is an example again - not because it contains duplicates in BSA vs loose, but because it provides update files and those update files contain BSAs. So in effect, MO beta 1.2.1 has replaced one loose/BSA conflict problem, with a different one - and the new one is going to be more common and is therefore more serious. I'm going to raise a bug report for Tannin. EDIT: Report raised at https://issue.tannin.eu/tbg/modorganizer/issues/644 For now, the problem can be avoided by any of: [*]Never unpacking any BSAs [*]Not upgrading to the latest 1.2.1 beta - meaning one does still have the loose/BSA duplicate issue, but this would be fairly rare (doesn't affect 3DNPC; does affect ETaC, but only one when wants vanilla not custom meshes) [*]Never using patch updates that contain BSAs (or, more precisely, never using the Merge installation option when the new archive contains BSAs that one is going to unpack) [*]in the case of Interesting NPCs, and similar mods, one could either re-download the entire new full archive (2GB), or, better, one could manually make one's own new version by merging the update .7z file into the full download .7z file. [*]That's what I'm going to do now - merge the v3.06 update archive into the v3.05 full download archive - so that I can re-install a properly updated Interesting NPCs.

Ok ok, you win at everything :)

Yes, it is! I just came here to post the same thing. I re-read z9's OP in the Ramifications thread, and noticed the following EDIT that hadn't been there (or I'd missed) last time I read it: [*]Incentivizes mod authors to provide BSA 'hotfixes' as loose files (NOTE: This has undesireable ramifications for MO users due to behavior of BSA extraction in MO ... BSA extracts last, so loose file hotfix is overridden by original version within the BSA! EDIT: this is fixed in the current beta and next release of MO) So I just ran a quick test: Took a BSA from a mod, and extracted it in a new directory Edited one of the script/source/ files, adding a bunch of random text. Deleted most of the rest of the files I extracted. Packed up the BSA along with the files I extracted, including the modified one. Installed this as a mod in MO, choosing BSA extraction Went to the Mod Organizer/mod/test-mod/ folder, and viewed the file I edited as a loose version Confirmed it was my edited version; i.e. the unedited version in the BSA had not overwritten it, despite the BSA being extracted after the loose files. Therefore: As of MO 1.2.1 beta, the BSA extraction is smart: it only extracts files in a BSA that do not already exist in the mod directory, i.e. that do not already exist as loose files. (So not quite as you said above cstarkey; the BSA is still extracted last, but it's a smart, partial extraction.) More importantly - you are neither confused nor hopeless, cstarkey :D To summarise: The "duplicate files in BSA vs loose, when extracting BSAs" issue is in fact a non-issue for any user who upgrades to the latest MO beta. And it will be a non-issue for all users (who keep up-to-date) once 1.2.1beta makes a final release.

250KB/s :( And that's after I recently upgraded to 'super fast' ADSL. I think it's meant to be a meg up, but it's ADSL probably at a few miles' distance, so.. yeah :) I do now at least get 2MB/s download. Sometimes, when there's a following wind and I put my router on the floor so the bits can flow downhill. (I really miss being in the city. Oh 120MB/s up/down cable, I hardly knew ye!)

Regarding Interesting NPCs, I think this information might be outdated? EDIT: Yeah I think it is; I've just found the thread where the information originated: https://forum.step-project.com/topic/3864-bsa-extraction-loose-files/ It was confirmed later in that thread that 3DNPC doesn't actually have any conflicting files in the BSA v loose. I just did some checking, and could find no dupes between the BSA and the loose files. I verified it two different ways, to be safe, and I verified both 3.05 and the latest 3.06: 1. In MO, I installed 3DNPC twice, into two new mod entries "3DNPC Extracted" and "3DNPC Unextracted". As the names suggest, in the first one I told MO to extract the BSA, and in the second I did not I then used Winmerge to compare the two directories ("Mod Organizermods3DNPC Extracted" vs "Mod Organizermods3DNPC Unextracted"). It found no instances of files that conflicted. So the result of extracting vs not extracting gave 100% identical results - except of course the Extracted had a bunch of script/music files that Unextracted did not, and Unextracted had the BSA where the other did not. 2. I also double checked outside MO: I extracted the 3DNPC archive with WinRar, then I used BSAOpt to extract the resulting BSA into a separate folder. I then used Winmerge again to compare the BSA-Unpacked folder against the main contents of the 3DNPC archive. No duplicate files were found. To be specific, the 3DNPC.bsa file contains only the folders "scripts" and "music", and the main archive does not have these folders - it contains Meshes, Seq, Sound and Textures. So I think Interesting NPCs is OK - maybe they used to have dupes but cleaned it up in a recent version?

4 more mins.. my upload speed sucks :) Here ya go! https://TheBloke.livedrive.com/item/61a119d7d3c34ac687052559de6b3fd3

I have the 10.2 version of the modular version - this any good to you? I'm uploading it now in case it's helpful, take a few mins.

If you haven't read it, this guide is recommended: https://www.creationkit.com/TES5Edit_Cleaning_Guide_-_TES5Edit It's a guide to cleaning up mods with TES5Edit. It mentions Wild Edits briefly, and also links to a recommended video by Gopher where he shows the cleaning of Skyrim's master files. You've probably already done that, but it's worth watching the video as there is a place where he finds a wild edit and shows what it looks like in TES5Edit, as well as talking about how it might have occurred (via the CK as CJ said; in fact I think he even opens up the CK to show an accidental creation of an ITM, which is similar. ) Worth watching to know what to look for/avoid when building your own mods.

Yeah, CJ added that comment to REGS recently, as a result of the discovery of loose/BSA conflicts, e.g. in 3DNPC and ETaC. If by BSA extraction thread you mean Ramifications of BSA extraction in Mod Organizer, then yeah I've read it and have participated quite a bit in the last few pages. I'm afraid I'm not following what you mean by "In the steps you described what you'd be doing is downloading the BSA when you click don't extract and then delete it, now how is that right?" - could you rephrase? The steps involve first installing the mod with BSA extraction, then installing it again without BSA extraction - and the second time, choosing to Merge the mod not Replace. During the first installation, MO unpacks all the loose files, then unpacks the BSA on top of them - BSA wins the conflict. But the second install, the BSA is not unpacked, and therefore the loose files which get written in a second time, now overwrite any duplicate BSA files. Remember the second install is a Merge, not a Replace. The result of the second install is that you now also have a BSA file (as well as its contents), and so this should be deleted. I've already done it with 3DNPC and confirmed that everything looks fine. I'm pretty sure there can't be any hidden issues - ultimately we're just talking about extracting multiple archives, and working around the fact that MO always extracts the BSA last into the same directory. But I wanted to double check, and to confirm if there might be any quicker/easier way; but I don't think there is.

EDIT: My suggestion is in fact unnecessary, the problem has been fixed in MO starting with 1.2.1 beta. MO now does a smart extraction of BSAs, such that it does not extract any file in a BSA that it detects already exists within the mod. Therefore dupe files, BSA vs loose, will always favour the loose files, as the mod author will have intended and tested. Actually it might be a fairly simple change for MO. If it did the following automatically, I think the problem can never occur; could you guys confirm? : [*]User installs a mod. MO extracts its contents to a temporary directory, e.g. Mod Organizer/mods/$$TMP.modxyz. [*]MO detects the mod contains a BSA, and the user chooses Extract (or has that set by default) [*]If the mod does not contain a BSA, then MO simply renames the temp directory to the final directory, e.g. Mod Organizer/mods/3DNPC, and the process is now finished. [*]MO extracts the BSA, into the proper mod folder, e.g. Mod Organizer/mods/3DNPC [*]MO then moves all the files from the temp directory, Mod Organizer/mods/$$TMP.modxyz into Mod Organizer/mods/3DNPC , overwriting as necessary [*]MO deletes the (now empty) temporary directory Mod Organizer/mods/$$TMP.modxyz [*]The result is that where there are BSA/loose file conflicts, the loose files always overwrite the BSA. The only cost is the creation and deletion of of a temporary directory, and an extra move/merge operation; but as this is happening on the same filesystem, it should be close to instantaneous (no data is moved, only filesystem records are updated.) Can you guys confirm that I've not missed anything, and the above automation in MO would resolve the problem for all mods with BSA/loose dupes?

@cstarkey, @noobz, @CJ: Damn, another week, BSA extraction issue discovered :( BSA extraction is awesome for fine-grained file conflict handling, as you said. What method are you guys using for fixing the issue when you're discovering it? The way that occurred to me was, using Interesting NPCs (3DNPC) as an example: Install mod as normal through MO, with BSA Extraction enabled Then right click, re-install. Choose Merge instead of Replace. This time, when the BSA extract option appears, do not choose it. Then access the Filetree, and Delete the BSA that will now exist as well as its extracted contents. Is that what you're doing? Things are further complicated in the 3DNPC example in that, for me at least, I have the full installer only for 3.05, then the 3.05->3.06 upgrade patch. Hitting Reinstall on my 3DNPC mod will re-install only the patch, not the full original. However, looking inside that archive, I see that it contains a full replacement BSA. So the above method still seems to work in this case; but other large mods that both have patches and contain BSAs might need to be handled more carefully. Am I right in thinking that there's no real justification for a mod like 3DNPC having conflicting files both loose and in the BSA? So we can ask them if they'd mind cleaning up the dupes in a future version? I can't see the purpose for it, at least in 3DNPC's case - there's no installer, no choice of BSAs, so those conflicted files in the BSA are (for non BSA-extracting users) never ever going to be used; loose will always win. But does that apply universally? CJ said that ETaC gives different results depending on whether BSA is extracted or not - custom or default meshes - and I assume that's intentional, because she has a complex choice of different BSAs in different circumstances? So does that mean that MJB couldn't remove those duplicates between loose and BSA files? In which case we need a better longer term solution than the above method; some extra features in MO as to how it handles BSAs could be useful.

Thanks for the feedback! I was going to mention this already yesterday but didn't want to add more walls of text: what about, in addition to the auto-management of Skyrim + DLCs I mention, also have automatic handling for Unofficial patches? EDIT: I've just been re-reading some of the earlier posts, and I realise now that you did already address 'treating Unofficial patches as special', and didn't like the idea. I do understand why (there might be others who want to be special in future.) Nonetheless, I do still feel the following is a workable solution. There won't ever be more official DLCs for Skyrim, and therefore there shouldn't need to be more unofficial patches. If there are, then, well, a line could be drawn - and, more to the point, if a future 'special' case comes along, hopefully at least it will be created with some deference to MO, which after all is now the official platform of STEP, which is not a small body of support. Also, I wonder if it could be made configurable. Not hardcoded, but a text config file that defines what makes the DLCs special (list of files to look for in Data) and what makes the Unofficial patches special (Nexus IDs, filenames, or false-flag ESM as others mention after my post.) That way, it can be updated in future without further development work. Finally, I would be glad to volunteer with coding assistance for this, or any other solution you decide to implement. My C and C++ is a bit rusty, but I should be able to help at least with some of the simpler but time-consuming tasks. (Or I can help far more if any of this can be done via Python plugins?) TLDR, brief summary: Extend my proposed auto-detect handling of vanilla + DLC to contain one extra 'special mod type', namely the unofficial patches. Then have automatic, default sorting for all of vanilla, DLC and unofficial patch content. If necessary, disable re-prioritisation of these mods, and provide smart enable/disable such that if Dawnguard is unticked, it auto unticks UDGP. In detail: 1. New user installs MO. He gets a popup "MO detected the following content, and has created mod list entries for it: Skyrim, Hearthfire, Dawnguard" 2. He then gets a second popup: "Please now download the Unofficial Patches for Skyrim, Hearthfire and Dawnguard. Using Skyrim mods without these unofficial patches is not supported by most mod authors, and strongly discouraged. Please click the following link to find download locations for these mods: <link to new or existing STEP wiki page which talks about Unofficial patches and has clear links to each of them>" 3. New special rules processing for installation of new mods: auto-detect Unofficial patches. Based on either: Nexus ID=xxxxx (e.g., 19 for UKSP, 23491 for UDGP), OR if the mod contains "Unofficial Skyrim Patch.esp and bsa". If a mod is installed that matches EITHER of these rules (both rules checked in case Nexus is not used), then the mod is considered to be the appropriate Unofficial patch, and is handled specially 4. Special rules for vanilla, DLC content, and for Unofficial patches: They are always auto-sorted to the top of the mod list, in the interleaved order I defined before: 0 | Skyrim 1 | Unofficial Skyrim Patch 2 | Hearthfire 3 | Unofficial Hearthfire Patch 4 | Dawnguard 5 | Unofficial Dawnguard Patch ... etc This time I have put them all in bold - they are now all special mods, where before only Skyrim/Hearthfire/Dawnguard/(Dragonborn) were special mods. ----------- OR, a modification to the above: Have auto-detected vanilla/DLC content (as per my original proposal), and have auto-detected Unofficial content (as mentioned above), BUT - then merge them together. I.e. the user gets a popup telling him/her to install the unofficial patches. When he/she does, MO automatically inserts it into the appropriate vanilla/DLC mod, and then changes the name from "Skyrim" -> "Skyrim (including Unofficial Patch)"; "Dawnguard" -> "Dawnguard (including Unofficial Patch)". Therefore the user has no choice as to how to sort the Unofficial patches, and no choice as to whether to disable them (once he's installed them.) They're just considered part of the vanilla content, and MO handles this automatically. (at least unless he disables this auto management in the options.) That would seem closest to how the Unofficial guys want it to be done, while making the least significant changes to MO's sorting/mod management paradigms. To be honest, if I were doing it I might even go the whole way - and have MO automatically download and install all the unofficial patches (that it has detected it needs based on installed DLC) the first time it's installed. But I don't know if that would be something you'd want to do. ----------- Sorry to keep banging on about my idea, but in particular now you've said there's no technical problems with it, I still do feel it keeps closest to MO's current flexibility and paradigms, whilst adding new transparency for users regarding default sorting, and also enabling easy per-profile management of DLC content. I will give up on the idea if you don't like these addendums :) But I had been going to mention that anyway so I thought I'd put it out there. Thanks again!

Yes; but the number of those conflicts is not insignificant, I would say. I have no knowledge of how WB sorts mods (only ever used it for Bash Patch), and I've not used NMM for more than a year (since I discovered MO and scrubbed NMM forever from my drive; Recycle Bin is too good for that crap.) So I don't have any knowledge of how these sort BSAs.What I do know though is that there are quite a number of mods whose installation instructions include the words "When asked whether to overwrite files, choose YES|NO". And/or: "For compatibility with ModX, install ModX first then install my mod and choose to Overwrite all files"I assume these are mostly scripts, but I believe there are some other kinds of files that can also sometimes be loose?So, with regard to: My experience is that many mod authors do include specific file-ordering instructions, and these could definitely differ from plugin order.Specifically, mod authors generally always assume NMM, and they give Overwrite instructions accordingly. Those instructions are separate to plugin ordering, where they will either say "use BOSS/LOOT", or they will give a specific order. But that's treated entirely separately from their instructions regarding any loose file conflicts.In other words, yes the mod author will know when he has conflicts and yes he will give instructions for prioritisation based on that: but he has no expectation of this being tied to the plugin order, at least in the case of scripts.So if we do suddenly have auto-sorting, I believe there's a definite risk of the defaults being out of line with the expectation and testing of many mod authors. And that's irrespective of whether a STEP guide is followed. (Indeed, a STEP user will be much better off than the average user, because the guide will be assuming MO where the mod author will not, and the whole purpose of the guide is to test and achieve multi-mod compatibility.)Without auto-sorting, any MO user who understands even the basics of how MO works could easily translate those NMM-specific instructions into MO mod-list ordering.My fear is that with auto-sorting, such a user is likely to just assume that the auto order 'must be right', when it may well not be - particularly with regard to scripts. And/or such users will start flooding the Nexus forums with "I use MO and it's sorted your mod ABOVE ModX, but you said I need to overwrite files from ModX, is that OK?" Which won't be good for anyone, least of all the reputation of MO (especially as apparently some prominent community members are already inclined to be down on it; to their shame.)

Regarding tying mod order (left pane) to plugin order (right pane): Same as Tech said, I have a number of concerns regarding this. While it might simplify things, it does seem to risk being a backwards step. Personally speaking, as long as it's possible to disable it and still achieve current MO behaviour, I would be fine. But I do feel that MO's ability to achieve fine-grained mod prioritisation, even on a file-by-file basis (when hiding conflicting files), is one of its great strengths. I extract all my BSAs, and pay close attention to the Conflicts tab of every mod. I've followed a number of STEP guides, both in terms of coarse-grained ordering of mods, and fine-grained control of individual files by file hiding. On top of that, I've done a lot of my own conflict resolution, in particular adjusting priorities and hiding files to ensure that conflicting scripts are applied in the order I think/know is best. There is already a 'soft' link between plugins and mod ordering: the "Potential mod order problem" warning, which causes me to sometimes re-order my mods in the left pane to match their positions in the right pane. To my mind, that is as much of a link as I would personally want to see. As I say, as long as it's a change that can be disabled, it won't hurt me personally, so if it's really the best or only solution for the community at large, then fair enough. But it does feel like a default removal of a useful ability, and I feel that the underlying problem that generated this thread could be fixed without needing to resort to that, if the "treat vanilla/DLC content like mods" solution I described above is workable (and I can't currently see why it wouldn't be.) On top of which, it would seem a big shame if STEP guides had to go through a lot of convoluted work to update to the new feature, as Tech described; and they couldn't just tell users to turn off the auto-sorting because that would still leave the underlying Unofficial Patch problem. So a solution to that problem that doesn't require auto-sorting, and thus doesn't invalidate every STEP guide, feels strongly preferable to me.

Tannin, Thanks for the further explanations. I understand that filesystem-based links of any kind are out of the question. But the system that I described does not, I believe, require them. Of course I may be wrong; not knowing the full details of MO's internals. I'd like to restate it again, in light of your further comments. I've also summarised the intended results in a TLDR section at the end. Apologies for the length, I do like to go into detail - especially in this case where I want to state my assumptions and thoughts on implementation, in case I am wrong in my expectation of how MO works and how it could be changed. Despite the length of the below, my intention and hope is that the system I describe is rather simple - certainly I feel it would be for the user to use, and more transparent and understandable than how things are now. And I feel that the internal implementation work in MO should not be too extensive, either, because it just boils down to "autodetect a bunch of specific file names in Data; based on these, generate some specially-flagged mod entries (left pane); these mod entries don't have a filesystem based Filetree, but rather a hardcoded list of files to use when processing that 'mod'" For simplicity, the following example assumes a new MO install, on a vanilla Skyrim Data directory. The 'upgrade' procedure on an existing install shouldn't be much different. Not covered: advanced options to allow for advanced users to disable/change the proposed default system. I am assuming that if a system like this was added, a couple of config options would also be added to disable it or change its operation. 1. MO scans the Data directory, and detects what ESMs and BSAs exist. It is looking for certain, named files. It finds: Skyrim.esm, update.esm, Skyrim - *.bsaDawnguard.esm, Dawnguard.bsaHearthfire.esm, Hearthfire.bsa 2. Based on finding these known files, it decides that this user has the following vanilla/DLC 'mods': SkyrimDawnguardHearthfire 3. This new user's mod list (left pane) will not start empty (as it would in current MO), but will contain three default entries: Skyrim, Dawnguard, Hearthfire These default entries are highlighted in bold, to differentiate them from normal mods that the user will install later. 4. Internally, these are special 'vanilla' mods. MO has created them automatically, and it has flagged them internally to indicate they are vanilla/DLC content. 5. There are two types of these 'vanilla' mods, and the type is applied internally by MO to provide the following special UI behaviour: Neither type of vanilla 'mod' can be removed from the list or re/uninstalled:right click -> Remove and Uninstall and Reinstall are greyed out for these mods.Vanilla typeThe mod called 'Skyrim'. This has a hardcoded priority 0, and no enable checkbox.Thus it cannot be de-activated, and it cannot be moved from the top of the mod list.DLC typeThe mods called 'Dawnguard', 'Hearthfire', 'Dragonborn' (In this example, the user does not have DB).These can be de-activated (unchecked), and they can be re-prioritised/moved6. These 'vanilla' mods have hardcoded file lists. These file lists are set by MO and cannot be changed, and they match the list of known files that MO detected in step 1. So the 'Skyrim' mod is hardcoded to contain:Data/Skyrim.esm, Data/Update.esm, and all the Data/Skyrim - *.bsa files. The Dawnguard mod is hardcoded to contain:Data/Dawnguard.esm and Data/Dawnguard.bsaetc. When I say hardcoded - what I mean is, when MO processes this mod, it detects it's a special mod (according to some kind of internal flag indicating as such), and then it reads an internal table to find out 'what files does this special vanilla/DLC mod contain?' Its Filetree is then provided according to the internal file lookup table I described in step 1. E.g. whenever MO processes Dawnguard, it knows that the Filetree for this mod is Data/Dawnguard.esm and Data/Dawnguard.bsa. That is exactly what the user would see if he looked at the UI Filetree for the 'Dawnguard' mod, and (most importantly), that is what files would be used when the virtual Data generator processes this mod. Ideally, this 'hardcoded file list' would actually be a text configuration file, rather than being actually hardcoded. 7. So no changes are made to the Data directory. No filesystem links are created. The Filetree of the vanilla/DLC mods exists only by virtue of MO's hardcoded rule for these special vanilla mods, which tells it the files to use (as per point 6) and that these files will be found in Data. ------------- From that point onwards, everything happens exactly as now in MO. So what has been achieved? Internally, not much has changed:- Data is not changed; there are no filesystem links; all the same ESMs and BSAs etc exist in Data as they do now, unchanged. What has changed is that Skyrim, and the official DLCs, have been forced into the standard MO mod structure. They are now visible in the left pane as usual. They now obey left-pane ordering like any other mod. In other words, we've gone from the current system - where MO has a special rule set for vanilla files it finds in Data - to a system where MO still has a special list of vanilla Data files, but it uses this to create standard mod file entries. Then it processes those Mod file entries exactly like for any other mod. Because of all this, the user can now re-prioritise the official DLC mods, and in an obvious and familiar and completely transparent way. Therefore he/she can now interleave the unofficial patches with the official content, to achieve the correct ordering regardless of whether BSAs or loose files are used. So the user can - and will, once he follows a STEP guide or any MO instructions - end up with the following left-pane mod list: (priority | mod name) 0 | Skyrim1 | Unofficial Skyrim Patch2 | Hearthfire3 | Unofficial Hearthfire Patch4 | Dawnguard5 | Unofficial Dawnguard Patch6 | .. other mods I have highlighted in bold the vanilla/DLC mods, that MO added automatically for this user. I would suggest that bold highlighting would also be applied in the real MO left pane, to indicate that these are 'vanilla game mods', which the user did not install him/herself but which MO automatically created out of his vanilla content. Finally, it provides a new feature for MO - easy, one-click enabling/disabling of DLCs, per-profile. The user can create a profile that does not use Dawnguard or Hearthfire or Dragonborn, or any combination, if for some reason he prefers to play the game without that DLC. Perhaps he dislikes something it changes, or perhaps it breaks a mod he really wants to play. Or, more commonly - perhaps she is a mod creator, who needs to test her mod both with and without different combinations of DLCs. With this new system, the modder can see that she can now very easily disable any/all DLCs on a per-profile basis, just like she would disable any other mod. TLDR: This is mostly a UI change. We have forced the vanilla/official content into the standard MO mod structure. We have replaced MO's hardcoded rules for processing vanilla/DLC content in Data, for a set of hardcoded rules for converting that vanilla/DLC Data content into a list of autodetected, auto-added, 'special' MO mods. When launching the game, vanilla/DLC content is no longer prioritised according to different, behind-the-scenes ordering rules. Instead, 100% of data for the game - vanilla, DLC, and all mods - is now visible in the left pane, and is prioritised according to the priorities set in the left pane. The loading of all content, and its ordering, is therefore much more transparent. It's simpler, and very easy for everyone to understand. Instructions for most users will be just : "Ensure your content in the left pane matches the order of the STEP guide, then launch BOSS/LOOT (possibly after adding custom BUM rules) to sort the right pane." And the work involved in changing MO would, I am assuming, be fairly minimal: new routine to autodetect content in Data and create left-pane mod list entries a new flag to allow 'special/vanilla/DLC' mods that can't be removed/uninstalled, and specifying that the Skyrim mod can't be re-prioritised or de-activated special handling for the the Filetree of these vanilla mods, such that their Filetree is not read from "Mod Organizer/mods/<mod>" as usual, but is set by a hardcoded, internal list of specific file names. Applying both in UI (Filetree tab) and, most importantly, in virtual Data directory generation. (ideally) a couple of new advanced configuration options, to disable or modify the default behaviour I describe.

But does MO have to touch the Data directory in order to implement 3? Imagine the following: [*]New user installs MO, on top of completely vanilla Skyrim + DLC installation [*]When he loads, he sees the Mod list, but it is not empty as now. Instead, MO has detected that he has a) Skyrim b) Hearthfire c) Dawnguard (let's say this guy does not have DB), and so it has given him a mod list with the following entries: [*]Skyrim [*]Hearthfire [*]Dawnguard [*]These mods would be formatted slightly differently to indicate they are vanilla/DLC content, e.g. perhaps they are in bold. [*]The Skyrim 'mod' would have a hardcoded priority = 0, and no activate/deactivate checkbox [*]Hearthfire and Dawnguard have normal controls - activate checkbox, and drag/drop priority changing. [*]Fields such as Nexus ID, rating, Endorse etc would be hardcoded to N/A or similarly disabled. [*]What has happened behind the scenes: [*]MO has detected the presence of Skyrim, Hearthfires and Dawnguard. It has done this probably just with filename matching in the Data directory. [*]It has created three default, virtual mods, which contain references to their assets. [*]These could be entirely 'virtual' references - i.e. no physical filesystem connection at all - or they could be implemented with filesystem soft/hard/junction links, as other posters have detailed. [*]Either way, MO has some new code that detects certain, named files in the data directory, and has converted these into mod entries. [*]The user now installs other mods, including the unofficial patches, as normal. [*]He will be told by STEP guides and other collected wisdom, that in MO he should put each Unofficial Patch mod entry directly after its corresponding vanilla 'mod'. [*]So once this user has the unofficial patches installed, his mod list with priorities would be: [*]0 | Skyrim 1 | Unofficial Skyrim Patch 2 | Hearthfire 3 | Unofficial Hearthfire Patch 4 | Dawnguard 5 | Unofficial Dawnguard Patch 6 | .. other mods So MO would have code to configure itself in the way that many (advanced) users are already doing by hand - treating their vanilla/DLC Data assets as mods, and allowing the DLCs to be re-prioritised. It would do that without making any physical changes to Data. It would achieve this either in a purely internal way, or by virtue of automatically creating filesystem soft links from Data to the appropriate Skyrim/DLC mod directory within MO's normal filesystem structure. So there would now be standard mod folders - "Mod OrganizermodsSkyrim" folder, "Mod OrganizermodsDawnguard", etc - created automatically on startup, and these could contain filesystem links to the appropriate files in Data, or MO could just handle that internally, knowing that the Skyrim mod always contains Skyrim.esm and named Skyrim*.bsa files, and populating the virtual directory and the Conflicts tab as such, without any actual filesystem reference. Then MO would handle its virtual Data, and game loading, exactly as it does now - with no conflicts, because it's treating all assets as mods, with no 'unmanaged' BSAs at all. Perhaps it would also be necessary to remove the checkboxes against the DLC BSAs in the Archive tab, such that there can't be a situation where the user has ticked, say, the Dawnguard DLC - and so has the ESP - but has for some reason unticked the Dawnguard.bsa file, which would cause that BSA to be loaded regardless because of the esp? This seems to me to maintain MO's data integrity paradigm. It does not require removing the existing BSA management feature - which worries me (admittedly more out of principle of not losing potential features, and ignorance of the full ramifications, rather than a firm understanding of any way it would hurt me as a user who extracts 100% of BSAs, including, soon, vanilla/DLC BSAs). In fact I would say it extends one of MO's core paradigms - the ability to have full control and management over a complex mod installation. It extends it because now all DLCs are manageable just like, and as easily as mods. If a user wants to start a game without Dawnguard running, he can do so with a single uncheck, in an obvious and logical place; exactly like how he'd disable any mod. DLCs can now be prioritised however one wants, like mods (though users should be advised that normally, only Unofficial patches should go between vanilla/DLC entries) and can easily have their conflicts/overwrites viewed, just like any other mod. All within the existing, familiar and effective infrastructure. Have I missed anything important that would break this? It does seem to me that if the ordering works fine when an advanced user manually makes mods out of the appropriate files, it should be perfectly possible for MO to automate that process - especially if combined with filesystem links, so that all MO's existing mod/file-management code still works unchanged. PS. Tannin, I just wanted to say thank you for what is I believe the most elegant, sophisticated, and simply genius solution for mod management I have seen, or can imagine. Earlier today I posted a much longer gushing endorsement to MO's main Nexus thread, but I wanted to mention it again here as I was replying to your post, and in case you don't check that Nexus thread as regularly. And also because you've needed to post here to defend MO (not from people in this thread, thankfully.) That completely baffles me. MO and its virtualised system is simply superb, and I don't understand why everyone doesn't use it - let alone why anyone would use minor issues like this one against it, rather than submit problem reports/work together to get them resolved as quickly as possible. I hope you're justifiably proud of your creation - it's fantastic!

Thanks for the details, Aiyen! And apologies for asking when you'd already explained. I hadn't seen your other response, I came to that thread late and read page 8 onwards, so missed your post and quotes of it on page 6/7 :) Sorry about that, I will read that post now and also check out SRLE. Thanks again!

Hey Aiyen, Could you confirm that, as z9 suggested, you are extracting all BSAs - including the vanilla Skyrim and DLC BSAs? I was going to post this in the other thread as a question, but I don't want to derail it with newbie-ish questions and I think you're doing exactly what I was thinking of doing: Before I even knew about this newly-discovered issue with BSA extraction, I had been considering unpacking all my BSAs - including all the vanilla/DLC Skyrim BSAs. Not for any great purpose, but simply because I was interested to see all of vanilla Skyrim's assets appear in my Conflicts tab, and see exactly what mods were overwriting what files from the vanilla game, versus official DLCs, versus files newly added by mods. I didn't actually do that, because I thought it would end up being too much noise in my Conflicts tab (nearly every mod would show a Conflict icon, where before I could use that icon to decide which ones I needed to review), and also because I wasn't 100% sure if there was any problem from doing that. So now I want to ask: is that what you're doing? And therefore it's OK to do? So: I convert my vanilla/official-DLC assets into MO mods - making one mod for Skyrim, another each for DG, DB, HF - and in those mod archives I include both the ESMs and the BSAs. Then I install those mods through MO, and it will extract those BSAs as normal. Thus the vanilla/DLC game content is treated identically to all my other mods: 100% loose files. Then in my Mod load order, I interleave the vanilla Skrim/DLC 'mods' with the Unofficial patches - as indeed MO's "Potential mod order load problem" warning would likely tell me to do, based on the LOOT sorting. So my mod order would read: Skyrim, USKP, Dawnguard, UDGP, Hearthfires, UHFP, Dragonborn, UDBP, HD DLC, HD DLC Patch, <rest of my mods> My entire game - vanilla, DLC and mods - would now be loose files, and loaded in an order defined only by MO. I would have 0 BSAs of any kind; my Archives tab would be empty (or all unchecked.) So is that what you're doing? Are there any problems with doing that - besides having the extra clutter in the MO Conflicts tabs? Are you unpacking all the Skyrim BSA files too (e.g. Skyrim - Sounds.bsa, -Meshes.bsa, etc), or only the DLCs? You said you had nothing checked in Archives, so it sounds like all? Thanks very much in advance!