POLL: Merged versus Separate in MO

[DoubleYou]

Our guides typically instruct users to merge multiple files provided by any given mod into the same "mod" in Mod Organizer. However, I know several of our more advanced members prefer to not do this, and install the individual mod files as separately, myself included. I have therefore created this topic so we can get a poll of users showing their preference in this matter, and perhaps spark discussion as to the best practice that perhaps would be better employed and promoted.

I will list some pros and cons of which I am aware, which may be added to based off your provided input.

Merge

Pros:

1. Less "mods" installed in the Left Pane.
2. No need to move multiple "mods" priority when changing priority.
3. Automatically names all mod files to the same name on installation (provided the meta information is downloaded for the mod archive), so no need to change the mod name in most cases.

Cons:

1. The last mod file merged into the main mod will set the version for the "mod" in Mod Organizer. Many times, due to optional files having a file version themselves, this will not accurately reflect the current mod's version, and may cause issues with the Check for updates mechanism.
2. Harder to keep track of your installed mod files.
3. If only one of the mod's file updates, many times you will need to reinstall all of the mod's files, and it may be unclear as to which time you may need to choose Replace for any particular update versus Merge.

Separate

Pros:

1. Update check is more accurate, as Mod Organizer will track the version of the optional file separate from the main file in most cases. A good example is kryptopyr's Patch Hub. If you merge all updates, the last installed file will be used to set the installed version, and will be the only file checked against for updates. If you install separately, Mod Organizer will check each individual file's version for update.
2. Easier to keep track of all the additional files from a mod you have installed.
3. If only one mod file updates, and not any of the others, you only need to install and Replace that mod file. There is no need to worry about whether or not you should choose Merge versus Replace. 

Cons:

1. More "mods" installed in the Left Pane.
2. If you need to change the mod's priority, you need to move multiple "mods."
3. You have to be very careful when installing additional files to ensure you are using a unique name. Most of the time, the dropdown contains a suitable name, but often you need to make your own name. This can also cause issues if you accidentally name the file wrong upon updating, and accidentally Replace the wrong mod file.
4. You have to ensure that you install the additional mod files in the correct priority against itself. This, however, is normally quite straightforward.

---

Additional information about this has been added to the wiki here: https://stepmodifications.org/wiki/Guide:Mod_Organizer/Merging_vs_Separating

[Mousetick]

- Separate clearly and naturally shows asset/resource overwrites and conflicts, whereas merge makes them disappear (in case the merged contents overlap).

- Separate allows more flexibility in sorting/prioritizing overwrites and conflicts.

- Separate allows more flexibility in grouping related files together, and organizing them in any way one wants. I actually don't like patches that are installed via FOMOD alongside the main mod. I'd prefer if they were supplied as separate files. For example, RWT has patches for other mods like Beyond Reach and Wyrmstooth, that are installed by its FOMOD. I go as far as taking them out of the RWT mod folder and putting them into their own mod folder, placed in the same group as Beyond Reach, or Wyrmstooth, respectively.

- Separate significantly increases the number of mods on MO2's left side, and the human brain doesn't handle long lists very well. To counter this, I create lots of separators to group mod folders into smaller lists of related mod folders.

- MO2 slows down a lot as the number of mod folders increases and it needs to reprocess/update overwriting conflicts, particularly when BSA archive parsing is enabled. But I'm running a potato, so it may be specific to me.

That said, there is probably a balanced middle-ground to be found between the 2 approaches. It seems a lot of users on Nexus prefer 'All-in-One' packages because they're easier to install. Ease of installation and shorter time spent micro-managing mods are good things in general. Some users, myself among them, prefer to have maximum freedom in micro-managing and maintaining their mod list - I'd guess they're in the minority.

[z929669]

  On 12/27/2022 at 11:33 AM, Mousetick said:

- Separate clearly and naturally shows asset/resource overwrites and conflicts, whereas merge makes them disappear (in case the merged contents overlap).

- Separate allows more flexibility in sorting/prioritizing overwrites and conflicts.

- Separate allows more flexibility in grouping related files together, and organizing them in any way one wants. I actually don't like patches that are installed via FOMOD alongside the main mod. I'd prefer if they were supplied as separate files. For example, RWT has patches for other mods like Beyond Reach and Wyrmstooth, that are installed by its FOMOD. I go as far as taking them out of the RWT mod folder and putting them into their own mod folder, placed in the same group as Beyond Reach, or Wyrmstooth, respectively.

- Separate significantly increases the number of mods on MO2's left side, and the human brain doesn't handle long lists very well. To counter this, I create lots of separators to group mod folders into smaller lists of related mod folders.

- MO2 slows down a lot as the number of mod folders increases and it needs to reprocess/update overwriting conflicts, particularly when BSA archive parsing is enabled. But I'm running a potato, so it may be specific to me.

That said, there is probably a balanced middle-ground to be found between the 2 approaches. It seems a lot of users on Nexus prefer 'All-in-One' packages because they're easier to install. Ease of installation and shorter time spent micro-managing mods are good things in general. Some users, myself among them, prefer to have maximum freedom in micro-managing and maintaining their mod list - I'd guess they're in the minority.

Expand  

Agree with all this. I do the same with the exception of parsing out FOMOD-installed patches. It's a good idea, but not worth the added effort for me (would be if my mod list were actually stable for playing rather than testing ... too much flux on my end).

This is likely your potato, as I don't notice any processing lag at this point with around 500 mods in the left pane (including many BSA, and I have archive parsing enabled).

---

For our guides, I see merging as a convenience for us in keeping instructions concise, but I also see the value of NOT merging for reasons stated. It may be worth considering each use case and favoring separate installs. The FOMOD-supplied patch problem is still an issue though. More than once, I have encountered the problem of not knowing for sure how I installed a particular mod because of this and wound up just reinstalling for OCD reasons rather than delving into deductive analysis ... which is an annoying time suck.

[z929669]

  On 12/26/2022 at 9:24 PM, DoubleYou said:

 

1. Update check is more accurate, as Mod Organizer will track the version of the optional file separate from the main file in most cases. A good example is kryptopyr's Patch Hub. If you merge all updates, the last installed file will be used to set the installed version, and will be the only file checked against for updates. If you install separately, Mod Organizer will check each individual file's version for update.

 

Expand  

I wasn't aware of this. I know KPH files are tracked individually, because I keep each one separate, but I wasn't clear that this was due to the Nexis files section they are under.

To be clear, KPH files are under "Miscellaneous Files", which probably does allow individual file tracking independent from the Main File(s) and associated global version. So can you confirm this API behavior is also attributed via "Optional Files" or other file sections?

Is it possible that the indicated check box tethers all "Main Files" to the global version and that keeping it unticked allows independent tracking of ANY file?

Looking at the API response json from KPH for one of the latest files (note that this data for any given file is retained in the json for all time, so if this file is updated, there will be a new unique entry for the file appended to the list after this entry):

,
    {
      "id": [
        343941,
        1704
      ],
      "uid": ,
      "file_id": 343941,
      "name": "Saints and Seducers - Voiced Narrative __ TCIY",
      "version": "1.1",
      "category_id": 5,
      "category_name": "MISCELLANEOUS",
      "is_primary": false,
      "size": 9,
      "file_name": "Saints and Seducers - Voiced Narrative __ TCIY-19518-1-1-1672115518.7z",
      "uploaded_timestamp": 1672115518,
      "uploaded_time": "2022-12-27T04:31:58.000+00:00",
      "mod_version": "1.1",
      "external_virus_scan_url": "https://www.virustotal.com/gui/file/0d520baa22f4da9ab21df3962252e95b0766a438124407a4e109957d124462c4/detection/f-0d520baa22f4da9ab21df3962252e95b0766a438124407a4e109957d124462c4-1672115520",
      "description": "Compatibility patch for Saints and Seducers - Voiced Narrative (by G The Generous) and TCIY. Replaces (do not use with) the Saints & Seducers TCIY patch (CC-Saints&Seducers_TCIY_Patch.esp). ESL-flagged.",
      "size_kb": 9,
      "size_in_bytes": 9035,
      "changelog_html": null,
      "content_preview_link": "https://file-metadata.nexusmods.com/file/nexus-files-s3-meta/1704/19518/Saints and Seducers - Voiced Narrative __ TCIY-19518-1-1-1672115518.7z.json"
    }
  ]

... I think the important part of this is the "version" / "mod_version". These are always equal for any given entry in my testing looking at Step SSE and KPH (Step doesn't get this file-level versioning as KPH does ... our files always look outdated to MO). KPH global version is 3.4, and notice how this value is not equal to either version string. MO is fetching the global version for Step but the file version for KPH, so it's not clear or documented anywhere I can find how to achieve this.

[DoubleYou]

I have no idea how MO2 seems to differentiate between the files. I just have noticed this feature. The Nexus checkbox you indicate simply updates the mod page's version at the same time of uploading whatever file you are modifying to the version you specify for the file. This is merely a convenience checkbox, allowing you to update without also having to change the version number in the Description as well, and I do not believe it is in any way related to the MO2 functionality of checking for updates. There are several ways that MO2 might be checking for an updated file, as the json includes a very detailed listing of information about each file installed, based off meta data it fetches through the Nexus API.

[z929669]

  On 1/2/2023 at 3:53 PM, DoubleYou said:

I have no idea how MO2 seems to differentiate between the files. I just have noticed this feature. The Nexus checkbox you indicate simply updates the mod page's version at the same time of uploading whatever file you are modifying to the version you specify for the file. This is merely a convenience checkbox, allowing you to update without also having to change the version number in the Description as well, and I do not believe it is in any way related to the MO2 functionality of checking for updates. There are several ways that MO2 might be checking for an updated file, as the json includes a very detailed listing of information about each file installed, based off meta data it fetches through the Nexus API.

Expand  

Yeah, the only file-specific data I see from that json that is consistent between multiple versions is "name", which seems more unreliable than a "file_id", which is unique per entry and varies probably by "file_name".

[Greg]

Look in the meta.ini file for each mod and you'll se the modid, fileid, and version number that Mod Organizer uses for this. The information here is for The Danse Dilemna that includes two files with different versions: FaceMaxsonV2_9 (Version 2.9) main file and the SearchAndDestroyAlterative optional file (Version 2.6) and looked at the meta.ini file that Mod Organizer creates.

[General]
gameName=Fallout4
modid=21923
version=2.6.0.0
newestVersion=2.6.0.0
...
installedFile="SearchAndDestroyAlternative-21923-2-6.7z"
...

[installedFiles]
1\modid=21923
1\fileid=97510
size=1

It looks as if this is structured so that it theoretically can support multiple files, but Mod Organizer overwrites this with the latest file installed if you use the Merge option. If you install both files as separate mods, Mod Organizer has the correct modid, fileid, and version information for each file.

EDIT: On second thought, it doesn't seem adding multiple files in [installedFiles] is all that useful anyway because the metadata has only one version and newestVersion element. In any case, I believe the Nexus API requires modid and fileid as parameters in the getFileInfo request that Mod Organizer uses to check for updates.

Is this what you are asking about?

[z929669]

  On 1/2/2023 at 4:50 PM, Greg said:

Look in the meta.ini file for each mod and you'll se the modid, fileid, and version number that Mod Organizer uses for this. The information here is for The Danse Dilemna that includes two files with different versions: FaceMaxsonV2_9 (Version 2.9) main file and the SearchAndDestroyAlterative optional file (Version 2.6) and looked at the meta.ini file that Mod Organizer creates.

[General]
gameName=Fallout4
modid=21923
version=2.6.0.0
newestVersion=2.6.0.0
...
installedFile="SearchAndDestroyAlternative-21923-2-6.7z"
...

[installedFiles]
1\modid=21923
1\fileid=97510
size=1

It looks as if this is structured so that it theoretically can support multiple files, but Mod Organizer overwrites this with the latest file installed if you use the Merge option. If you install both files as separate mods, Mod Organizer has the correct modid, fileid, and version information for each file.

EDIT: On second thought, it doesn't seem adding multiple files in [installedFiles] is all that useful anyway because the metadata has only one version and newestVersion element. In any case, I believe the Nexus API requires modid and fileid as parameters in the getFileInfo request that Mod Organizer uses to check for updates.

Is this what you are asking about?

Expand  

Yeah, the key is that modID and fileID must remain constant while the versionID (or equivalent) changes as the basis for comparison. However, the fileID always changes in the json (which adds each updated file as an independent json record, regardless of the name being the same) when the same name-string file is updated.

MO must be comparing eacch file versionID to the global versionID .... but notice that this isn't the behavior for KPH files json, which all have equal file "version"/"mod_version" (as all Nexus files) that differ from KPH global versionID.

Compare KPH to Realistic Boat Bobbing Patch Hub behavior in MO. If you install KPH as individual files, MO tracks each distinctly and correctly with "version"/"newestVersion" compare (in meta.ini). If you do the same for RBBPH, this is NOT the case. My question is: why is it working for KPH but not others?

[Greg]

I wonder if this is because RBBPH has everything in Main Files and Nexus isn't setup to track mixed versions in Main Files?

[z929669]

  On 1/2/2023 at 5:38 PM, Greg said:

I wonder if this is because RBBPH has everything in Main Files and Nexus isn't setup to track mixed versions in Main Files?

Expand  

zactly ... don't know but really want to know

[Mousetick]

It's impossible to reliably track individual files from Nexus because they're only identified by a unique ID, their version number is basically meaningless. So MO2 must use "heuristics" to try and determine if it's been updated from various bits of metadata, and unsurprisingly it's doing a pretty bad job of it, compounded by the fact MAs don't always update the mod version number (the one shown on the main mod page).

Examples:

• MyWonderfulModFile.7z from Mod "My Wonderful Mod" has Nexus file ID 12345. The MA has given this file a version "1.0".
• Later on the MA uploads MyWonderfulModFile.7z for "My Wonderful Mod" and gives it version "1.1". Its Nexus file ID is 98765.
• There is no way to tell that MyWonderfulModFile has been updated from version "1.0" to version "1.1", since they're different Nexus files, with they own unique IDs.

The unique file ID and game ID (e.g. Skyrim SE, Fallout 4) are all that's necessary to identify a file on Nexus. They can be used to retrieve an archived file with the download API for example. The parent mod, mod version or individual version are useless, but MO2 stores them so it can try to keep track of them, and show relevant information to the user.

There is an extra field in the meta.ini that seems to indicate the type of file: nexusFileStatus. From what I can gather, its value is

• 1 for a Main file
• 3 for an Optional file
• 5 for a Miscellaneous file
• 1000 for a deleted file (or perhaps Old file, or archived file, ...) not sure

Greg is probably guessing correctly that MO2 only tracks versioning of Main files (nexusFileStatus=1), by comparing the file's version number to the parent mod's version number. MO2 is doing a best effort of guesstimating if a Main file is up to date, but it falls apart when the parent mod's version number is itself incorrect, or if there are multiple Main files with differing version numbers.

For the other types of files, MO2 doesn't track versioning, they're always "up to date". Or something like that.

[z929669]

  On 1/2/2023 at 6:32 PM, Mousetick said:

It's impossible to reliably track individual files from Nexus because they're only identified by a unique ID, their version number is basically meaningless. So MO2 must use "heuristics" to try and determine if it's been updated from various bits of metadata, and unsurprisingly it's doing a pretty bad job of it, compounded by the fact MAs don't always update the mod version number (the one shown on the main mod page).

Examples:

• MyWonderfulModFile.7z from Mod "My Wonderful Mod" has Nexus file ID 12345. The MA has given this file a version "1.0".
• Later on the MA uploads MyWonderfulModFile.7z for "My Wonderful Mod" and gives it version "1.1". Its Nexus file ID is 98765.
• There is no way to tell that MyWonderfulModFile has been updated from version "1.0" to version "1.1", since they're different Nexus files, with they own unique IDs.

The unique file ID and game ID (e.g. Skyrim SE, Fallout 4) are all that's necessary to identify a file on Nexus. They can be used to retrieve an archived file with the download API for example. The parent mod, mod version or individual version are useless, but MO2 stores them so it can try to keep track of them, and show relevant information to the user.

There is an extra field in the meta.ini that seems to indicate the type of file: nexusFileStatus. From what I can gather, its value is

• 1 for a Main file
• 3 for an Optional file
• 5 for a Miscellaneous file
• 1000 for a deleted file (or perhaps Old file, or archived file, ...) not sure

Greg is probably guessing correctly that MO2 only tracks versioning of Main files (nexusFileStatus=1), by comparing the file's version number to the parent mod's version number. MO2 is doing a best effort of guesstimating if a Main file is up to date, but it falls apart when the parent mod's version number is itself incorrect, or if there are multiple Main files with differing version numbers.

For the other types of files, MO2 doesn't track versioning, they're always "up to date". Or something like that.

Expand  

But as I mentioned, do a simple compare of the behavior with RBBPH and KPH. MO is not able to track individual files for the former, but it IS able to track file-specific versions for the latter:

The only differences between the two are that 1) the former uses Main Files, and the latter uses Misc Files. 2) the former has changelogs by file, and the latter has global changelogs. Both have global versions that differ from the file versions (to more/less extent).

So it may well be that MO is doing things differently according to nexusFileStatus, but it isn't clear or documented anywhere I can find. I'd love to have the algorithm.

[Mousetick]

  On 1/2/2023 at 6:44 PM, z929669 said:

The only differences between the two are that 1) the former uses Main Files, and the latter uses Misc Files. 2) the former has changelogs by file, and the latter has global changelogs. Both have global versions that differ from the file versions (to more/less extent).

Expand  

Your screenshot shows exactly how I think MO2 is trying to infer whether a file is "up to date":

• All KPH are "green" (up to date) even though the main KPH mod is at version 3.4. Which is why I'm speculating it's simply not keeping track of Miscellaneous files versions, it simply shows them.
• All RBBPH are "red" (outdated) because the main RBBPH mod is at version "1.1", and they are Main files.

Why are you saying MO is unable to track individual file versions for RBBPH? It shows they're at version "1.0", which is accurate:

[z929669]

  On 1/2/2023 at 6:57 PM, Mousetick said:

Your screenshot shows exactly how I think MO2 is trying to infer whether a file is "up to date":

• All KPH are "green" (up to date) even though the main KPH mod is at version 3.4. Which is why I'm speculating it's simply not keeping track of Miscellaneous files versions, it simply shows them.
• All RBBPH are "red" (outdated) because the main RBBPH mod is at version "1.1", and they are Main files.

Why are you saying MO is unable to track individual file versions for RBBPH? It shows they're at version "1.0", which is accurate:

Expand  

I misspoke. I'm saying that MO is not "checking for updates" with the 'true' result with RBBPH mods as it is for KPH. This may well be due to MO using a different method base on nexusFileStatus, but it isn't clear. Which values allow individual file tracking, and which don't (we can presume Main Files don't and all others do, but there's no way to know for sure without creating a mod page and testing this with a bunch of different files, which seems disruptive and potentially a violation of Nexus ToS).

I will definitely test it when we update the SSE files for 2.2.0. I just want to take a "best approach" without mucking things up when that time comes.

[Mousetick]

MO2 attempts to organize and make sense of what is an unstructured mess on Nexus. It mostly doesn't work for tracking updates: because one file with two versions, say an old version and a new version, is actually two separate files. The only thing that links these two separate files is their parent mod. On top of that, there is no consistency in file names or file formats from one version to another, and on top of that, the parent mod's version number is unreliable - due to MAs' complete carelessness and Nexus' lack of data structuring. In short, there is no reliable metadata to tell if a particular file has been updated - it's all guesswork.

DY's point still stands, but not for the purpose of tracking updates which is broken anyway IMHO. By merging files, the last file merged into the mod folder overwrites the previous one's metadata, and becomes the mod's metadata (meta.ini). So merging a patch into the main mod will show the patch's version number, instead of the mod version number for example. It's even worse when merging files from different mods.