-
Posts
350 -
Joined
-
Last visited
-
Days Won
13
Everything posted by fireundubh
-
Improving Conflict Detection by Removing Duplicates
fireundubh replied to fireundubh's question in Mod Organizer Support
You're speaking as a pack user, which makes sense given the venue, but most people don't use a pack of any kind. I doubt any of the pack creators have gone to the lengths that I described above. I'd bet that pack creators "resolve" conflicts by a process artists generally refer to as "eyeballing." If you're fine with "good enough," great! I'm not. I don't use any packs. As you can see from the scrollbars in my screenshot of MO, I have around 200 mods installed, including some rather large overhauls. MO reports that 70 of those 200 mods conflict, which probably equals several thousand files. To get everything working together, I need to be really certain about whether the reported conflicts are serious. -
Improving Conflict Detection by Removing Duplicates
fireundubh replied to fireundubh's question in Mod Organizer Support
This is not about modders... This is about users. Do you not understand the problem? See below. We've all been under that impression, certainly, but it turns out that's just not true. The whole point of using MO is to install mods without screwing around in the game installation directory. Some mods that would otherwise need to be installed in a certain order can be simply reordered in MO. That's what the Priority column is showing you. You use the conflict flags in the Flags column, and the Conflicts tab, to determine how to reorder mods. But since 80-90% of the reported conflicts are duplicates, you cannot rely on anything that MO tells you about conflicts. Therefore, MO does not provide you with actionable information that actually helps you determine how to reorder mods. Therefore, the whole point of using MO—for everyone!—suffers from this oversight.Screenshots: You can't rely on any of the conflict flags in the Flags column. The flags are updated based on the same data available in the Conflicts tab. The Conflicts tab shows you all "conflicts," which includes duplicate files that don't matter to anyone. This window is telling you one thing and one thing only: "there might be conflicts between this mod and others. Maybe." -
Improving Conflict Detection by Removing Duplicates
fireundubh replied to fireundubh's question in Mod Organizer Support
I'm strictly speaking of conflict detection which, practically speaking, is broken when 80-90% of the reported conflicts are false positives. The first step is recognizing you have a problem, but you still need to know what that problem is and how to solve that problem; MO doesn't help you at all in that regard. MO's ability to detect conflicts is a bit like this math problem: "There are two trains running in opposite directions. One is moving at 200 mph, the other is moving at 300 mph, and there are several people standing somewhere on both tracks. When will the two trains be parallel?" I was going to suggest an "export report" button for the Conflicts tab, so that you could save the file listing (with junk data) to a CSV file, but I'd much prefer if MO's conflict detection features made an effort to remove junk data from the results. Sadly, that's not going to happen, apparently. "File Size," "Last Modified Date," and "Resolution" columns would be great for that report, too. You'd still need to generate MD5 hashes for each file in the CSV. MO couldn't do that on export because that might involving extracting BSAs, but having a premade "Conflicts and Duplicates" spreadsheet would eliminate at least nine of the steps above and make writing an automated conflict detection tool much easier. -
Improving Conflict Detection by Removing Duplicates
fireundubh replied to fireundubh's question in Mod Organizer Support
edit: Originally posted in Guides & Tutorials. Mod Organizer's conflict detection features are, for all practical purposes, BROKEN. The conflict flags don't indicate actual conflicts; they merely suggest the possibility of conflicts. The Conflicts tab is just as useful, spitting out non-actionable data at the file level. You can assume that 80-90% of the reported file conflicts are false positives. Tannin has stated that he will not fix this problem. Therefore, it is up to you to put in the effort to identify conflicts between your mods. In order to identify real conflicts between your mods: Download and install the File Checksum Integrity Verifier from Microsoft to a system folder or add the location of FCIV to your system PATH variable.Determine which mods you want to compare for conflicts.Extract the respective BSAs, if any, to their respective mod folders but first account for loose files. You want loose files to overwrite anything extracted from the BSAs.Generate an MD5 hash index for each mod. On a command line, run: fciv -r -md5 %directory% > %modname%.txt (where %directory% is the path to a mod's folder and %modname% is the name of that mod; if there are spaces in the path or name, enclose the path or name with quotes)Repeat that command for each mod you want to compare. (Note: You can generate an index for the entire mods folder but that process is more involved.)Open each %modname%.txt in a text editor.You'll see content that looks like this: 450eca558d9c60bb31f37e18a9836197 locations - heljarchen farm\HeljarchenFarm.esp ecf16f79575439b08f3b8a7cdabd558b locations - heljarchen farm\meshes\aars\bookset\bookset01.nif 28efe0958121f0ecc0022057084e48fe locations - heljarchen farm\meshes\aars\bookset\booktower.nif 709e9ff1f9e0d58b026bff222b90837c locations - heljarchen farm\meshes\aars\bookset\booktowermini.nif 8599dc9fc4e7472f3456682166dfacfe locations - heljarchen farm\meshes\aars\foodcontainer\basket03apple.nifAdd tabs between the MD5 hash and the file path. Add tabs after the parent folder in the path (e.g., "locations - heljarchen farm").Copy the file listing to an Excel spreadsheet (or open source equivalent.)Add headings to the spreadsheet: MD5 Hash, Mod, File Path.Apply conditional formatting to the File Path column: Conditional Formatting > Highlight Cells Rules > Duplicate Values (Light Red Fill)Apply a color filter to the File Path column: Filter > File Path > Filter By Color (Light Red Fill)Copy the filtered results to a new spreadsheet named "Conflicts and Duplicates."On the new spreadsheet, remove duplicates: Remove Duplicates > Check: MD5 Hash and File PathOn the new spreadsheet, apply a color filter to the File Path column again: Filter > File Path > Filter By Color (Light Red Fill)Remove the rows where the File Path equals "meta.ini." (Mod Organizer users only)You now have a list of actual conflicts! Examples: daf3c0244aa0075791ae06856e9ff00b locations - morskom estate\meshes\food\cratemead01.nif e890dfa2e376509fdd3c12d58a708716 locations - heljarchen farm\meshes\food\cratemead01.nif b516833efee0393a75a64e1d73034791 locations - windstad mine\meshes\treasures\trjewelbox01.nif c134d0af83c7ac607322a51a8e64f860 locations - morskom estate\meshes\treasures\trjewelbox01.nif 58b5af8918a0a95ecbeb36d98b339e10 locations - morskom estate\textures\aars\openbook\largebookpaper04.dds fb1e370aed8ae94e47642afef8b89a24 locations - immersive college of winterhold\textures\aars\openbook\largebookpaper04.ddsNote that some of these conflicts might just be the difference between an optimized texture and an unoptimized texture. You'll need to compare the files individually, or write a script that does a file size, file modified date, and/or image resolution comparison on every file in the list. -
Improving Conflict Detection by Removing Duplicates
fireundubh replied to fireundubh's question in Mod Organizer Support
A duplicate is a conflict in name only. Duplicates have no impact on the game whatsoever. When 80-90% of the so-called conflicts have no impact on the game, MO offers nothing more than a placebo. "Mod A conflicts with Mod B [in some way]" is not useful information. How can you even say that? You're a software engineer. What if I told you that "Mod Organizer doesn't work on Windows 9"? You'd ask, "What do you mean by 'doesn't work'?" The information that MO provides re: conflicts is a good first step but that's all that it is. No, that files can change is irrelevant to the user at that time they're sorting out conflicts. Mods do not miraculously change without user intervention. When a user adds or updates a mod, all the user would need to do is rebuild the index, assuming the user wants to identify and resolve conflicts. Anyone who has played Skyrim with mods knows that they have to run all sorts of programs after every single change, from LOOT to Wrye Bash to FNIS to any growing number of SkyProc patchers and so on. There's a cost to modding—time spent resolving conflicts—so real conflict detection that actually works is priceless. That's super important information! To anyone looking at their set of mods for conflicts, the state of their set of mods *right now* is what matters. You're the MO expert. You know best about implementation. I'm just speculating on how a solution might be found. 1. Don't write an index that gets corrupted! ;)2. Notify users when they need to rebuild the index. Well, BSAs don't give you a whole lot of information about the files within them, so unless there's a way to uniquely identify those files and there might be, there isn't much of a choice.Maybe the solution is improving MO's implementation of conflict detection, or maybe the solution is a separate app that runs through MO.In any case, it'd be nice if you didn't think the worst of all of my suggestions. -
Improving Conflict Detection by Removing Duplicates
fireundubh replied to fireundubh's question in Mod Organizer Support
Currently, based on the above test, 80-90% of the conflicts reported are duplicates, which severely limits the usefulness of MO's conflict detection system. Since BSAs pose a problem for hash generation, I'd just calculate hashes for files belonging to non-BSA mods when the user clicks a "build index" button. Save the index as a blob and let the user know when the index needs to be rebuilt. I'd also store the file modified date, file size, and image resolution in this index. MO could use that data to tell you whether a conflicting asset is newer or older, larger or smaller, or a higher or lower resolution, in addition to improving the accuracy of conflict detection overall. -
Which of Azirok's patches need to be updated? With Azirok's long absence, I think making patches for Requiem needs to be a community project.
-
I believe my experience with SMC was a) installing every mod supported by SMC and b) watching Skyrim fail to load. So that was fun. I don't remember if I fixed that, but the main problem with SMC is that you can't check with MO if any mods need to be updated.
-
Improving Conflict Detection by Removing Duplicates
fireundubh posted a question in Mod Organizer Support
I have installed a number of location mods that appear to have conflicting assets. Heljarchen FarmImmersive College of WinterholdMorskom EstateVjarkell CastleWindstad MineI extracted each BSA and generated MD5 hashes for every file in each directory. I imported the results into a spreadsheet and confirmed that the totals below match Mod Organizer's results. Heljarchen Farm has 74 conflicts with the other mods.Immersive College of Winterhold has 49 conflicts with the other mods.Morskom Estate has 151 conflicts with the other mods.Vjarkell Castle has 2 conflicts with the other mods.Windstad Mine has 130 conflicts with the other mods.Mod Organizer finds a total of 406 conflicts between these mods but most of these conflicts aren't meaningful. When I removed the duplicates and filtered for actual conflicts (same file names, same file paths, different MD5 hashes), there was a dramatic difference in the number of conflicts. Heljarchen Farm has 6 actual conflicts with the other mods.Immersive College of Winterhold has 14 actual conflicts with the other mods.Morskom Estate has 16 actual conflicts with the other mods.Vjarkell Castle has 2 actual conflicts with the other mods.Windstad Mine has 10 actual conflicts with the other mods.Only 48 (24 vs. 24) of the 406 conflicts reported by Mod Organizer are actual conflicts that could impact the player in the game. Duplicates should be removed from conflict detection and the Conflicts tab. -
I would like to create my own categories, like "SKSE Plugins" and "My Mods." edit: Hey, you can add custom categories already! I would like to be able to create my own tags and manage mods with tags, like "has Requiem patch" and "cleaned with TES5Edit." I would like to define unique row background colors for categories and tags. Tag colors take precedence. I would like to add any number of horizontal separators with/without centered section labels to the mod list. Would anyone else like these features?
-
Skyrim Redone Guide?
fireundubh replied to fireundubh's topic in General Skyrim LE Discussion & Support
Ugh. SkyRe looks horrible to try to install and get working with other mods. Requiem is so much easier... at least when Azirok isn't MIA. -
Is there a SkyRe guide? Installation with ReProccer and compatibility patches seems a little... hard.
-
Wow. You never saw Highlander? You've seen Star Wars, right?These are not the droids you're looking for.What are you going to say now? That I just don't want to show you the droids? Seriously?
-
There can be only one!
-
That's not synonymous with recommending against bashed patches. They are simple! I'm right and you're wrong, except if I admit I'm wrong and that wouldn't necessarily mean you're right.
-
@Smile44: Stop telling people to not use a bashed patch. You're wrong. Azirok has never recommended against bashed patches. However, he has recommended deleveling the leveled lists merged in the bashed patch. To everyone else, bashed patches that merge the leveled lists are always necessary if you have ANY mods that add items, spells, or NPCs to the game world. Just because you have some number of Azirok's patches doesn't mean you can forgo the need to merge the leveled lists. I thought we already went over this...
-
ImageMagick: Legacy DDS Support
fireundubh replied to fireundubh's question in Other Utilities Support
And the patch is in. When ImageMagick 6.8.9-8 is released, you'll be able to use ImageMagick to do everything that DDSopt does and more. -
I was going to ask matortheeternal to make this script, then I thought I'd do it myself, then I became (quickly) annoyed by Pascal UI programming, and then I saw this. So happy this isn't my project.
- 6 replies
-
- SKYRIMLE
- bashed patch
-
(and 3 more)
Tagged with:
-
For those of you who have used ImageMagick to perform optimizations on your textures, you should be aware that ImageMagick does not support legacy DDS files. I discovered this issue last night and reported the problem to the developers. One of the developers said he would add legacy DDS support to his todo list. Thread: Legacy DDS Support
-
Bashed Patch does more harm than good?
fireundubh replied to beej175560's question in Wrye Bash Support
Vision? What are you talking about? A "vision" is what you want something to be. I don't have a vision for the bashed patch. I have an explanation of how the bashed patch works because the bashed patch is identical to every other plugin and that explanation never changes, but nowhere did I ever suggest, hint, or propose a vision for the bashed patch. Don't reframe the discussion in a weird way. -
Bashed Patch does more harm than good?
fireundubh replied to beej175560's question in Wrye Bash Support
This wouldn't be an issue for Requiem players who delevel the bashed patch's merged leveled lists. ;)Where can I get that .esp? I want to take a look. See! :) -
Bashed Patch does more harm than good?
fireundubh replied to beej175560's question in Wrye Bash Support
Never heard of it. Nobody has said any different... Or you can write a guide, and add a note to the STEP instructions where you tell readers to Import Names/Stats so you can avoid OPs like this one. They'd agree. You can't really split your focus as a programmer.You can put a little effort into a lot of projects, but one project will take up the majority of your time. That one project is not Wrye Bash for anyone. -
Bashed Patch does more harm than good?
fireundubh replied to beej175560's question in Wrye Bash Support
The term "winning override" is defined as the last overriding record in the load order. You've said twice now that CBash copies the winning override. We agree. Stop arguing with yourself. The impact is much less significant since, presumably, you've installed mods whose items you actually want distributed and these mods are ordered properly in your load order. Leveled lists are merged from bottom to top, so you end up with only the winning overrides. In the event that you want to switch things around, you can use TES5Edit to customize the leveled lists and, fortunately, you don't have to screw around with hundreds of records in all but the rarest of circumstances. Hmm, I guess I was misremembering Import Names/Stats working on every mod. In any case, if you have only a few mods bash tagged early in your load order, using Import Names/Stats will revert all of the changes made in between there and the patch. For example: Importing Names/Stats from only these source mods in a huge load order would be bad. Oh, yeah, some new life was breathed into Wrye Bash in the last two months but there were 100 commits in 2013, which is less than 0.3 commits per day. Not exactly "active" development. -
Bashed Patch does more harm than good?
fireundubh replied to beej175560's question in Wrye Bash Support
When CBash takes the winning override, determines what changes to make, modifies the winning override, and copies the winning override into the bashed patch, CBash is still copying an entire record (i.e., the winning override.) If CBash copied only the name or the stats into the plugin, the file would be corrupted. You can't separate names and stats from records in plugins. You can do it in memory but who cares? Users are concerned with the end results. You're assuming that every plugin is bash tagged though. I'm not. If you use Import Names and Import Stats without bash tags and you're surprised that the names and stats are wrong in the game, you're probably not going to guess "user error." You're just going to blame bashed patches. See the OP. Wrye Bash for Skyrim is not such a patcher. It is defunct and no longer actively maintained. The main developers vanished and, apparently, there is a grand total of two people, generously speaking, in the Skyrim community who know Python and none are willing to pick up the pieces. There are two best practices here:Bash tag every plugin for use with Import Names and Import Stats.-- OR --Leave Import Names and Import Stats unchecked; and create a patch with TES5Edit. Either way, you're not going to get out from doing more work. -
Bashed Patch does more harm than good?
fireundubh replied to beej175560's question in Wrye Bash Support
I explained why most people shouldn't use Import Names and Import Stats in my edited post. If you're knowledgeable enough to add the correct bash tags to every plugin in your load order though, you can ignore that advice. On the other hand, if you're knowledgeable enough to add the correct bash tags to every plugin in your load order, you don't need a guide in the first place. If anyone wants to know more about bash tags, read this. The documentation also explains Import Names and Import Stats.