Jump to content
  • 0

[WIP] Mator Smash

Mator

Asked by Mator,

 Share

Question

Mator Jarl

Mator

Posted
Posted

kWdBG.jpg

 

Mator Smash

VIDEO TUTORIAL - For v0.0.7

GamerPoets Video - For v0.4.1

 

 

Mator Smash is now available on Nexus Mods. 

Nexus Mods Mod Page

 

 

As of v1.0.0, Mator Smash has a "Quick Patch" button

As of v0.5.1, Mator Smash will copy records with "errors" in them (such as unresolved/unexpected references)

As of v0.5.0, Mator Smash has new algorithm features and is more stable.

As of v0.4.1, Mator Smash merges redundant plugins properly.

As of v0.4, Mator Smash now produces complete record prototypes

As of v0.3, Mator Smash supports Fallout 4 and Skyrim SE

As of v0.2.2, Mator Smash allows you to modify tags on plugins from within the program.

As of v0.2.1, Mator Smash has Smash Settings for Skyrim for every Bash Tag Wrye Bash offered for Oblivion.

 

Changelog:

 

  Reveal hidden contents

 

 

 

Purpose
Mator Smash is an application built on the xEdit framework developed to replace the primary functionality of Wyre Bash - the creation of patch files to combine (or merge) conflicting overrides to overcome Bethesda's "Rule of One". Wyre Bash used to provide this functionality in the days of Oblivion through the usage of bash tags, keywords that would be put in the description of the mod that would then tell Wyre Bash what types of records it should try patching. However, the current status is that Wyre Bash only provides this for a handful of record types for Skyrim, and this has been the case for quite a while now. There are several users (most notably Sharlikran) who have been working to update Wyre Bash's codebase to work with Skyrim, but it's been an uphill battle. As of now, it's not certain whether Wyre Bash will ever offer the same functionality it did in the days of Oblivion for Skyrim mods. That's where Mator Smash comes in.

 

Why is this necessary?
As per Skyrim's "Rule of One", any two mods that modify the same record but in different ways will conflict with each other in a way that can't be resolved asides from the creation of a compatibility patch. Because the number of Skyrim mods exceeds 40,000, there is no conceivable way to manually build enough compatibility patches to make every possible load order (or even a small fraction of them) work with each mod operating as intended. Mator Smash automates the most important part of the task of compatibility patch creation - the combining of conflicting edits. This is something that we've been needing for a very, very long time.

 

How does it work?
Warning: technical jargon ahead. Where Wyre Bash has separate procedures for the patching of individual record types, Mator Smash has a few generic procedures for the patching of ALL record types. This means that Mator Smash has, with very little development effort, achieved the capacity to patch more records than Wyre Bash has ever been capable of patching. This is achieved through a recursive traversal method which traverses override record structure, comparing subrecords between a master record, a source override record, and a destination override record. Upon the basis of this comparison, certain subrecords are written to the destination record while others are skipped and others are deleted from it.

 

Development Status
I began development of Mator Smash as a standalone application built on top of the xEdit API 10/18/2015. Here's the GitHub repository. Check out the design folder in the repository for various files relating to my design process. I'll be live streaming most of my development on livecoding.tv. I'll be streaming mainly on weekends 9:00AM-9:00PM PST. Because of major code reuse from merge plugins standalone, smash will be entering public beta very soon.

Mator Smash is currently available as a proof of concept script (which is slow and lacking in features/a clean user experience). You can download v0.9.4 of that script here. This script can currently correctly patch a massive number of different types of records. FAR MORE than Wyre Bash or any other currently available solution, and more than any solution that has ever existed for Skyrim or any other game.

 

Implemented Features

  • Dynamic, flexible settings: Smash settings are comprised of a tree which has information on how records/subrecords should be handled. This will allow users to control how a patch is created on a per-subrecord basis. This will make smash relevant regardless of the nature of the mods being patched, or the game the mods come from. Each node (record/subrecord) will have the following data associated with it-
    • Process: Whether or not smash should process this node and perform conflict resolution for it if a conflict is found.
    • Preserve Deletions: If a mod deletes an element inside of this node, that deletion will be preserved in the generated patch.
    • Treat as single entity: If a mod overrides any subrecord in this node, the entire node will be replaced with that mod's version of it.
    • Chaining: Linking allows you to copy multiple side-by-side elements when any one of those elements changes.
    • Override Deletions: Allows a plugin to restore deleted elements.
    • Force Value: Forces values from a particular plugin to be used, and only allows plugins which require this plugin to perform further conflict resolution on affected records.
  • Tags: As of v0.2.1, all Bash tags that were offered with Wrye Bash for Oblivion are now available for Skyrim.  Smash will automatically detect tags and apply smash settings to the plugins that have them, assuming you have smash settings that correspond to the tags.
  • Managed smashed patches: Patches you build will be tracked in the program, so you can quickly and easily rebuild them when your load order changes. You can also have multiple patches for your load order, if you feel that is desirable.
  • Multi-language support: I built a really clean, extensible, and easy to use language system when I developed Merge Plugins Standalone. Smash will use the same language system to support multiple languages out-of-the-box. As a translator, all you have to do is make a text file to create a translation and have users install it in their lang folder. The rest is handled by the program.
  • Fast and easy setting creation: Select some plugins to build a setting to specifically handle the conflicts they are capable of creating in a load order.  Toggle or set flags for multiple nodes at once.  Select similar nodes in a setting tree.  Conflict resolution has never been quite so powerful, flexible, and easy.

 

Planned Features

Smash is now pretty much feature complete.  Further development energy will be spent on porting it to a zEdit application mode, zSmash, which will include a new, streamlined workflow.

  • +1 3
  • Answers 314
  • Created
  • Last Reply

Top Posters For This Question

Popular Days

Top Posters For This Question

Popular Days

Popular Posts

GamerPoets
GamerPoets

... just throwing it out there that there's a semi insane but glad to be helpful video creator out there who will be happy to create certain things for certain topics if certain people ever want them

Mator
Mator

Mator Smash VIDEO TUTORIAL - For v0.0.7 GamerPoets Video - For v0.4.1     Mator Smash is now available on Nexus Mods.  Nexus Mods Mod Page     As of v1.0.0, Mator Smash has a "Quick Patch" button As

Mator
Mator

The ETA is largely dependent on the amount of time I can put in now that I have a full-time summer job.  It's not a situation I've been in before, so I can't really say much with certainty. Smash is a

Posted Images

314 answers to this question

Recommended Posts

  • 0
GamerPoets Jarl

GamerPoets

Posted
Posted

... just throwing it out there that there's a semi insane but glad to be helpful video creator out there who will be happy to create certain things for certain topics if certain people ever want them at certain times. Generally he needs certain things to be taught to him like he's a child because this certain person takes a while to grasp things lol but ... he really does like being helpful = ) 

(GP has graduated from having issues to full blown subscriptions)

  • +1 4
  • 0
Mator Jarl

Mator

Posted
  • Author
Posted
  On 6/16/2017 at 7:16 AM, mrsmexythebeast said:

Hey Mator, I had a quick question. Mator Smash essentially combines the functionality of a bashed patch and a merged patch (and does each one better), but how would we go about patching a mod with instructions to be included in the bashed patch but not the merged patch? Quite amateur modder here and new to Mator Smash, so I'm sorry if this is a silly question. The mod I'm talking about specifically is Scarcity. Taken from Scarcity's mod page: 

Thanks for your help and your great work on both this and Merge Plugins; they are life-savers.

I think you can just ignore that then.  The mod author probably recommended against using the merged patch because it is in many ways buggy, and forwards things in an odd fashion.  You could ask the author for more information.  Smash should handle it perfectly.  If you wanted to be extra safe you could apply the Bash.All setting to the Scarcity ESPs.

 

  On 6/17/2017 at 6:17 AM, GamerPoets said:

... just throwing it out there that there's a semi insane but glad to be helpful video creator out there who will be happy to create certain things for certain topics if certain people ever want them at certain times. Generally he needs certain things to be taught to him like he's a child because this certain person takes a while to grasp things lol but ... he really does like being helpful = ) 

(GP has graduated from having issues to full blown subscriptions)

Feel free to shoot me an email or PM whenever you feel up for it.  :)

 

I do think if we get to the point of you making a video I will need to make a Nexus Mods page for Mator Smash.

  • 0
Darth_mathias Talos

Darth_mathias

Posted
Posted
  On 6/17/2017 at 6:17 AM, GamerPoets said:

... just throwing it out there that there's a semi insane but glad to be helpful video creator out there who will be happy to create certain things for certain topics if certain people ever want them at certain times. Generally he needs certain things to be taught to him like he's a child because this certain person takes a while to grasp things lol but ... he really does like being helpful = ) 

(GP has graduated from having issues to full blown subscriptions)

Personally, I would love to see a Tutorial series on Mator Smash from you Michael as you are always my first point of call for video tutorials. Me and my wife have been nibbling around the edges of Samsh for a while she will be takig a look at it with the possibility of replacing Wrye Bash patch for our SRLE Extended LOTD guide but she frist needs to work out for herself how it works and a tutorial from you would really help with that.

  • 0
dreadflopp High King

dreadflopp

Posted
Posted

Is there a smash setting that you can set so that a plugin overwrites previous plugins even though the data of the record is identical to skyrim.esm? Example, magicPerk.esp doesn't forward changes from USLEEP and the author says it shouldn't. The smash.all setting does forward USLEEP changes per design, if USLEEP is included in the patch. I know you probaly can edit the smash-setting on all records that magicPerk.esp conflicts with but it would be easier if you didn't have to.

  • 0
twizz0r Citizen

twizz0r

Posted
Posted

Hey Mator!

 

FO4/MO2: Any advice on hunting down the source record of this error?

 

...

Removing ITPO: DLC03_miscmod_mod_RadiumRifle_Mag_SmallQuick "Radium Rifle Standard Quick Eject Magazine" [MISC:030407E8]
Removing ITPO: DLC03_miscmod_mod_RadiumRifle_Mag_LargeQuick "Radium Rifle Large Quick Eject Magazine" [MISC:030407E7]

Exception removing ITPOs: Overflow while converting variant of type (LongWord) into type (Integer)

 

i know the mod that's causing this (Frost). The smashed patch causes a CTD for FO4 before the menu. If i remove the smashed patch, FO4 runs fine. If i skip frost.esp and re-smash, it runs fine.

 

Can u/l logs if you think it would help, but i don't see anything illustrative.

 

Thanks!

  • 0
twizz0r Citizen

twizz0r

Posted
Posted
  On 6/16/2017 at 7:16 AM, mrsmexythebeast said:

Hey Mator, I had a quick question. Mator Smash essentially combines the functionality of a bashed patch and a merged patch (and does each one better), but how would we go about patching a mod with instructions to be included in the bashed patch but not the merged patch? Quite amateur modder here and new to Mator Smash, so I'm sorry if this is a silly question. The mod I'm talking about specifically is Scarcity. Taken from Scarcity's mod page: 

 

Thanks for your help and your great work on both this and Merge Plugins; they are life-savers.

MS has a smash setting for Scarcity. Use that rather than any bash tags or the Smash.all setting and you should be fine.

  • 0
austen1000 Watcher

austen1000

Posted
Posted

Greetings. I seem to be having trouble using this properly with Fallout 4. It works great with all the other games I have, but, for some reason, whenever I go to load the Fallout 4 profile up, the load order within its plugin selection list is messed up, causing some mods to not be able to load due to masters for them loading above their own masters (for example, Armorkeyword.esm loads above all the DLC esms). The load order in Wyre Bash (which I use for the BAIN installer at this point) and FO4Edit is normal, and this issue is absent from all other games I've tested.

 

Also, any chance that Morrowind/OpenMW will ever be usable with this?

  • 0
TechAngel85 Akatosh

TechAngel85

Posted
Posted

There are a few things holding me back from this fully replacing Bashed Patching and these are basically just features that WB has, which I have become accustomed to.

 

One is the WB system automatically gathers up all the leveled lists and provides this to the user for management. This is an excellent feature and one of the main reasons I use Bashed Patches. Smash could provide the same functionality as an option for the user. If they are just making a single Smashed Patched for their mod list (as I assume most users would do), it would be nice to have the option of including merged leveled lists into the Smashed Patched just as WB does; without having to worry about anything extra.

 

The second feature holding me back is the ability to include game tweaks right into my Bashed Patch. This reduces plugin count and/or unnecessary console commands. It's a time saving feature. I've also always desired this feature to be expanded upon so that all the game settings could be included. This way users wouldn't have to worry about setting them up elsewhere. A few checkboxes and you're done.

 

Lastly, from what I can tell and from first impressions, Smash currently requires knowledge of the Bash Tag system to get its full functionality. This is a fading...dying?...system which few are fluent with anymore. Finding well written documentation for the tags, by itself, can be an en-devour. Then learning how to apply appropriate tags to mods is another. I think this is evident in that we have become accustomed to WB auto-selecting the mods it wants to merge and most users just going with it. Smash doesn't have this functionality, from what I can tell.

 

It could be possible to do an initial scan of the mods to collect some of this information and provide a couple of presets to merge a few basic plugins with basic edits, as WB does. Then provide the option to merge leveled lists from all mods and include game tweaks. If Smash could get to that point and do a better job at patching than WB, then WB would fall out of use for its patching function due to Smash meeting its goal of Smashing being better than Bashing. ^_^

 

These are just my observations during my short time with the program.

  • 0
kabepo Guard

kabepo

Posted
Posted
  On 1/3/2018 at 1:41 AM, TechAngel85 said:

Lastly, from what I can tell and from first impressions, Smash currently requires knowledge of the Bash Tag system to get its full functionality. This is a fading...dying?...system which few are fluent with anymore. Finding well written documentation for the tags, by itself, can be an en-devour. Then learning how to apply appropriate tags to mods is another. I think this is evident in that we have become accustomed to WB auto-selecting the mods it wants to merge and most users just going with it. Smash doesn't have this functionality, from what I can tell.

 

It could be possible to do an initial scan of the mods to collect some of this information and provide a couple of presets to merge a few basic plugins with basic edits, as WB does. Then provide the option to merge leveled lists from all mods and include game tweaks. If Smash could get to that point and do a better job at patching than WB, then WB would fall out of use for its patching function due to Smash meeting its goal of Smashing being better than Bashing. ^_^

 

 

My understanding is that the Smash functionality is completely independent to the Bash Tag system. Smash can understand and emulate Bash Tags, but you do not need to use Bash Tags at all, I don't. I use the "Smash.All" setting and clone it and edit it so it patches records the way I want them patched.

 

Smash does have functionality for auto-selecting the mods it wants to merge and merging plugins, but it does not popup a list to notify you and ask for confirmation, unlike Wrye Bash.

 

I've been using Mator Smash instead of Wrye Bash to produce patches for STEP Core and Extended, and find it more powerful and creates a much better patch for my load order.

  • 0
alt3rn1ty General

alt3rn1ty

Posted
Posted (edited)
  On 1/3/2018 at 1:41 AM, TechAngel85 said:

TLastly, from what I can tell and from first impressions, Smash currently requires knowledge of the Bash Tag system to get its full functionality. This is a fading...dying?...system which few are fluent with anymore.

https://wrye-bash.github.io/docs/Wrye%20Bash%20Advanced%20Readme.html#patch-tags

 

Utumno is just about drawing the refactoring of all of Wrye Bash code to a close, when 307 is out of beta, one of the next priorities is getting Patchers for all supported games in a better state, and along with that will come Bash Tags for the same.

 

The snag is its a huge project and only one expert for such a long time working on his own. I have tried Python occasionally but its beyond my abilities and time to invest

 

Anyway the biggest support for Bash Tags is through submissions from mod authors to the LOOT masterlist (as it used to be for the BOSS masterlist), which in accordance with the documentation linked above is utilized by Wrye Bash (and occasionally rebased on the latest when a release of Wrye Bash goes live). That always made sense because ITM and UDR reports were usually submitted to the masterlist too .. Dont see much of that going on these days because the much larger communities dont engage as well as they used to.

Modding communities these days are more of a candy shop than a work shop, but things like collating Bash Tags etc needs a lot of people invloved to keep up with the massive explosion in mod production since Oblivion.

 

Reference dying .. Its rather still developing for the newer games and its taking a long time. But the other option of having Mod Authors use Wrye Bash just to edit the mods plugin description (top right box) then click save easily and add a few bash tags that it needs .. Probably such is now not so widely known because MO became so prevalent as the go to manager for so long, along with NMM and its one click mod management stealing a lot of potential people who may have learned the intricacies of modding the games at a greater level of understanding <shrugs>

 

UegiwVn.png

Edited by alt3rn1ty
  • 0
Piranhaha Citizen

Piranhaha

Posted
Posted (edited)

I used Mator Smash to make individualized profiles for about 80 different mods that showed conflicting records in SSEedit. When I tried to launch the game with the patch enabled, the main menu window comes up and starts to load (e.g. the point where, if you have a large data folder, the New, Continue, etc buttons aren't yet visible but you see the dragon logo). However, before the menu buttons become visible it CTDs with no error message. By selectively removing plugins from the smashed patch I tracked the cause down to three esps, any of which cause the issue when included in the patch (even without the other two). These ESPs are:

 

  • Undeath
  • Animated Weapon Enchants
  • My Audio Overhaul Skyrim Patches merged patch

I started trying to debug using my AOS patches as the test case and encountered the following:

  • When using a new smash setting on AOS patches in which none of the fields are toggled, the patch still causes a CTD
  • When using the Bash.Audio setting that I had cloned as Smash.AOS_Patches, the smashed patch actually contains 7 fewer records than when I exclude AOS Patches from the smashed patch.
  • I wrote a quick SSEedit script to dump all FormIDs touched by the smashed patches, and found the 7 FormIDs that are in the working smashed patch but are absent in the non-working smashed patch. They are all WEAP entries.
  • In 4 of these 7 entries, the entire entry was identical to the previous entry in the load order (AOS_Patches merged).
  • In the other 3, the entry had inherited and overwritten the INAM record from Skyrim.esm in preference to the value in AOS_Patches merged (this is because Bash.Audio doesn't include WEAP INAM records, but including them and rebuilding the patch doesn't resolve the crashing).
  • When I deleted these 7 WEAP entries from the working (AOS_Patches-excluded) smashed patch, resulting in a smashed plugin that appears (in SSEedit) to be identical to the CTD-inducing version, the game is still able to launch. Despite having the same amount of records as the non-working patch and also being the same filesize, the CRC32 checksum of this edited file is different from the non-working file.
  • I can make a second smashed patch in which AOS_Patches and the non-working smashed patch 1 are smashed together, and the game is able to launch normally with smashed patch 2 activated. However this isn't really useful because smashed patch 1 is the conflict winner in the load order and as such overwrites any desired changes from AOS_Patches. I suppose I could drag AOS_Patches below it but having to do the same with Undeath would significantly interfere with my load order.

 

It seems like there's something aside from records that's being edited when I include AOS_Patches (and presumably Animated Weapon Enchants and Undeath). I'm kind of at the limit of my knowhow as to how to debug this though. All of my plugins are form44 in case that matters; the AOS_Patches is generated by Merge Plugins standalone.

 

Anyone have any ideas for troubleshooting?

 

Edit: I figured it might help if I upload the smashed patches. 

 

1: Working version of smashed patch (AOS_Patches.esp excluded): https://mega.nz/#!IqYhxBgD!0lYmHodc3tvHZggCcc4ydh_kgL_d1o_PsXOSLECoWPk

 

2: Non-working version of smashed patch (AOS_Patches.esp included): https://mega.nz/#!dqgH2YzR!5q77PLyW6phtprV03qz8tkulXQrkTN98JsbozVib--A

 

3: Working version of smashed patch (AOS_Patches.esp excluded) where I removed the 7 extra records in SSEedit such that its record list matches that of the non-working version: https://mega.nz/#!ImwSUZLK!gyoB6ThGPEd6Id5vK8cmeXqRORdAqEXLwyUNLp6sHxQ

Edited by Piranhaha
  • 0
Piranhaha Citizen

Piranhaha

Posted
Posted (edited)

Edit: I figured it out! Please see below - this may be something that should be fixed in Mator Smash when you have time.

 

To follow up on my previous topic, I noticed that in the file header of the smashed patches, even though Number of Records is identical between my culled working patch and the nonfunctional one, the former has a Data Size of 4801 while the latter is 4842. I modified my script (code below) to report on the data size for each record within the patches. The script returns a blank for most of the values (for reasons that I don't understand) but I compared the ones that were returned as numbers and found two records that were each shifted by 10 (bytes?), accounting for 20 of the 41 (byte?) size difference. Both records were MGEF with FormIDs 000F3F10 and 000F392F. For example, the latter has a data size of 1114 in the working smashed patch, but 1124 in the crashing patch. 

 

I tracked down the size difference to the KSIZ entry. In Skyrim.esm this MGEF comes with a keyword entry which AOS preserves, but Ordinator then removes it. AOS patches then brings it back, but since Bash.Sound doesn't include keyword records, my smashed patch doesn't forward them: https://imgur.com/Nk6YF1p

 

As you can see, what ends up happening is that the smashed patch carries through Ordinator's keyword deletion. The difference is that when I include AOS Patches using Bash.Sound, it populates the KSIZ - Keyword Count field with "0" whereas the smash without AOS Patches leaves it blank. I don't know what's going on under the hood to cause that behavior, of course. What I do know now is that "0" does not seem to be an acceptable parameter in this position. When I manually deleted this KSIZ entry for both of the above FormIDs in the crashing smashed patch, the game was able to load successfully (which I guess makes it not a crashing smashed patch anymore). So I guess perhaps there should be a validation function at the end of patch generation which looks for KSIZ values of zero and replaces them with null?

 

Here's the script that I used to dump FormIDs and their corresponding data sizes. Sorry for the clunkiness; I'm a chemist who learned Pascal this evening explicitly for this purpose :). If anyone could let me know why most of my data sizes are being recorded as blank, I'd appreciate it (I still want to investigate why there's a remaining difference of 21 (bytes?) between my two smashed patches in case it could lead to errors down the road). The two FormIDs that I mentioned above are the only ones where Diffchecker found a difference in size, so the other difference(s) must be buried in the blanks.

 

Thanks!

{  Prints the FULL name of selected records to the log. } 

unit MyDebugScript; // This is the unit name that will contain all the script functions

uses mteFunctions; // import Mator's functions

// global variables
var  slFormIDs: TStringList; 
var slDataSizes: TStringList;
var  f: IInterface;
var  g: IInterface;
//

// Called when the script starts
function Initialize: integer; 
begin
slFormIDs := TStringList.Create;
slDataSizes := TStringList.Create;
end;

// Called for each selected record in the TES5Edit tree
// If an entire plugin is selected then all records in the plugin will be processed

function Process(e : IInterface) : integer;
var test : string;
var test2: string;
var recordHeader: IInterface;
begin  	
	test := GetLoadOrderFormID(e); //this is apparently considered better practice than just FormID(e)	
	recordHeader := ElementByIndex(e, 0);
	test2 := geev(recordHeader, 'Data Size');
	slFormIDs.Add(IntToHex(test,8)); // converts the formID to the value seen in SSEEdit.
	slDataSizes.Add(test2);
end; 


// Called after the script has finished processing every record
function Finalize : integer;
var
  i: integer;
  s: string;
  slFormIDs_size: integer; // test to see if I can get stringlist count
  filename: string;
begin
	AddMessage('element count:');
	slFormIDs_size := slFormIDs.Count;
	AddMessage(IntToStr(slFormIDs_size) + ' records found');
	AddMessage('FormIDs:');
	for i := 0 to slFormIDs_size - 1 do begin
		s:= slFormIDs[i];
		//AddMessage(s);
	end;
	
	filename := ProgramPath + 'Edit Scripts\FormIDs.txt';
	AddMessage('Saving record list to ' + filename);
	slFormIDs.SaveToFile(filename);
	slFormIDs.Free;

	
	filename := ProgramPath + 'Edit Scripts\Data_Sizes.txt';
	AddMessage('Saving record list to ' + filename);
	slDataSizes.SaveToFile(filename);
	slDataSizes.Free;
	
	
	AddMessage('end script');
end;
end.
Edited by Piranhaha
 Share
Go to question listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

By using this site, you agree to our Guidelines, Privacy Policy, and Terms of Use.

  I accept