Jump to content
  • 0
FullOfChocolate

Looking for documentation behind the concepts of xEdit.

Question

I'm struggling with the TES5Edit manual because of my lack of understanding of the key concepts: Records, FormIDs, filenodes, references to base and non-base objects. Oh boy.

I'm getting cranky with these modern Wiki things and... video tutorials and whatnots. They're like jiggsaw puzzles to me, turned upside down. Need to get to the source. I read manuals for fun and cried a little when Mozilla stopped shipping a proper help file with Firefox. And if that was before you were even born... well, you can just get out of my lawn. And please let the dog bite you on your way out.

 

Is there an official, properly constructed "modder's manual" lurking somewhere that the creators of xEdit might have used to build their program? And what resources did you guys use to master this stuff. I can't believe it could've been the Creation Kit Wiki. That, with all due respect, is a mess. Or is the current modding environment such that competence in the field is necessarily limited by the lack of proper documentation.

 

Thank you.

Share this post


Link to post
Share on other sites

8 answers to this question

Recommended Posts

  • 0

The backbone of TES5Edit is not something the creators of TES5Edit came up with themselves, it's something that's inherent to how Bethesda encodes their files.  TES5Edit was largely reverse engineered from Bethesda files and tools, and has been referred to by the developers as the "Skyrim Plugin Decoding Project".

 

Records, formIDs, filenodes, and references also weren't created by Bethesda.  They trace their roots back to basic data storage concepts and databases.  In the context of databases you'll see the word record (or row) used very often.

 

The Bethesda plugin files themselves are comprised of a bunch of packed records.  What packed means is hard to explain from a technical standpoint, but the result of packing the records (or arrays, depending on the context) is an increase in space and time efficiency.  Like records in most databases, these records have unique IDs.  These IDs are essentially just unsigned integers, and are equivalent to DWords in 32-bit implementations.  That is to say, they're a data type that takes up 32 bits of space, which gives them a maximum value of 232 or 4,294,967,296 or 168 or $FFFFFFFF (all of these numbers are equivalent).  You'll recognize the final representation of this number as the maximum FormID possible in any Bethesda game.  FormIDs are just hexadecimal representations of the ID associated with a record (or form).

 

Form is roughly synonymous with record.  More technically speaking, you could consider a Form to be a type of record (in the database sense) with certain header information as defined by bethesda (record flags, record signature, formID among those).

 

Your reference to FileNodes seems out of context.  As a TES5Edit user I can imagine no reason why you'd be concerned with that.  If you can clarify what you mean maybe I can help.

 

Regarding references.  A reference is just what it sounds like it is - a reference.  It's like a pointer in programming.  You basically have an address which tells you where another object is so you can make relationships between various records.  In this case, the address is the record's FormID.  The difference between base and non-base records is more esoteric.  To understand this we have to delve into the concept of a load order.

 

So, from what we've said so far, we've established there are records, and that we can have up to about 4.3 billion records in the game (barring memory limitations, of course).  The thing is, we might want to separate our records into logical units (or for users to be able to create new logical units with records in them) in the future.  For Updates, DLC, or Mods.  This is where the notion of a file and a load order prefix comes into play.  If we have more than one file now, we have to make sure that all of the content in those files lines up and that cross-file references will work.  This requires us to be absolutely certain that a FormID isn't taken already.  If each file just arbitrarily assigned FormIDs in the 4.3 billion record space, there'd be no guarantee that File A and File B wouldn't try to make completely different records at the same FormID and thus cause serious issues to occur.  Thus we reserve the first two hexadecimal digits (a space of $FF = 256 = one byte) for an affix that will be assigned to files based on the order that they are loaded into the game.  This makes certain that no two files will try to create records at the same FormID when the game runs, as the Record's FormIDs will be allocated at execution time to prevent conflicts.  This also allows us to set a priority order for changes, so two files that change the same thing can be ordered so one of them wins out over the other.

 

Ok, so we have a load order now.  The next question is why do we have override/injected records (I think that's your question).  While we allocated our FormIDs to complete different spaces to prevent unwanted conflicts, in so doing we made it so patching is impossible.  However, it is extremely valuable to implement patching in a way that doesn't require modifying the original files, as it makes installation and management much easier.  This is where master files, override records, and injected records come into play.  An override record is a record that has been allocated outside of the file's FormID space and that collides with another existing record's FormID.  That is to say, records in a file are all initially allocated to FormIDs in the range $xx000001-$xxFFFFFF ($xx being the number of masters the file has).  These are defined as the FormIDs IN that file.  NOTE that these FormIDs will be mapped to $yy000001-$yyFFFFFF when the file is loaded into the game, with $yy corresponding to the file's load order position.

 

In contrast, FormIDs at $00000001-$00FFFFFF would be considered as referring to FormIDs in the first "master" of the file.  So if I have a file "Test.esp" which has Skyrim.esm as it's master, records that are part of Test.esp will be in the $00000001-$00FFFFFF range and records that are being put into Skyrim.esm are in the $00000001-$00FFFFFF range.  If any record already exists in Skyrim.esm at the specified FormID it will be "overridden" by the version in Test.esp, else what happens is what's called "injecting records".  Similar to how FormIDs in the file get mapped based on the load order position of the file, FormIDs that refer to a master get mapped to the load order of the master.  So $00000001 in Test.esp gets mapped to $00000001 in Skyrim.esm because Skyrim.esm is at load order position 0 (thus not a particularly good example).

 

I'd imagine you're probably pretty confused now, but there's the information dump you seemed to be asking for. 

 

 

Cheerio.

-Mator

Edited by Mator
  • Upvote 2

Share this post


Link to post
Share on other sites
  • 0

A lot of this is also learned from the Bethesda forums

 

Historically a lot of the game specific terminology comes from the official modding tools topics on BGS forums, like the Construction Set for Morrowind and Oblivion, and Creation Kit for Skyrim .. in the sub forums here https://forums.bethsoft.com/forum/13-the-elder-scrolls/

 

There are also some sticky topics which are hugely informative no matter how old they are, because a lot of the subject matter still applies, for example a lot of the material in Oblivion Mods forum, the sticky "Oblivion Mods FAQ" https://forums.bethsoft.com/topic/449239-oblivion-mods-faq/ 

 

.. is still useful for anyone starting out modding Skyrim

 

To catch up on terminology that a lot of us have become familiar with over time ( for me personally I have been frequenting the official forums since 2005, and other support forums for Morrowind long before joining the official support forum ), will probably take a quite a bit of time, but you will pick it up with experience.

Edited by alt3rn1ty

Share this post


Link to post
Share on other sites
  • 0

Small fix to file FormIDs (numbers that are stored in plugin):

 

".. records that are part of Test.esp will be in the $01000001-$FFFFFFFF range and records that are being put into Skyrim.esm are in the $00000001-$01FFFFFF range"

 

First number is the index of master file in TES4 plugin header starting from 00 (overrides), any numbers above the index of the last master are new records added by a plugin. If first number of FormID is matching the index of some master file but record with that FormID doesn't exist there, it is injected record.

Share this post


Link to post
Share on other sites
  • 0

Beware some old terminology though which may have been superceeded ( new knowledge / methods / game engine )

 

One example is Archiveinvalidation - Depending on the game there are various methods and recomendations - Oblivion for example has three methods, but the best of those would be BSA Redirection ( as used by Wrye Bash ) for that game ..

 

.. Whereas for Skyrim, its not longer a matter for concern. Same as time stamps on files which at one point was important .. now load order is all you need to get correct. See BSAs and You sticky https://forums.bethsoft.com/topic/1354395-update-bsas-and-you/

 

And tools were very well developed for Oblivion, but for Skyrim they are still playing catch-up ( apart from TES5Edit which is superb )

 

No one site or topic covers everything you need - So feel free to contribute to the community and make one :D

Share this post


Link to post
Share on other sites
  • 0

Small fix to file FormIDs (numbers that are stored in plugin):

 

".. records that are part of Test.esp will be in the $01000001-$FFFFFFFF range and records that are being put into Skyrim.esm are in the $00000001-$01FFFFFF range"

 

First number is the index of master file in TES4 plugin header starting from 00 (overrides), any numbers above the index of the last master are new records added by a plugin. If first number of FormID is matching the index of some master file but record with that FormID doesn't exist there, it is injected record.

 

Thanks for that correction.  I'm amazed I got everything else right.  :o

Share this post


Link to post
Share on other sites
  • 0

STEP is currently in discussion about this very subject and we will endeavour to provide a guide that covers as much as possible for all the xEdit variants in the near future. (participation in this project is not restricted to STEP admin though and all are encouraged to join in if they can)

Share this post


Link to post
Share on other sites
  • 0

If you want a deep dive into the TES5 mod file format, you can find it documented at Tes5Mod: Mod File Format. This isn't for the faint of heart, and I mention it only because  I personally get a much better understanding of things by poking around the raw data to learn how to take it apart and put it back together. Unfortunately, some of the records (notably NAVM) haven't been documented anywhere to my knowledge and some of the information may still be a bit confusing. I've edited some of it to hopefully try to clarify what I found when breaking it apart.

 

I still need to finish going through this to finish correlating the raw data with what's shown in the Creation Kit, but I need to find some time to get back to it.

Share this post


Link to post
Share on other sites
  • 0

Sorry for the late reply. I had problems with the computer.

 

I had this wonderfully naive thought that modding was actively supported by Bethesda. Meaning lots of deep documentation and tools. The mod file format documentation looks useful though. I had trouble understanding what kinds of records and fields there are. And how they're represented in TES5 of course. Seeing how someone could come up with all that by just taking something apart does give me hope :) For a start it might be easiest to try to take a compatibility patch and see how one of those is put together.

 

Thanks to you all for the help.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By packwatch
      Screenshot of the problem:

      I dragged SSEEdit from my 1080P Monitor to my 4K Monitor. Most things scaled properly, except for the font and title bar. I did it again. The font got smaller.
      My monkey brain thought this was funny, so I continued until I couldn't read it anymore. Well, there was NOTHING funny about it AT ALL.
      When I restarted, the font stayed the same.
      Overriding Windows DPI scaling to System for the Mod Organizer and xEdit applications didn't work. Running xEdit manually outside of MO didn't work. Changing theme didn't work. Deleting and reinstalling xEdit didn't work. Setting my windows scaling to 350% makes the font a little bigger, but still barely readable. Changing my Main Display in Windows/Nvidia and relaunching didn't work.
      I've been crying from troubleshooting an infinite loading screen error for the past 2 days and I need xEdit to help troubleshoot it. I miss Lydia. Please help.
    • By keithinhanoi
      Over time, and by reading comments of mod authors, I have learned that for certain record types displayed in TES5Edit, when using the conflict filter feature, some records are incorrectly identified as conflicting (ie., "conflicting" = overridden by a plugin which comes later in the load order.)
       
      For example, ripple, the author of Inconsequential NPCs has explained that location persistent references (LCPR sub-records in LCTN / Location records) supplied by different plugins are not actually overwritten by the last mod in the load order LCTN for certain locations (source). The implication here is that all those references are combined and used from all mods with that record type when Skyrim is started. So in other words, when making a compatibility patch in TES5Edit, you do not need to copy overrides for those particular records.
       
      I have asked in a number of places which records types do not need to be carried forward into compatibility patches, but have never received a reply, and still to this day have not found a definitive list in one place. Well, I'd like to change that, and I need your help, if this is something you are knowledgeable about.
       
      Below is a list of record types, grouped by category, that I have read comments saying they are incorrectly identified as conflicting, because they are actually combined at runtime:
       
      A List of Non-Conflicting Record Types seen in TES5Edit [WIP]
      Default Object Manager (DOBJ)Record sub-record types:
      DNAM - Objects  (Confirmed here) Dialogue Topic (DIAL) Record sub-record types:
      TFIC - Info Count  (Confirmed - sources: here & here) Dialogue Information (INFO) Record sub-record types:
      PNAM - Previous Info  (Confirmed here) Idle Animation (IDLE) Record sub-record types:
      ANAM - Related Idle Animations  (Confirmed here) Location (LCTN) Record sub-record types:  (Confirmed here)
      ACPR - Actor Cell Persistent Reference LCPR - Location Cell Persistent Reference RCUN - Reference Cell UNique ACSR - Actor Cell Static References LCSR - Location Cell Static Reference RCSR - Reference Cell Static Reference ACEC - Actor Cell Encounter Cell LCEC - Location Cell Encounter Cell RCEC - Reference Cell Encounter Cell ACID - Actor Cell Marker Reference LCID - Location Cell Marker Reference ACEP - Actor Cell Enable Point LCEP - Location Cell Enable Point NOTE: Other LCTN sub-record types require conflict management.
      (Confirmed - sources: here, here, here & here)

      Story Manager Quest Node (SMQN) Record sub-record types: (Confirmed - source: here & here)
      SNAM - Child sub-records QNAM - Quest Count / Quests Story Manager B??? Node (SMBN) Record sub-record types:
      SNAM - Child sub-records  (Confirmed here) For more details about how the above listed sub-record types merge at runtime, please see this excellent opening thread post by Arthmoor from 12 March 2014. Many thanks to him for confirming / explaining all of these, and a tip of the hat to MonoAccipitor for noticing Arthmoor's post.
       
      I will update this list with additional confirmed non-conflicting record types based on your replies.
      Thanks in advance for your help, and let's hope others can benefit from this list!
    • By KillingMachine1914
      Greetings, 
       
      As it was stated in the title, it seems DynDOLOD is not applying modded statue of Azura (Sexy Statues Skyrim v2 NSFW Warning) when generating LODs. When I launch the game, vanilla Azura appears instead of the modded variant. The modded statue of Azura will only appear once I get close enough to it. Also, I installed a mod which resize the statue 2x its normal to make it visible all across Skyrim (Elizabeths Tower - Azura Shrine SSE).
       
      I have checked SSEEdit to see what defines the conflict and it turns out DynDOLOD.esp set the statue as persistent only and removed "is full lod" definition which was there in Elizabeth's Tower. I have very little idea of what those terms mean but my gut tells me that it has something to do with the trouble I am facing currently. Hence, is there a method to overcome this matter? 
       
      Thank you very much in advance.
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.