Project talk:Data Dictionary

From Step Mods | Change The Game
Revision as of 06:30, February 12, 2014 by DoubleYou (talk | contribs) (→‎Discussion)

Vista-file-manager.png
Archive


New Properties for STEP Guides - WIP (Discussion Page)

Following is a work-in-progress list of current and proposed attributes that can be associated with mod pages. Feel free to add anything you might think is useful. Let's try to keep it organized, so group similar or related properties as best you can. Note that Baseline has been removed from this list on purpose, that should be a Pack attribute based on the Pack's guidelines. ~FarloUser Farlo Sig.pngTalk 19:56, July 15, 2013 (MDT)

Color key:

  • Not changed - Represents at most minor changes being made to the property.
  • Addition - Proposed additional/replacement property.
  • Removal - Property proposed to be removed/replaced.
[edit]
Mod Info
  • Author (string) - The author(s) of the mod.
  • FullName (string) - Full name of the mod according to source.
  • Description (string) - Brief mod description. (Confirmed)
  • SourceName (string) - Supported mod source. Specifies a URL created from "Steam" or "Nexus" mod ID. "Other" prompts for user input of a full URL string.
    • SourceID (string) - "Steam" or "Nexus" mod SourceID if either is specified in SourceName.
    • SourceURL (URL) - Full URL if SourceName is specified as "Other".
  • Section (string) - STEP installation "blocks" specified using "allows value" declarations. (Confirmed)
  • Category (category) - Broad category/"class" of the mod, similar (if not identical) to the Nexus categories. (Confirmed)
  • Baseline (string) - Specifies the recommended mod option when more than one are available. (Confirmed)
  • ForumTID (string) - Specifies a URL created using the thread ID of the mod on the STEP forums.
  • External URL (object) - Multiple-instance template containing:
    • ExternalURL (URL) - URL string specifying a page associated with the mod; other hosts, Facebook, etc. (URL)
    • ExternalLabel (string) - A human-meaningful label associated with ExternalURL (string)
  • HasResource (string) - Indicates how a mod's resources files are packaged. (BSA, Loose, None).
  • DLCRequired (string) - Indicates which DLC's are required by the mod.
  • DLCSupported (string) - Indicates which DLC's are supported by the mod via an addon.
Mod Flags
  • DocDescription (Boolean) - Indicates that the mod includes a description of what the mod does.
  • DocInstall (Boolean) - Indicates that the mod includes installation instructions.
  • DocUninstall (Boolean) - Indicates that the mod includes un-installation instructions.
  • HasScript (Boolean) - Indicates that the mod uses scripts.
  • CompatibleBAIN (Boolean) - Indicates that the mod package has BAIN support.
  • CompatibleFOMOD (Boolean) - Indicates that the mod package has FOMOD support.
  • IsOptimized (Boolean) - Applies to texture mods only. Indicates that the mod textures are correctly formatted (but not that they are necessarily size/constraint optimized).
  • HasSKSE (Boolean) - Indicates that the mod depends on SKSE.
  • HasMCM (Boolean) - Indicates that the mod has MCM functionality and depends on SkyUI.
  • HasSkyProc (Boolean) - Indicates that the mod contains a SkyProc patcher.
  • LoreFriendly (Boolean) - Indicates that the mod is considered lore friendly. (Confirmed)
  • IsClean (Boolean) - Indicates that the mod's EPS/M files are free of ITM/UDR errors. (Confirmed)
  • HasPlugin (Boolean) - Indicates that the mod has a plugin file (ESP, ESM, None). (Confirmed)
    • CleanPlugin (Boolean) - Indicates that the mod's plugin is clean. Dependent on HasPlugin. (Confirmed)
  • CleanUninstall (Boolean) - Indicates that mod can be deactivated without corrupting save games.
  • IsCore (Boolean) - Indicates if a mod is CORE to the STEP experience. (Confirmed)
  • AffectsFPS (Boolean) - Indicates if a mod has a noticeable FPS impact. (Confirmed)
  • AffectsVRAM (Boolean) - Indicates if a mod has a noticeable VRAM impact. (Confirmed)
  • PerformanceAvailable (Boolean) - Indicates if a mod has a performance version available. (Confirmed)
  • QualityAvailable (Boolean) - Indicates if a mod has a quality version available. (Confirmed)
  • QualityItems (object) - Multiple-instance template containing: (Confirmed)
    • QualityValue (string) - Available graphical tiers or options available (2048, 1024, High, Low, etc.). (Confirmed)
    • AffectsFPS (string) - Indicates any noticeable FPS impact (positive, negative, none).
    • AffectsVRAM (string) - Indicates any noticeable VRAM impact (positive, negative, none).
Recommendations
  • Recommendations (Text) - Contains either the short note, or a link to the Mod page if detailed instructions are available. This property is specified by Template:Recommendations.
  • Notes (string) - Mod-specific information or special notes.
  • Conflicts (?) - Figure out some method to denote salient conflicts of particular interest.
[edit]
Pack Info
  • Author (string) - The author(s) of the pack.
  • Description (string) - Brief pack description.
  • ForumTID (string) - Specifies a URL created using the thread ID of the mod on the STEP forums.
  • RequiredPack (string) - (Optional, multiple allowed) Pack(s) which are required for the pack to work. STEP:Core is an underlying requirement for all Packs and is thus excluded from this list.
  • CompatiblePack (string) - (Optional, multiple allowed) Pack(s) which are known to be compatible with this Pack, either inherently or through some kind of compatibility instruction/patch.
  • ModList Table - List of mods in the pack and specific installation instructions; similar to the current STEP guide's mod tables.

The following won't be stored semantically but are text boxes for the various text associated with the pack's description, installation, etc.

  • Description (none/plaintext) - Introduction, long description, purpose and goals, author notes, etc.
  • Pre-Installation Setup (none/plaintext) - Explanation of any pre-requisite steps that need to be taken before installing the pack's mods.
  • Post-Installation Config (none/plaintext) - Configuration instructions and anything else that should be done after installing the mods. Also outro and special credits.
ModList

The mod list for the pack will define a number of the following:

  • ModListObject (object) - Multiple-instance template containing:
    • Order (number) - Order of the mod in the pack's list.
    • ModName (string) - Name of the mod.
    • Notes (string) - Pack-specific notes regarding the mod's installation.
[edit]
Guide Info
  • Guide intent, description and background
  • Guide-specific procedures
  • Guide-specific INI settings
  • Summary
  • Credits
[edit]

IDEAS BEGIN HERE!


Discussion

STEP Guide Categories

I propose that we split Section J - Animations and Effects into two different categories. Section J should include only mods that add/modify in-game effects, like Burn Freeze Shock Effects. We would then add a new section near the end of the guide specifically for mods that affect Animations. This would include mods that edit the skeleton, like XPMS, and raw animations, like No Spinning Death Animation. We should also create a new category at the end for the STEP Patches. Maybe this section should include the Bashed Patch creation as well? --DoubleYou (talk) 01:30, February 12, 2014 (EST)


Mods - Description

Proposed adding Short Mod Description (1 sentence, possibly from the Nexus API); currently this isn't in either the mod page or the STEP guide. It would be hard to include this in a STEP guide (core or pack) since it doesn't fit well in a multi-column format like those used in the guide.

Agreed, at the very least it'll help fill out the mod pages. ~FarloUser Farlo Sig.pngTalk 18:20, July 16, 2013 (MDT)

Mods - Category

Proposed change to broad categorization of the mod (NOT the STEP section, similar idea to the Nexus categories but hopefully with categories more relevant to STEP) ~FarloUser Farlo Sig.pngTalk 18:20, July 16, 2013 (MDT)

Mods - IsOptimized

The flag is somewhat problematic since it applies only to textures and can be hard to verify especially for non-core mods. Moreover, we may still want to separately optimize/reoptimize the normal maps even if the color map textures are adequately optimized. ~Kelmych Talk 12:26, July 16, 2013

I think that this is a good flag for things that really should not be run through DDSopt for whatever reason. We can change the property and tag names easily enough if necessary. ~z929669 Ixian Insignia.png Talk 15:21, July 16, 2013 (MDT)
What should we do if the color map textures are optimized but the normal maps are too large. Should there be tgwo flags, or just ignore the normal map issues? Kelmych (talk) 21:11, July 16, 2013 (MDT)
I'd say keep only one flag. Textures that are already optimized won't be affected anyway. We can add exceptions to the INI where there are specific use cases that might get borked. ~z929669 Ixian Insignia.png Talk 00:50, July 18, 2013 (MDT)
IMO we should just track the file's optimization, they can be whatever resolution the author wants, unless I'm missing something about them being "too large" (is that a thing?). Unless everything is optimized it shouldn't be tagged as such. ~FarloUser Farlo Sig.pngTalk 23:44, July 16, 2013 (MDT)
Kelmych (talk) 17:12, November 16, 2013 (EST) Based on comments about the STEP Mod Texture table and experience using this table in developing and testing a batch file for bulk optimization, I feel that the interpretation of the flag that Z recommends is the correct one; it should be one flag that means it should not be run through DDSopt without a very good reason. The revised STEP Mod Texture table in the DDSopt guide has this flag (column 2). So far the mods I've evaluated and that have this flag as Yes don't need normal map resolution changes.

Mods - Optimization-related Data

Kelmych (talk) 17:39, November 16, 2013 (EST) A few months ago I changed the information included in the STEP Mod Texture table in the DDSopt guide based on comments from some users who said it the entries needed to be easier to interpret while making decisions about optimization parameters and choices. I feel that the fields for mod-specific data about optimization should actually be in the Mod Data Dictionary and the data itself entered in a set of parameters on a Mod page, perhaps in an Optimization tab. There would also need be a new template so the data could be assembled to create a new version of the STEP Mod Texture table in the DDSopt guide. The new flags and data element I suggest are shown below. The values for these are independent of the pack or packs that include the mod. The equivalent of the IsOptimized flag is already in the STEP Mod Texture table so it is not included below. Note that values for all these parameters are available for the STEP mods in the STEP Mod Texture table in the DDSopt guide.

Kelmych (talk) 20:20, November 17, 2013 (EST) We don't test individual mods and decide whether optimization helped. There isn't enough of a change from optimizing a single mod to get useful indications on how to optimize. Recommendations on optimization are based on testing with many mods optimized simultaneously and extracting results based on different optimization choices for classes of textures used. I created the flag recommendations based on the texture classes that we've found through testing are important in determining how to optimize a mod. Results of testing have affected the choice of properties, but not the value (e.g., True or False) for these properties. The property values are characteristics of the mod itself.
Kelmych (talk) 20:20, November 17, 2013 (EST) The flags and texture resolution string help a user determine how to optimize a mod (settings, filters,...), and to understand how much effort and time it will take to optimize the mod. This information is not typically available in the Nexus mod description and some isn't even available when looking at the Windows explorer window for individual textures. Much of the information I use to determine the flag values for an individual mod is taken from logs created by DDSopt. Flag value differences between different versions of a mod versions or with mod updates are rare.
Kelmych (talk) 17:57, November 16, 2013 (EST) recommended mod optimization-related parameter additions (flags might use shorter names)
The following are binary flags
HasTextures
Kelmych (talk) 20:20, November 17, 2013 (EST) This is just an indicator flag; if it is not true for a mod then the other properties are not relevant
HasExteriorTextures
Kelmych (talk) 20:20, November 17, 2013 (EST) Exterior textures are important since they seem to be a primary cause of high VRAM use. When optimizing textures most users are now reducing the resolution of exterior textures to half the resolution of the other textures. In my system, as an example, I got a roughly 15% decrease in VRAM use by doing this. This flag is present so a user knows that when optimizing this mod an extra step is needed with different optimization parameters for the exterior textures.
HasUncompressedTextures
Kelmych (talk) 20:20, November 17, 2013 (EST) Mods with uncompressed textures need an extra processing step during optimization with different processing parameters to properly optimize the uncompressed textures. Often it isn't apparent that a mod has some uncompressed textures, and this flag would let users know that they need to identify the uncompressed textures and optimize them with different optimization parameters.
HasAllUncompressedTextures
Kelmych (talk) 20:20, November 17, 2013 (EST) For mods whose textures are all uncompressed, the optimization processing parameters are different than the usual parameters used to optimize a mod. If all the textures are uncompressed, then the optimization can be done in a single step. This flag might not be needed since a mod with the HasCompressedTextures as True and MultiStepOptimizationRecommended as False is probably equivalent to this flag.
HasCollapsableTextures
Kelmych (talk) 20:20, November 17, 2013 (EST) Collapsable textures are nxn color maps with the same color in every cell. DDSopt will collapse these into a single cell with the same color. Most graphics cards handle these properly but a few don't. When users have texture problems one of the first causes they suspect is that DDSopt collapsed some of the textures. This usually does not prove to be the actual problem, but this flag lets users know which mods actually have such textures so they can more efficiently determine the cause of texture problems.
MultiStepOptimizationRecommended
Kelmych (talk) 20:20, November 17, 2013 (EST) This flag indicates whether a mod has any exterior textures, uncompressed textures, or normal maps that should be reduced in resolution. If any of these conditions exist then optimization of a mod's textures should be done in multiple steps each with a different subset of the textures and different optimization parameters. If the flag is false the user knows that ordinary optimization will be fine with this mod and no special processing is needed.
The following are strings
MaxTextureResolutionByModVersion
Kelmych (talk) 20:20, November 17, 2013 (EST) This string will contain the max texture resolution for each version of the mod that contains textures with different resolutions. This is particularly useful in setting an appropriate value for the Resolution Limit parameter in DDSopt optimization, and for users to determine which mods need to have textures reduced in size during optimization.


Mods - LoreFriendly

Proposed removal: not a whole lot of meaning for most mods and isn't well defined. Also mostly useless for our purposes. ~FarloUser Farlo Sig.pngTalk 18:20, July 16, 2013 (MDT)

it is often difficult to assign/verify this flag ~Kelmych Talk 12:26, July 16, 2013

Mods - IsClean/HasPlugin

Proposed: Fully specify if mod has plugins and that they are clean. Also, disambiguate clean plugins from clean uninstall (or eliminate clean uninstall altogether)

Mods - IsCore

Proposed: Remove, core mods -- will be in STEP:Core, this attribute isn't tied to the mod itself.

Agreed ~FarloUser Farlo Sig.pngTalk 18:20, July 16, 2013 (MDT)

Mods - FPS and VRAM

Proposed: should have way to indicate whether effect is positive or negative ~Kelmych Talk 12:26, July 16, 2013

Only Vano89's mod positively impacts performance, so this really denotes a negative impact. ~z929669 Ixian Insignia.png Talk 15:25, July 16, 2013 (MDT)
But that might not be true for much longer, what if someone made a "craptastic computer pack", it might be nice to know that a mod has a positive effect. I think a choice of positive/none/negative would be well suited. ~FarloUser Farlo Sig.pngTalk 18:20, July 16, 2013 (MDT)
I was thinking about relative improvements compared to some STEP baseline, in which case a mod that reduces quality a small amount but improves FPS a lot might get a positive check here. Skyrim Performance Plus would certainly get a plus here also. Kelmych (talk) 21:11, July 16, 2013 (MDT)
When Z, Stopping, and I were talking we kind of came to the conclusion that baseline wouldn't be a universal aspect since packs can target differing computer ranges. The AffectsFPS and AffectsVRAM tags would then be compared to Vanilla; for example an HD mod would obviously negatively impact FPS and VRAM whereas some gameplay mod wouldn't affect either. ~FarloUser Farlo Sig.pngTalk 23:44, July 16, 2013 (MDT)

Mods - Recommendations/Notes

Proposed: Attach mod-specific notes to mod pages. Packs will also need recommendations relating to specific mods. Template:Recommendations could be used at the Pack level as it currently stands.

Proposed: s4n suggested adding capability to set flags (plus mod IDs) to note conflicts.

Mods - Conflicts

how about a list of important conflicts using the STEP mod index in the list Kelmych (talk) 21:11, July 16, 2013 (MDT)

We thought about this, but it'd be a huge pain in the ass to keep every mod updated with the mods it conflicts with. Not really sure how to go about this one. ~FarloUser Farlo Sig.pngTalk 23:44, July 16, 2013 (MDT)

Packs - ForumTID

We're just giving each pack it's own thread right? Some of the bigger ones could be given a subforum (STEP, SR, etc.) but we can deal with that on a case-by-case basis) ~FarloUser Farlo Sig.pngTalk 20:32, July 16, 2013 (MDT)