STEP:Mod Testing

Mod Testing Guide

In-depth mod testing process for Step Modifications

by: The Step Team  | Forum Topic

WIP

This Guide is currently outdated and requires updating to current processes/standards. It will be worked on over time.

To ensure consistent and reliable mod testing, it's essential to minimize variations that can lead to a range of issues and inconsistencies. This guide will outline standardized techniques for mod testing that Step Modifications employs for evaluating all mods featured in our official modding guides. The recommendations provided progress through system and software setup before delving into the mod testing process.

Understanding Mod Testing

Mod testing encompasses various aspects. However, in the context of this guide, it refers to the assessment of mods for potential inclusion in a curated mod list. This evaluation involves several key components:

1. Stability Testing: assesses a mod's stability when integrated with other mods in the mod list. It aims to identify any conflicts or issues that may arise from mod interactions.
2. Functionality Testing: mod functionality is scrutinized to ensure it operates as intended. Any deviations from the mod's intended purpose are noted and addressed.
3. Mod Comparison: mods with similar content are compared to determine which one provides the best solution.
4. Lore and Mandate Assessments: mods are evaluated against Elder Scrolls lore to ensure they align with the established lore and adhere to the Step Mandates. This ensures that the mod fits seamlessly into the Elder Scrolls universe.

It's important to acknowledge that mod testing can be a time-consuming and meticulous task. Nevertheless, it can also be rewarding and enjoyable. Mod testing is ideally suited for individuals who are genuinely dedicated to the process. Consequently, turnover rates among users involved in official testing are high, as many underestimate the demands of mod testing; quickly become overwhelmed by the complexity of the task.

Following is described the detailed process required for testing mods in a manner that ensures consistency and facilitates recommendations (or not) for a Step Game Guide. Our aim is to maintain a process that is straightforward and efficient, yet given the complexity and extensive nature of some mods, this process may not encompass every conceivable mod testing scenario. While common scenarios are provided, it's crucial for testers to exercise best judgment when faced with situations not explicitly covered.

System Preparation

Before embarking on mod testing, it is essential to establish a stable system and game installation. Without them, distinguishing whether a mod is incompatible with the mod list or the system becomes significantly more challenging. Testers should have their systems configured according to one of the Step Game Guides. Nevertheless, here are some general reminders and best practices to ensure the environment is adequately prepared for mod testing.

General Setup

• Ensure the System Setup Guide is complete
• Ensure a clean and pure installation of the Step Game Guide for which testing is being performed

  A pure installation means nothing exists within the mod list that isn't included in the Game Guide; the installation should be 100% within the bounds of the Guide.

Game Settings

All game INIs, launcher, and in-game settings should be configured according to the Game Guide for which testing is being performed. Therefore, any post-installation changes should be reverted back to the Guide's instructions for the duration of testing.

Mod Organizer Profile Setup

Step Modifications' officially supported mod manager is Mod Organizer, and it proves itself more helpful than other managers for mod testing. Follow these instructions to set up two new profiles in Mod Organizer for testing:

1. Open Mod Organizer and select the instance for the game for which testing is being performed
2. Select the icon
3. Copy the Step Game Guide profile
4. Name the new profile: Vanilla
   • Make sure the Local Savegames box is not checked for the testing profiles
   • Make sure the Automatic Archive Invalidation box is selected for the testing profiles
5. Name the new profile: GuideName vX.X Testing
   • Make sure the Local Savegames box is not checked for the testing profiles
   • Make sure the Automatic Archive Invalidation box is selected for the testing profiles
6. Select the newly created Vanilla profile and ensure no mods are enabled

Vanilla Save

Though playing through vanilla content may be at the top of anyone's list, this step is not optional. A vanilla save point is required for some comparisons, as well as, providing a default environment from which to start the majority of testing.

1. Select the Vanilla profile
2. Start a new game from this Vanilla profile
3. Play through any introductory quests until the first point is reached that allows for free roaming of the game
4. Once at a free roaming point, save the game
   • Fallouts: play until at the starting vault's exterior
   • Skyrim: play until at the exterior of Helgen cave
   • Starfield: play until the Frontier can be freely piloted and save at NA Spaceport

   Tip: Use the console to create a save with a custom name for quick reference. (save SaveName)

External Testing Procedures

All testing begins outside the game. Each of the following steps should be completed and notes taken from each before proceeding to in-game testing.

Step 1 - Forum Mod Topic

Read the mod's topic opening post (OP)

The mod's topic OP often contains an outline of what needs to be tested for a particular mod. Make note of this and use it when in-game testing begins. If nothing is outlined in the OP, it's likely the mod has not been tested or reviewed enough to include it. In such cases, testers will establish a outline by editing the OP. This outline should include what mod options should be tested, how to test, steps for making the mod compatible with other mods, etc. Most of this information will be gathered from the actual testing process below.

Step 2 - Nexus Page

Read the Nexus Page in its entirety

• Description: make note of any special installation/uninstall instructions, any known internal or mod compatibility issues, and of any potential conflicts with supported DLCs and other mods within the Step Game Guide.
• Posts: a complete read is not necessary; however, testers should attempt to develop a sense of user satisfaction. Posts can also reveal a list of possible bugs not listed within the Bugs section. Unless a mod is fully developed with few to no bugs being reported, some sort of interaction from the mod author is preferred to ensure the mod isn't abandoned.
• Changelog: changelogs often provide a sense of what direction the mod it headed, how often issues are addressed, features being edited, etc.
• Bugs: review any active bug reports. Too many bugs may disqualify a mod from being included within a Step Game Guide.

Step 3 - Validation

Validate the archive package using MO or 7zip

Observe if the downloaded archive is properly structured and configured for installation or not. For more complex installations, Mod Organizer FOMODs are preferred; however, MO can install BAIN type installations as well. FOMODs can be validated using an online validation tool, XML Validator, or by using FOMOD Validator. When installing mods using Mod Organizer, the bottom pane will also have a report of potential issues with the package.

Step 4 - xEdit

Inspect plugins using xEdit

Knowledge of xEdit and conflict resolution is required for this step. Load the entire Step Game Guide and the mod being tested with xEdit. Review and note any issues of the mod failing to carry forward changes from DLCs, mods, and patches included within the Step Game Guide. These notes help with creating correct conflict resolution via official patches.

Step 5 - Installation/Uninstall

• Validate installation procedure listed on the Nexus page.
• Validate uninstallation procedure listed on the Nexus page.

Step 6 - Inspection

Inspect the installation in MO

1. Install the mod as it will be used for testing
2. Run LOOT and note information

   Unless the mod is new on the scene, LOOT will recognize it and provide some valuable information such as if the mod is clean or dirty, requires other mods, etc.

3. Using Mod Organizer, take note of any asset conflicts that appear against vanilla files and other mods in the Step Game Guide.

   Extract any BSAs to ensure the conflicts are being read. BSAs should not remain extracted unless required for further testing. Otherwise, remove the extracted BSA contents before continuing any testing.

Step 7 - Mandate

The final step is to evaluate the mod against the Step Modifications Mandate. Ensure the mod does not fall into the "...not about" sections. If it does, no more testing is required. Post findings why the mod doesn't fit the Mandate to the mod's topic. If the mod succeeds in passes this step, continue on to the remainder of testing the mod.

Testers should update the mod's topic OP on the forums with any relevant information gathered from the above testing.

In-Game Testing Procedures

This process is the most important and will detail steps required for testing mods in a consistent way so they may be recommended (or not) for a Step Game Guide. It will be as simple and streamlined as possible, however, with the complexity, breadth, and depth that mods can be, this process will never be able to cover all mod testing scenarios. Below we provide the most common of scenarios, however, when the scenario isn't covered best-judgment must be used.

NOTE:

The process for capturing comparison screenshots can be found following these procedures; which are for gameplay testing.

Pretesting Setup

Enable the testing profile
Testing for Step Game Guides should be completed from the profile set up, above, for that Guide in Mod Organizer: GuideName vX.X Testing.

Step 1 - The Vanilla Experience

It's important to be familiar with the original content whether it be textures, quests, game mechanics, etc. Therefore, if there is no familiarity with the changes being tested, the original content should be experienced first. If familiarity does exist this step may be skipped. To complete this step:

1. Load the Vanilla profile in MO, launch the game, and load the Vanilla save
2. Fast travel to or use the command console to load the area closest to the mod's altered content

   Tip: before starting the content, make a new save that can be used later for faster testing

3. Play the game for a period of time to experience or view the vanilla content
4. Close the game and review findings for the next step

Step 2 - Experience the Mod

1. Relaunch Mod Organizer and select the Testing profile
2. Activate the mod being tested
3. Relaunch the game and load the Vanilla save or a save created for testing from Step 1
4. Allow the mods to initialize
5. Play the game for a period of time; assessing the mod:
   • Does it work as intended?
   • Does it look as intended?
   • Does it fit in with the game's content/ambiance/environment?
   • Are there any issues?
6. Close the game and review the findings

Final Step - Review

1. Gather the findings from all testing sources including the External testing.
2. Summarize a review and post this on the mod's topic on the Step Forums. Content to include are, but not limited to:
   • Issues with file structure and installation
   • Conflicts with vanilla or guide content (and solutions, if any)
   • Whether the mod met or fell below expectations
   • Any in-game issues
   • Personal assessment of whether the mod should or shouldn't be included in a Step Game Guide

NOTE

Be mindful to remain objective. Do not write anything that will reflect upon the mod's author in a personal manner. Only review the craftsmanship, reliability, operation, etc of a mod, itself. Mods are assessed for guide inclusion, not for judgment against authors!

Capturing Comparison Sets

When testing mesh and/or texture based mods, comparison screenshots are required and are to be posted for the community to review. These shots are not for the staff to use for evaluation, since in-game testing should be completed, but rather they are for the community's benefit and staff reference.

NOTE

When screenshots or videos are needed for comparisons, provide them from in-game sources. Do not use "studio" applications to provide shots and/or video. In-game lighting conditions and rendering can change the appearance of many textures compared to studio applications. Therefore, it is very important to capture these comparisons from within Skyrim itself.

Capture Procedure

Comparison shots need to include a Vanilla shot, current Guide content shot, and a shot of the content of the mod being tested. To capture screen shots that work best for these comparisons use the best practices outlined below. This outline will allow the shots to be of the exact same screen, in each shot, with the only differences being the asset replacements.

1. Active Immersive HUD, if not already, for all MO profiles used in the testing, including the vanilla profile. This will auto hide the HUD so that it doesn't have to be done manually from the console.
2. Load the vanilla profile and vanilla save file. The mod being tested should not be active.
3. While in-game, locate the vanilla texture(s) that need to be captured. Explore until the texture(s) are found or fast travel or COC to the texture's location, if known.
4. Once the texture is found, line the shot up in the frame. Pay attention to lighting and angles to make sure the shot will be good for compares.
5. Ensure there is nothing that will interfere in the shot such as an NPC walking into the frame.
6. Save the game with a new save, if not loading from a previous save.
7. Load the save and Do NOT touch that mouse!
8. Once the game loads, make sure there is no text on the screen and Immersive HUD has hidden the HUD elements. When all is good, press the screen capture key for the program being used to capture.
9. Now exit the game.
10. Switch to the Guide profile.
11. Repeat steps 7-9 above for the Guide shots.
12. Now in the current Guide profile, active the mod being tested and ensure it is overwriting the desired assets.
13. Repeat steps 7-9 to capture the shots from the mod being tested.

If completed correctly, a set of three compares should exist: a vanilla shot, a Guide shot, and a shot with the mod included. These can be used to post comparison shots on the forums. Simply repeat this process for all assets that require a comparison set for any given mod. When only one texture is being tested, it is nice to provide several compares from various angles/locations/lighting situations for a more complete comparison set.

NOTE

The remainder of the guide contains helpful console commands, tips, tools, and more.

Helpful Console Commands

Below is a list of helpful console commands to use while testing. Use these to expedite the testing while in-game. They are specific to Skyrim and other Bethesda game. To open the in-game console press the [ ~ ] (tilde) key, which is normally located just below the ESC key on a standard keyboard. Press it again to close the console.

Toggles

• tmm 1

  This will toggle all map markers on; thus allowing you to fast travel to any location quickly. Enter 0 in place of 1, to reset to default.

• tgm

  Toggles God Mode on/off making you invincible. Health, magicka and stamina will not run out either with enabled.

• tcl

  Toggles collisions on/off. Don't use while falling or you may crash to desktop.

• tfc

  Toggles Free Camera Mode on/off so you can fly around the environment. 'tfc 1' will pause the camera.

• tm

  Toggles menus on/off. Useful when taking screenshots. Immersive HUD/Immersive HUD - iHUD Special Edition can be used to achieve this without having to use the console.

• tai

  Toggles Artificial Intelligence so characters will not react to the player.

• tcai

  Toggle Combat Artificial Intelligence so NPCs will not attack the player.

Player Commands

player.additem formID ###

Adds the item to the player's inventory. Replace 'formID' with the item code of the item. Codes can be found here. Replace the '###' with the number of items to add. For example, to add gold: player.additem f 200 (this adds 200 gold to the player's inventory.)
Tip: leading zeroes can be dropped in the item code.

player.addspell formID

Adds a specified spell to the player's spell list. Spell codes can be found here.

Other Commands

• coc locationName

  Transports the player to a specified location. Replace the locationName with the name of the location. A list of location names can be found here and here.

• coc qasmoke

  Transports the player to the developer testing hall.

• unlock

  Unlocks the targeted object (doors, chests, etc). To target an item/object click on it while the console is open.

• Kill

  Kills a targeted enemy.

• killall

  Kill all non-essential NPCs within the player's vicinity.

• Resurrect

  Resurrects a dead target.

• set timescale to ##

  Changes the timescale of the game. 20 is the default setting. Setting this below 10 can cause issues.

External Tools

Capture Software

FRAPS is an good program for both screen and video captures in-game, as well as, capturing FPS data; however, feel free to use the preferred program for captures. The paid version of FRAPS provides more functionality and is recommended. PNG is the preferred format for screen captures due to its accuracy in capturing correct colors, saturation, tints, etc. JPEG is not recommended because some of the image's originality can be lost; however, it can be used if PNG isn't an option. Do not use GIF format for screen captures!

For video captures, AVI, MKV, and MP4 file formats (containers) are best. Using one of these three formats or a higher quality one for capturing video is very important for proper captures in high definition. Use the H.264 codec, if possible, and MPEG-4 as a second option. Video for true HD should be captured in at least 1080p. Audio for video captures, if it can be set, should be no less than a 48khz sample rate and no less than a 128kbps bit rate (96khz sample/384kbps bit rate is recommended for true HD audio). Use this information for encoding edited videos for compares as well. Adobe Premiere Elements is excellent for this, but rather expensive.

When uploading captures for posting compares on the forums, please use a 3rd party service to host captures. Do not store your captures on the STEP wiki! Imgur and Postimage have proven to be an excellent free image hosting sites for uploading screen captures to. Please use the editor functions on the forums to post compares. For posting video captures, please use YouTube. Other services have proven themselves annoying for community members to use; most requiring an account to view the videos.

Quick tip: Unless the monitor resolution is 1920x1080 or higher (and capturing at that resolution), do not encode your videos in 1080p. The result will be blurry due to upscaling. If below 1920x1080, 1600x900 for example, encode videos in 720p.

Texture Viewers

Nvidia Texture Tools

Intel Texture Works Plugin - allows you to open, edit, save DDS files in Adobe Photoshop using the latest formats (including BC7).

Gimp users can download a DDS plugin. For BC7 support, Gimp users can see these instructions.