STEP:Mod Testing

Step Guide-Mod Testing

An in-depth manual to mod testing for Step Modifications

by: The Step Team  | Forum Topic

To ensure consistent and reliable mod testing, it's essential to minimize variations that can lead to a range of issues and inconsistencies. This guide will outline standardized techniques for mod testing that Step Modifications employs for evaluating all mods featured in our official modding guides. The recommendations provided progress through system and software setup before delving into the mod testing process.

Understanding Mod Testing

Mod testing encompasses various aspects. However, in the context of this guide, it refers to the assessment of mods for potential inclusion in a curated mod list. This evaluation involves several key components:

1. Stability Testing: assesses a mod's stability when integrated with other mods in the mod list. It aims to identify any conflicts or issues that may arise from mod interactions.
2. Functionality Testing: mod functionality is scrutinized to ensure it operates as intended. Any deviations from the mod's intended purpose are noted and addressed.
3. Mod Comparison: mods with similar content are compared to determine which one provides the best solution.
4. Lore and Mandate Assessments: mods are evaluated against Elder Scrolls lore to ensure they align with the established lore and adhere to the Step Mandates. This ensures that the mod fits seamlessly into the Elder Scrolls universe.

It's important to acknowledge that mod testing can be a time-consuming and meticulous task. Nevertheless, it can also be rewarding and enjoyable. Mod testing is ideally suited for individuals who are genuinely dedicated to the process. Consequently, turnover rates among users involved in official testing are high, as many underestimate the demands of mod testing; quickly become overwhelmed by the complexity of the task.

Following is described the detailed process required for testing mods in a manner that ensures consistency and facilitates recommendations (or not) for a Step Game Guide. Our aim is to maintain a process that is straightforward and efficient, yet given the complexity and extensive nature of some mods, this process may not encompass every conceivable mod testing scenario. While common scenarios are provided, it's crucial for testers to exercise best judgment when faced with situations not explicitly covered.

System Preparation

Before embarking on mod testing, it is essential to establish a stable system and game installation. Without them, distinguishing whether a mod is incompatible with the mod list or the system becomes significantly more challenging. Testers should have their systems configured according to one of the Step Game Guides. Nevertheless, here are some general reminders and best practices to ensure the environment is adequately prepared for mod testing.

General Setup

• Ensure the System Setup Guide is complete
• Ensure a clean and pure installation of the Step Game Guide for which testing is being performed

  A pure installation means nothing exists within the mod list that isn't included in the Game Guide; the installation should be 100% within the bounds of the Guide.

Game Settings

All game INIs, launcher, and in-game settings should be configured according to the Game Guide for which testing is being performed. Therefore, any post-installation changes should be reverted back to the Guide's instructions for the duration of testing.

Mod Organizer Profile Setup

Step Modifications' officially supported mod manager is Mod Organizer, and it proves itself more helpful than other managers for mod testing. Follow these instructions to set up two new profiles in Mod Organizer for testing:

1. Open Mod Organizer and select the instance for the game for which testing is being performed
2. Select the icon
3. Copy the Step Game Guide profile
4. Name the new profile: Vanilla
   • Make sure the Local Savegames box is not checked for the testing profiles
   • Make sure the Automatic Archive Invalidation box is selected for the testing profiles
5. Name the new profile: GuideName vX.X Testing
   • Make sure the Local Savegames box is not checked for the testing profiles
   • Make sure the Automatic Archive Invalidation box is selected for the testing profiles
6. Select the newly created Vanilla profile and ensure no mods are enabled

Vanilla Save

Though playing through vanilla content may be at the top of anyone's list, this step is not optional. A vanilla save point is required for some comparisons, as well as, providing a default environment from which to start the majority of testing.

1. Select the Vanilla profile
2. Start a new game from this Vanilla profile
3. Play through any introductory quests until the first point is reached that allows for free roaming of the game
4. Once at a free roaming point, save the game
   • Fallouts: play until at the starting vault's exterior
   • Skyrim: play until at the exterior of Helgen cave
   • Starfield: play until the Frontier can be freely piloted and save at NA Spaceport

   Tip: Use the console to create a save with a custom name for quick reference. (save SaveName)

External Testing Procedures

All testing begins outside the game. Each of the following steps should be completed and notes taken from each before proceeding to in-game testing.

Step 1 - Forum Mod Topic

Read the mod's topic opening post (OP)

The mod's topic OP often contains an outline of what needs to be tested for a particular mod. Make note of this and use it when in-game testing begins. If nothing is outlined in the OP, it's likely the mod has not been tested or reviewed enough to include it. In such cases, testers will establish a outline by editing the OP. This outline should include what mod options should be tested, how to test, steps for making the mod compatible with other mods, etc. Most of this information will be gathered from the actual testing process below.

Step 2 - Nexus Page

Read the Nexus Page in its entirety

• Description: make note of any special installation/uninstall instructions, any known internal or mod compatibility issues, and of any potential conflicts with supported DLCs and other mods within the Step Game Guide.
• Posts: a complete read is not necessary; however, testers should attempt to develop a sense of user satisfaction. Posts can also reveal a list of possible bugs not listed within the Bugs section. Unless a mod is fully developed with few to no bugs being reported, some sort of interaction from the mod author is preferred to ensure the mod isn't abandoned.
• Changelog: changelogs often provide a sense of what direction the mod it headed, how often issues are addressed, features being edited, etc.
• Bugs: review any active bug reports. Too many bugs may disqualify a mod from being included within a Step Game Guide.

Step 3 - Validation

Validate the archive package using MO or 7zip

Observe if the downloaded archive is properly structured and configured for installation or not. For more complex installations, Mod Organizer FOMODs are preferred; however, MO can install BAIN type installations as well. FOMODs can be validated using an online validation tool, XML Validator, or by using FOMOD Validator. When installing mods using Mod Organizer, the bottom pane will also have a report of potential issues with the package.

Step 4 - xEdit

Inspect plugins using xEdit

Knowledge of xEdit and conflict resolution is required for this step. Load the entire Step Game Guide and the mod being tested with xEdit. Review and note any issues of the mod failing to carry forward changes from DLCs, mods, and patches included within the Step Game Guide. These notes help with creating correct conflict resolution via official patches.

Step 5 - Installation/Uninstall

• Validate installation procedure listed on the Nexus page.
• Validate uninstallation procedure listed on the Nexus page.

Step 6 - Inspection

Inspect the installation in MO

1. Install the mod as it will be used for testing
2. Run LOOT and note information

   Unless the mod is new on the scene, LOOT will recognize it and provide some valuable information such as if the mod is clean or dirty, requires other mods, etc.

3. Using Mod Organizer, take note of any asset conflicts that appear against vanilla files and other mods in the Step Game Guide.

   Extract any BSAs to ensure the conflicts are being read. BSAs should not remain extracted unless required for further testing. Otherwise, remove the extracted BSA contents before continuing any testing.

Step 7 - Mandate

The final step is to evaluate the mod against the Step Modifications Mandate. Ensure the mod does not fall into the "...not about" sections. If it does, no more testing is required. Post findings why the mod doesn't fit the Mandate to the mod's topic. If the mod succeeds in passes this step, continue on to the remainder of testing the mod.

Testers should update the mod's topic OP on the forums with any relevant information gathered from the above testing.

In-Game Testing Procedures

This process is the most important and will detail steps required for testing mods in a consistent way so they may be recommended (or not) for a Step Game Guide. It will be as simple and streamlined as possible, however, with the complexity, breadth, and depth that mods can be, this process will never be able to cover all mod testing scenarios. Below we provide the most common of scenarios, however, when the scenario isn't covered best-judgment must be used.

NOTE:

The process for capturing comparison screenshots can be found following these procedures; which are for gameplay testing.

Pretesting Setup

Enable the testing profile
Testing for Step Game Guides should be completed from the profile set up, above, for that Guide in Mod Organizer: GuideName vX.X Testing.

Step 1 - The Vanilla Experience

It's important to be familiar with the original content whether it be textures, quests, game mechanics, etc. Therefore, if there is no familiarity with the changes being tested, the original content should be experienced first. If familiarity does exist this step may be skipped. To complete this step:

1. Load the Vanilla profile in MO, launch the game, and load the Vanilla save
2. Fast travel to or use the command console to load the area closest to the mod's altered content

   Tip: before starting the content, make a new save that can be used later for faster testing

3. Play the game for a period of time to experience or view the vanilla content
4. Close the game and review findings for the next step

Step 2 - Experience the Mod

1. Relaunch Mod Organizer and select the Testing profile
2. Activate the mod being tested
3. Relaunch the game and load the Vanilla save or a save created for testing from Step 1
4. Allow the mods to initialize
5. Play the game for a period of time; assessing the mod:
   • Does it work as intended?
   • Does it look as intended?
   • Does it fit in with the game's content/ambiance/environment?
   • Are there any issues?
6. Close the game and review the findings

Final Step - Review

1. Gather the findings from all testing sources including the External testing.
2. Summarize a review and post this on the mod's topic on the Step Forums. Content to include are, but not limited to:
   • Issues with file structure and installation
   • Conflicts with vanilla or guide content (and solutions, if any)
   • Whether the mod met or fell below expectations
   • Any in-game issues
   • Personal assessment of whether the mod should or shouldn't be included in a Step Game Guide

NOTE

Be mindful to remain objective. Do not write anything that will reflect upon the mod's author in a personal manner. Only review the craftsmanship, reliability, operation, etc of a mod, itself. Mods are assessed for guide inclusion, not for judgment against authors!

Capturing Comparison Sets

When testing mesh and/or texture based mods, comparison screenshots are required and are to be posted for the community to review. These shots are not for the staff to use for evaluation, since in-game testing should be completed, but rather they are for the community's benefit and staff reference.

Screenshots and videos are provided them from in-game sources. Do not use "studio" or "editor" applications! In-game lighting conditions and rendering can change the appearance of many game assets; therefore, it's important to capture these comparisons from within the game, itself. Any capture software may be used as long as the saved image is a 1:1 match in color and quality to the in-game image. Some recommendations can be found within the External Tools header below.

NOTE

Do not use PNG as the file format when posting compares on the forums. They tend to be large files, which in turn cause long loading times on pages. JPG/JPEG should be sufficient and is about one quarter the payload of PNGs. Only use PNG when transparency is required.

Capture Procedure

Comparison shots and videos need to include:

1. Vanilla reference
2. current Guide reference
3. mod reference

This process will allow captures to be of the exact same reference point, in each capture, with the only difference being asset replacements.

1. Active a HUD mod that auto-hides the HUD for all MO profiles, including Vanilla (i.e, for Skyrim active Immersive HUD). This will auto-hide the HUD so that it doesn't have to be done manually from the console. If the Game Guide doesn't include one of these mods, the HUD will have to be manually hidden or remain present in the captures.
2. Load the Vanilla profile and Vanilla save file

   The mod being tested should only be active in the Testing profile.

3. While in-game, locate the vanilla reference(s) that need to be captured.
4. Once the reference is found, line the capture up in the frame
   • Pay attention to lighting and angles to make sure the capture will be good for compares.
   • Ensure nothing will interfere in the shot such as an NPC walking into the frame.
5. Once the capture is lined up, create a new save
6. Now load the save that was just created

   Do NOT touch the mouse! Doing so may cause the capture to because misaligned.

7. Once loaded, ensure there is no temporary text on the screen and the HUD has been hidden

   Manually hide the HUD (see commands below) or simply leave it in the capture when no mod exists for auto-hide.

8. Take the capture using the hot key for the capture software being used
9. Exit the game
10. Switch to the Game Guide profile
11. Repeat steps 6-9 above for the Guide captures
12. Switch to the Testing profile
13. Active the mod being tested and ensure its conflict resolution is as desired
14. Repeat steps 6-9 above for the mod captures

Posting Captures

Post the comparison capture(s) on the Step Forums. Repeat this process for all assets that require a comparison set for any given mod. When only one asset is being tested, it is encouraged to provide several captures from various angles/locations/lighting situations for a more complete review.

When posting captures on the Step Forums:

• Please use the Editor buttons/functions on the forums to post.
• Use a 3rd party service to host captures. Do not store your captures on the Step Wiki!
  • Imgur and Postimage have proven to be an excellent, free image hosting sites for uploading screen captures to.
  • Use YouTube for hosting video captures.

    Other services have proven themselves annoying for community members to use; most requiring an account to view the videos.

NOTE

The remainder of the guide contains helpful console commands, tips, tools, and more.

Helpful Console Commands

Provided is a list of helpful console commands to use while testing. They are specific to Bethesda games.
To open the in-game console press the ~ (tilde) key, which is normally located just below the Esc (escape/exit) key on a standard keyboard. Press it again to close the console.

Toggles

• tmm 1

  Toggles all map markers on; thus allowing fast travel to any location. Enter 0 in place of 1, to reset to default.

• tgm

  Toggles God Mode on/off. While in God Mode the PC will be invincible with unlimited health, magicka, and stamina.

• tcl

  Toggles collisions on/off; thus allowing the PC to walk through anything.

• tfc

  Toggles Free Camera Mode on/off. tfc 1 will pause the camera.

• tm

  Toggles menus on/off. Useful when taking captures.

• tai

  Toggles all NPC AI on/off. NPC will typically not move when toggled off.

• tcai

  Toggles all NPC Combat AI on/off. NPCs will interact normally, but will not actively attack the player.

Player Commands

• player.additem formID ###

  Adds the item to the player's inventory.

  Replace formID with the baseID of the item. These IDs can be found here. Replace the ### with the number of items to add.

  For example, to add gold in Skyrim: player.additem f 200

  Tip: leading zeroes can be dropped from IDs. So 0000000f becomes just f.

• player.addspell formID

  Adds a specified spell to the player's spell list. Spell IDs can be found here.

Targeted Commands

Targeted commands require the reference to be selected in the console. To target a reference, use the mouse to click on it while the console is open.

• disable

  Disables a targeted, enabled reference.

• enable

  Enables a targeted, disabled reference.

• Kill

  Kills a targeted NPC.

• Resurrect

  Resurrects a dead NPC.

• unlock

  Unlocks a targeted object (doors, chests, etc).

Other Commands

• coc locationName

  Transports the player to a specified location. Replace locationName with the name of the location.

  A list of Skyrim location names can be found here and here.

  coc qasmoke is Skyrim's developer testing hall with containers for all game items.

• killall

  Kills all non-essential NPCs within the player's vicinity.

• set timescale to ##

  Changes the timescale of the game.

  Change ## to the desired value. 20 is default. Setting below 10 may cause issues.

External Tools

Capture Software

Feel free to use the preferred program for captures. Some 3rd party software used by the staff include:

• FPS Monitor
• FRAPS - doesn't play well with some Windows features

Screen Captures

The preferred format for screen captures is PNG due to its lossless nature. JPEG can be used as an alternative and is recommended for large comparison sets to reduce page loads. Do not use GIF format for screen captures! The simple solution for screen captures for some games will be to utilize their own built-in capture feature:

• Skyrim: use BethINI to set the capture path and use the PrtSc (print screen) key
• Fallout 4: hotkey the photo mode capture key

Video Captures

The preferred formats (containers) for video captures are AVI, MKV, or MP4. Using one of these formats or a higher quality one is important for proper video captures in high definition. Capture in at least 1080p for fullHD, but always in a 16:9 format. Also use the following information for encoding:

• Use H.264 codec; MPEG-4 as a second option
• Audio: no less than a 48khz sample rate and no less than a 128kbps bit rate (96khz sample/384kbps bit rate is recommended for true HD audio)

Texture Viewers

• Gimp

  A DDS plugin is available for download. For BC7 support, see these instructions.

• Intel Texture Works Plugin

  Plugin for DDS formats for Adobe Photoshop.

• Paint.NET
• Nvidia Texture Tools

  Standalone tool and Adobe Photoshop plugin available.