STEP:Mod Testing

Step Guide-Mod Testing

An in-depth manual to mod testing for Step Modifications

WIP

This guide is still under development and awaiting updates.

To ensure consistent and reliable mod testing, it's essential to minimize variations that can lead to a range of issues and inconsistencies. This guide will outline standardized techniques for mod testing that Step Modifications employs for evaluating all mods featured in our official modding guides. The recommendations provided progress through system and software setup before delving into the mod testing process.

Understanding Mod Testing

Mod testing encompasses various aspects of mod evaluation. Within the context of helping out as a Step Mod Tester, it refers to the assessment of mods for potential inclusion in a curated Step Mod-Build Guide (aka, "Game Guide"). This evaluation involves a few key components:

Stability & Performance Testing: A mod's stability when integrated with other mods in the mod list is assessed. It aims to identify any conflicts or issues that may arise from mod interactions.
Functionality Testing: Mod functionality is scrutinized to ensure the mod performs as expected. Any deviations from the mod's intended or percieved purpose/scope are noted and addressed.
Mod Comparison: mods with similar content are compared to determine which one provides the best solution.
Lore and Mandate Assessments: mods are evaluated against Elder Scrolls lore to ensure they align with the established lore and adhere to the Step Mandate. This ensures that the mod fits seamlessly into the 'vanilla' game universe.

Mod testing can be time-consuming and meticulous... even tedious. Nevertheless, it can also be rewarding and enjoyable. It's ideally suited to those who are knowledgeable, pedantic, and genuinely dedicated to the process. Consequently, Mod Tester turnover rates are high.

System Preparation

It is essential to have a clean and current Step build by strictly following a Step Guide applicable to the mods being tested. It's equally important to have a stable and well-maintained PC. Without these, distinguishing between a mod-related issue and a system-related issue is all but impossible.

In summary, follow the Step Guide!

Mod Organizer Profile Setup

Step Modifications' officially supported mod manager is Mod Organizer, and it proves itself more helpful than other managers for mod testing. Follow these instructions to set up two new profiles in Mod Organizer for testing:

Open Mod Organizer and select the instance for the game for which testing is being performed
Select the icon
Copy the Step Game Guide profile
Name the new profile: Vanilla
- Make sure the Local Savegames box is not checked for the testing profiles
- Make sure the Automatic Archive Invalidation box is selected for the testing profiles
Name the new profile: GuideName vX.X Testing
- Make sure the Local Savegames box is not checked for the testing profiles
- Make sure the Automatic Archive Invalidation box is selected for the testing profiles
Select the newly created Vanilla profile and ensure no mods are enabled

Testing Preparation

All testing begins outside the game. Each of the following steps should be completed and notes taken from each before proceeding to in-game testing.

Step 1 - Forum Mod Topic

Read the mod's topic opening post (OP)

The mod's topic OP often contains an outline of what needs to be tested for a particular mod. Make note of this and use it when in-game testing begins. If nothing is outlined in the OP, it's likely the mod has not been tested or reviewed enough to include it. In such cases, testers will establish a outline by editing the OP. This outline should include what mod options should be tested, how to test, steps for making the mod compatible with other mods, etc. Most of this information will be gathered from the actual testing process below.

Step 2 - Nexus Page

Read the Nexus Page in its entirety

Description: make note of any special installation/uninstall instructions, any known internal or mod compatibility issues, and of any potential conflicts with supported DLCs and other mods within the Step Game Guide.
Posts: a complete read is not necessary; however, testers should attempt to develop a sense of user satisfaction. Posts can also reveal a list of possible bugs not listed within the Bugs section. Unless a mod is fully developed with few to no bugs being reported, some sort of interaction from the mod author is preferred to ensure the mod isn't abandoned.
Changelog: changelogs often provide a sense of what direction the mod it headed, how often issues are addressed, features being edited, etc.
Bugs: review any active bug reports. Too many bugs may disqualify a mod from being included within a Step Game Guide.

Step 3 - Validation

Validate the archive package using MO or 7zip

Observe if the downloaded archive is properly structured and configured for installation or not. For more complex installations, Mod Organizer FOMODs are preferred; however, MO can install BAIN type installations as well. FOMODs can be validated using an online validation tool, XML Validator, or by using FOMOD Validator. When installing mods using Mod Organizer, the bottom pane will also have a report of potential issues with the package.

Step 4 - xEdit

Inspect plugins using xEdit

Knowledge of xEdit and conflict resolution is required for this step. Using xEdit from MO, load all plugins for the entire Step build, including the mod(s) being tested if applicable. Review and note any issues of the mod's plugin(s) failing to carry forward changes from the DLC and CC content, the Unofficial Patches, or other mods. Also check for any existing conflicts with the current Step Patches. This is all considered in updating the Step Patches in development.

Step 5 - Installation/Uninstall

Validate installation procedure listed on the Nexus page.
Validate uninstallation procedure listed on the Nexus page.

Step 6 - Inspection

Inspect the installation in MO

Install the mod as it will be used for testing
Run LOOT and note information
Unless the mod is new on the scene, LOOT will recognize it and provide some valuable information such as if the mod is clean or dirty, requires other mods, etc.
Using Mod Organizer, take note of any asset conflicts that appear against vanilla files and other mods in the Step build.
Extract any BSAs to ensure the conflicts are being read. BSAs should not remain extracted unless required for further testing. Otherwise, remove the extracted BSA contents before continuing any testing.

Step 7 - Mandate

The final step is to evaluate the mod against the Step Modifications Mandate. Ensure the mod does not fall into the "...not about" sections. If it does, no more testing is required. Post findings why the mod doesn't fit the Mandate to the mod's topic. If the mod succeeds in passes this step, continue on to the remainder of testing the mod.

Testers should update the mod's topic OP on the forums with any relevant information gathered from the above testing.

In-Game Testing Procedures

This process is the most important and will detail steps required for testing mods in a consistent way so they may be recommended (or not) for a Step Game Guide. It will be as simple and streamlined as possible, however, with the complexity, breadth, and depth that mods can be, this process will never be able to cover all mod testing scenarios. Below we provide the most common of scenarios, however, when the scenario isn't covered best-judgment must be used.

NOTE:

The process for capturing comparison screenshots can be found following these procedures; which are for gameplay testing.

Pretesting Setup

Enable the testing profile
Testing for Step Game Guides should be completed from the profile set up, above, for that Guide in Mod Organizer: GuideName vX.X Testing.

Step 1 - The Vanilla Experience

It's important to be familiar with the original content whether it be textures, quests, game mechanics, etc. Therefore, if there is no familiarity with the changes being tested, the original content should be experienced first. If familiarity does exist this step may be skipped. To complete this step:

Load the Vanilla profile in MO, launch the game, and load the Vanilla save
Fast travel to or use the command console to load the area closest to the mod's altered content
Tip: before starting the content, make a new save that can be used later for faster testing
Play the game for a period of time to experience or view the vanilla content
Close the game and review findings for the next step

Step 2 - Experience the Mod

Relaunch Mod Organizer and select the Testing profile
Activate the mod being tested
Relaunch the game and load the Vanilla save or a save created for testing from Step 1
Allow the mods to initialize
Play the game for a period of time; assessing the mod:
- Does it work as intended?
- Does it look as intended?
- Does it fit in with the game's content/ambiance/environment?
- Are there any issues?
Close the game and review the findings

Final Step - Review

Gather the findings from all testing sources including the External testing.
Summarize a review and post this on the mod's topic on the Step Forums. Content to include are, but not limited to:
- Issues with file structure and installation
- Conflicts with vanilla or guide content (and solutions, if any)
- Whether the mod met or fell below expectations
- Any in-game issues
- Personal assessment of whether the mod should or shouldn't be included in a Step Game Guide

NOTE

Be mindful to remain objective. Do not write anything that will reflect upon the mod's author in a personal manner. Only review the craftsmanship, reliability, operation, etc of a mod, itself. Mods are assessed for guide inclusion, not for judgment against authors!

Presenting Testing Results

When testing mesh and/or texture based mods, comparison screenshots are required and are to be posted for the community to review. These shots are not for the staff to use for evaluation, since in-game testing should be completed, but rather they are for the community's benefit and staff reference.

Screenshots and videos are provided them from in-game sources. Do not use "studio" or "editor" applications! In-game lighting conditions and rendering can change the appearance of many game assets; therefore, it's important to capture these comparisons from within the game, itself. Any capture software may be used as long as the saved image is a 1:1 match in color and quality to the in-game image. Some recommendations can be found within the External Tools header below.

NOTE

Do not use PNG as the file format when posting compares on the forums. They tend to be large files, which in turn cause long loading times on pages. JPG/JPEG should be sufficient and is about one quarter the payload of PNGs. Only use PNG when transparency is required.

Capture Procedure

Comparison shots and videos need to include:

Vanilla reference
current Guide reference
mod reference

This process will allow captures to be of the exact same reference point, in each capture, with the only difference being asset replacements.

Active a HUD mod that auto-hides the HUD for all MO profiles, including Vanilla (i.e, for Skyrim active Immersive HUD). This will auto-hide the HUD so that it doesn't have to be done manually from the console. If the build doesn't include one of these mods, the HUD will have to be manually hidden or remain present in the captures.
Load the Vanilla profile and Vanilla save file
The mod being tested should only be active in the Testing profile.
While in-game, locate the vanilla reference(s) that need to be captured.
Once the reference is found, line the capture up in the frame
- Pay attention to lighting and angles to make sure the capture will be good for compares.
- Ensure nothing will interfere in the shot such as an NPC walking into the frame.
Once the capture is lined up, create a new save
Now load the save that was just created
Do NOT touch the mouse! Doing so may cause the capture to because misaligned.
Once loaded, ensure there is no temporary text on the screen and the HUD has been hidden
Manually hide the HUD (see commands below) or simply leave it in the capture when no mod exists for auto-hide.
Take the capture using the hot key for the capture software being used
Exit the game
Switch to the Game Guide profile
Repeat steps 6-9 above for the Guide captures
Switch to the Testing profile
Active the mod being tested and ensure its conflict resolution is as desired
Repeat steps 6-9 above for the mod captures

Posting Captures

Post the comparison capture(s) on the Step Forums. Repeat this process for all assets that require a comparison set for any given mod. When only one asset is being tested, it is encouraged to provide several captures from various angles/locations/lighting situations for a more complete review.

When posting captures on the Step Forums:

Please use the Editor buttons/functions on the forums to post.
Use a 3rd party service to host captures. Do not store your captures on the Step Wiki!
- Imgur and Postimage have proven to be an excellent, free image hosting sites for uploading screen captures to.
- Use YouTube for hosting video captures.
  Other services have proven themselves annoying for community members to use; most requiring an account to view the videos.

NOTE

The remainder of the guide contains helpful console commands, tips, tools, and more.

Helpful Console Commands

Provided is a list of helpful console commands to use while testing. They are specific to Bethesda games.
To open the in-game console press the ~ (tilde) key, which is normally located just below the Esc (escape/exit) key on a standard keyboard. Press it again to close the console.

Toggles

tmm 1
Toggles all map markers on; thus allowing fast travel to any location. Enter 0 in place of 1, to reset to default.
tgm
Toggles God Mode on/off. While in God Mode the PC will be invincible with unlimited health, magicka, and stamina.
tcl
Toggles collisions on/off; thus allowing the PC to walk through anything.
tfc
Toggles Free Camera Mode on/off. tfc 1 will pause the camera.
tm
Toggles menus on/off. Useful when taking captures.
tai
Toggles all NPC AI on/off. NPC will typically not move when toggled off.
tcai
Toggles all NPC Combat AI on/off. NPCs will interact normally, but will not actively attack the player.

Player Commands

player.additem formID ###
Adds the item to the player's inventory.

Replace formID with the baseID of the item. These IDs can be found here. Replace the ### with the number of items to add.

For example, to add gold in Skyrim: player.additem f 200
Tip: leading zeroes can be dropped from IDs. So 0000000f becomes just f.
player.addspell formID
Adds a specified spell to the player's spell list. Spell IDs can be found here.

Targeted Commands

Targeted commands require the reference to be selected in the console. To target a reference, use the mouse to click on it while the console is open.

disable
Disables a targeted, enabled reference.
enable
Enables a targeted, disabled reference.
Kill
Kills a targeted NPC.
Resurrect
Resurrects a dead NPC.
unlock
Unlocks a targeted object (doors, chests, etc).

Other Commands

coc locationName
Transports the player to a specified location. Replace locationName with the name of the location.

A list of Skyrim location names can be found here and here.

coc qasmoke is Skyrim's developer testing hall with containers for all game items.
killall
Kills all non-essential NPCs within the player's vicinity.
set timescale to ##
Changes the timescale of the game.

Change ## to the desired value. 20 is default. Setting below 10 may cause issues.

External Tools

Capture Software

Feel free to use the preferred program for captures. Some 3rd party software used by the staff include:

FPS Monitor
FRAPS - doesn't play well with some Windows features

Screen Captures

The preferred format for screen captures is PNG due to its lossless nature. JPEG can be used as an alternative and is recommended for large comparison sets to reduce page loads. Do not use GIF format for screen captures! The simple solution for screen captures for some games will be to utilize their own built-in capture feature:

Skyrim: use BethINI to set the capture path and use the PrtSc (print screen) key
Fallout 4: hotkey the photo mode capture key

Video Captures

The preferred formats (containers) for video captures are AVI, MKV, or MP4. Using one of these formats or a higher quality one is important for proper video captures in high definition. Capture in at least 1080p for fullHD, but always in a 16:9 format. Also use the following information for encoding:

Use H.264 codec; MPEG-4 as a second option
Audio: no less than a 48khz sample rate and no less than a 128kbps bit rate (96khz sample/384kbps bit rate is recommended for true HD audio)

Texture Viewers

Gimp
A DDS plugin is available for download. For BC7 support, see these instructions.
Intel Texture Works Plugin
Plugin for DDS formats for Adobe Photoshop.
Paint.NET
Nvidia Texture Tools
Standalone tool and Adobe Photoshop plugin available.