STEP:Mod Testing

Mod Testing Guide

Mod testing for the Step Guides

by: The Step Team  | Forum Topic

Template:TOC right It's beneficial to reduce as much variation as possible during mod testing, since variation can cause a host of issues and inconsistencies. Therefore, this Guide will lay out standardized techniques for testing mods. Step Modifications uses this process for all mods used within the official modding guides. Recommendations are given from at the hardware level up; progressing through software and setup, and then to mod testing.

What is Mod Testing?

Mod testing can be many different things; however, for the purposes of this Guide, mod testing is the testing of mods for inclusion to a mod list. This involves testing a mod's stability against other mods in the mod list, testing the mod's function to make sure it's working as intended, comparing it to other mods of similar content to find the best solution, and in the case for Step Modifications, checking it against Elder Scroll lore to make sure the mod is lore-friendly and meets the Step Mandates. Step Modifications has staff members, titled Curators, whom this job falls to, however, all staff usually do some amount of mod testing.

Mod testing can be long and tedious work at times, but it can be fun and enjoyable too. This is to say, mod testing is only for those that have a real dedication to it. As such, the turnover rate on users testing for projects tends to be rather high since these users can quickly become "worn out", if they don't realize what they've gotten themselves into.

Step Curators

To become a Curator, see here:

Overview of Testing Procedure

This is an overview of what the mod testing process involves.

• Each mod is examined in multiple ways in game to confirm functionality and compatibility:

  This is non-negotiable for all Step Curators. If a mod hasn't been tested in-game, it must be clearly stated so when posting reviews.

  • Test in relevant locations of the game world.
  • Test in relevant quests.
  • Test against other mods from the mod list. If there are issues, determine if there is an issue with the mod or a mod conflict (repeat as required).
  • Test with a new game and with adding the mod to an existing game (as required).
  • Note changes to the game's Appearance, Gameplay, etc.
  • Note quantitative changes like VRAM, GPU, CPU, RAM, and FPS usage.
• Post a review of the mod with personal findings with supporting data such as screenshots, benchmarks, etc. to the mod's forum topic.
• If issues are found, be courteous and contact mod authors with evidence and constructive criticism on how to improve any areas of the mod with issues or in need of improvement.

System Preparation

A stable system and game installation is critical before mod testing can begin. Without a stable computer and game installation, it will be much more difficult to determine if the mod itself is incompatible with the mod list, or if the mod is incompatible with the system it's running on. Below users will find some general guidance on how to increase system and game stability, as well as, how to setup a system for testing.

Device Drivers

A driver is software that allows computer programs to interact with hardware devices. Most drivers are updated relatively frequently, especially drivers for graphic devices such as video cards. Driver updates often fix bugs, expand compatibility, and provide performance improvements. As such, it is important to have up-to-date drivers for all hardware on the system.

If the system being used for mod testing is pre-built (i.e. an off-the-shelf computer under a brand name such as HP, ACER, or ASUS) an application is usually included that assists users in installing vendor specific driver updates. If this is the case, it's important to use this application since many pre-built systems have drivers specifically designed for their proprietary hardware devices, rather than using generic drivers. Consult the manual or the manufacturer's website for more information on how to do this on pre-built systems. An exception to this is the video card drivers. The latest drivers for these cards should always be obtained from the card company's website (AMD or Nvidia).

If the system is custom built, then there are several excellent applications to assist with driver updates. One great tool is Driver Genius. This program will download an up-to-date list of drivers for almost all hardware devices, scan the system using that list, and specify drivers that are using outdated drivers. The free version will provide users with a list of these outdated drives, while the paid version will also download and install the drivers. Users can manually search the web for the drivers using the list of outdated drivers the application provides.
It is not recommended to allow this program to update your graphics drivers. This is best done manually by downloading them directly from AMD or Nvidia.

NOTE:

Never use beta drivers for a system used for mod testing!

Overclocking

If the system being used testing is currently overclocked it is optional, but recommended to, revert all timings back to their default/stock settings. This includes the CPU, GPU, and RAM. Although the chances are low, it's possible for a new mod that is resource intensive to instigate stability issues even if the overclock settings were stable to begin with. CPU and RAM overclocks can be managed from the motherboard's BIOS or UEFI, and GPU overclocks are usually managed by a 3rd party program.

Bloatware and Unnecessary Programs

Bloatware and unnecessary programs can slow down a computer and use valuable resources in the background. These programs include browser tool bars, shareware programs installed during the installation of a separate program, bloatware that was installed by the computer manufacturer, and programs installed by the user which are no longer needed. Many of these programs automatically start at system boot, eating up system resources and slowing down the computer. It's recommended to remove these programs for a more stable system. There are a few tools which will help users do this:

• PC Decrapifier
• CCleaner

PC Decrapifier is a great tool that will help detect and remove many of the programs described above. It will scan both new and used computers for any bloatware and unnecessary programs and then run the uninstallers for these programs. This isn't foolproof; however, so manually scanning through the installed programs list is recommended. Remove any programs that are no longer needed on the system; however, be careful of uninstalling unknown programs because they might be important. For any unknown programs, Google the names and read about them before uninstalling.

CCleaner is a powerful system tool with multiple functions. At this point, the one of most of interest is the Startup list. This is found under the Tools tab and then by clicking on Startup. Programs listed here will start automatically when Windows is started. Users should review the list and disable any programs that are not required from the start of the computer. Most programs listed here can be started manually by opening the relevant programs, thus, not necessary from system boot and can be disabled. Be sure to review the list carefully! Some programs may be desired such as audio software, graphic card software, fan control software, printers, etc. Use best judgement when deciding what to disable and search the web for anything that is unknown.

Ghosts in the Machine

Over time a computer will generate many useless files that, at the minimum, will do nothing harmful or, at worst, could cause stability issues. These files can include temporary files left by programs that are no longer needed, redundant or unused registry entries and more. CCleaner is used to complete some of these steps.

System Registry Cleaning

NOTE

Step Modifications is aware of the debate on registry cleaning. Ultimately, cleaning the system registry is the user's choice. Are goal is to provide relatively safe instructions to complete this process.

Before cleaning the registry, it's recommended to restart the computer system. This is important because Windows will clean up some of the registry on its own when the system reboots. Once the system has been restarted, it is safe to proceed with the cleaning; however, be sure to back up the registry when asked in the event it needs to be restored.

Cleaning the system registry is an advanced process and is only recommended if users know what they are doing. It is not a click and done process! Some research is required. To use CCleaner for registry cleaning:

1. Run CCleaner and click on the Registry tab on the left.
2. Select options desired for scanning and click, Scan for Issues.
3. Once complete, research what it finds. Some items may not be desirable to delete.
4. When the research is complete, click Fix selected issues...
5. Create a backup of the registry before continuing. This backup will be required if the registry needs to be restored.
6. Close CCleaner.

Removing Unused Files

To clean the out old program and system files:

1. Run CCleaner, it should already be on the Cleaner tab.
2. Leave the default selections and click Analyze. It will scan the system for unused and unneeded files.
3. Once finished, click Run Cleaner and let it delete all those files.
4. Close CCleaner.

This is a point in this process to also do a thorough cleaning of old, personal files; removing them from the system. Users can backup files to an external device or cloud service, delete files no longer needed including old log files, downloaded software installers, and so on. Get the hard drives as clean as possible before continuing.

Spyware and Viruses

Ensuring the system is clean of spyware and viruses is critical to a stable system. The first step is to ensure there is an anti-virus installed with up-to-date definitions running at all times. Most users here could be considered power users and; thus, Microsoft Security Essentials should be all that is needed. It's lightweight and minimal and is perfect for users that follow "best practices". Avast is also well respected in the IT community, but is a bit heavier than MS Security Essentials. Both are free to use and there is no need for the paid version of Avast. The free version has everything anyone will ever need to protect their systems. User should run regular scans of their systems using the anti-virus software.

Now that the system is protected, ensuring it clean and free of any possible spyware is the next step. There are several free programs for this; however, one of the best is SUPERAntiSpyware. This program comes in several different versions, including a portable version. The free version will do the job well so no need to pay for it. SUPERAntiSpyware will scan for any potential spyware issues and clean them from the system. Close the program completely when it's done and uninstall it. It's not recommended to run the program in the background consistently as your anti-virus program would. Simply run it once a month along with the rest of the regular monthly maintenance on the system. Another great program is, Malwarebytes Anti-Malware.

As always with spyware and viruses, the best solution is prevention by following common best practices.

Disk Fragmentation

On hard disk drives (HDDs), files that are regularly used by the computer will fragment over time (located on separate parts of the disk). This increases read times and potentially causes stability issues. Defragmenting pieces fragmented files back together by placing them in order on the disks. It's always good practice to make sure the HDDs are defragmented before doing a round of testing. A recommended program is Auslogics Disk Defrag, which allows full drive defragmenting as well as specific file and/or folder defragmenting.

NOTE

If the system has a solid state drive (SSD), please disregard the defragmentation advice detailed above. Defragmenting a SSD offers no benefit and can significantly decrease the life span of the drive.

Skyrim Stability and Settings

Launcher Settings

Unless intimately familiar with the game settings and INIs, it is recommended to leave the launcher settings as they were set in the Step Modifications Guide. However, they may be changed to the hearts content, as long as the game remains stable. If the settings are changed, keep the following in mind:

• Select a screen resolution of 1920x1080 or higher. This will provide a true high definition (HD) testing environment.
• Some type of antialiasing should be enabled with at least an equivalent to 4x. SMAA can be used in place of normal AA
• AF should be set to 16x.
• All view distance sliders should be at least halfway; or equivalent to medium-high settings.
• Distant object detail should be medium, at the lowest; however, High or Ultra settings are preferred.
• FXAA should be disabled (unchecked).

INI Settings

It is recommended to test a mod on default high or ultra INI settings from BethINI.

uGrids

This INI setting MUST be set to the default values. For those who use higher values, be warned that these higher values are 100% inherently unstable. It may be stable as a rock for some setups; however, do not test mods with anything but the default value. The chances that a new mod may cause stability issues with higher uGrids values is very high, and again, most Skyrim players will be using the default value for uGrids.

Mod Testing Setup & Profiles

The required and only supported manager for mod testing is Mod Organizer. It has proven more helpful for mod testing purposes than any other manager. MO's feature set speaks for it, however, for mod testing it has some key advantages:

• Creates a virtual Data folder, which means it will keep the game directory completely clean.
• Installs mods in separate folders in its own directory, which translates to organized mods that are quick and easy to track.
• Ability to create multiple profiles without the need for a third party app. This is essential to mod testing.
• Full control over BSAs and the ability to extract them during installation, which allows the removal of some unnecessary or unwanted assets (when needed).
• Nexus integration for downloading, installing, and updating.
• Portable design that requires no installations. Just unpack and use.
• Automatic program updating. No more need to keep track of MO's updates.
• Mod version tracking and notification when mods are out-of-date.
• Profile specific INI files. No need to keep track of multiple INIs for multiple play-throughs. Also essential for mod testing.

Mod Testing Setup

• Game installed to a non-Windows directory.
• All DLC's installed and cleaned with xEdit.
• Step Modifications Guide installed to a separate profile.

General Setup

1. Ensure a clean install of Steam and game for which the testing is being completed for (non-Windows directory). See [[SystemSetupGuide|System Setup Guide.
2. Install and properly configure the Step Modification Game Guide for which the testing is being completed for.
3. FRAPS can be used for screenshots and performance benchmarking or any other program that doesn't interfere with the game.

Mod Organizer Profile Setup

Follow these instructions to set up the profiles in Mod Organizer for testing:

1. Create a new profile in MO.
   • Profile names suggestions are Vanilla and GuideName.
2. For testing:
   • Make sure the Local Savegames box is not checked for the testing profiles.
   • Make sure the Automatic Archive Invalidation box is selected for the testing profiles.
3. Start a new game from a Vanilla profile (only Extenders and Unofficial Patches active), play through until a the first point is reached that allows for free roaming of the game (e.g. for Skyrim play until leaving Helgen and save as soon as the dragon flies away after exiting the cave). These saves are the vanilla save files from which to test new mods, so don't overwrite them!

External Testing Procedures

External testing is where all testing starts. Use the following steps as a guide to testing any mod externally before moving on to testing in-game. Each step should be completed and notes taken from each before in-game testing occurs.

Step 1 - Opening Post/Workflow

Read the mod's topic opening post (OP).

Here, an outline of what needs to be tested for a particular mod can be found. Make note of this and use it when in-game testing begins. If nothing is outlined in the OP, take steps to establish an outline for testing by editing the OP. This outline should include what mod options should be tested, how to test, steps to take to make the mod compatible with other mods, etc.

Step 2 - Nexus Page

Read the Nexus Page in its entirety.

Read the mod's Nexus page description completely, as well as, the changelog (if provided). Make note of any special installation/uninstall instructions, any known issues or mod compatibility issues with the mod, and of any potential conflicts with the DLCs and/or other mods within the Step Modification Guide. The changelog often provides a sense of what direction the author is taking the mod and what might be expected from a mod and its author in the future; as far as how often they fix issues, features being adding/edited, etc. Update the OP on the forums with any relevant information.

Step 3 - Mod's Comments

Read the Nexus comments.

A complete read is not necessary; however, try to develop a sense of user satisfaction, and a list of possible bugs to attempt to confirm or deny (bugs can also be listed in the page's Bug tab, when provided). This is also a good way to find out how active the author is with the mod. No replies from the author over a long period of time usually indicates they have been absent for a while and/or may no longer be actively supporting the mod. Unless a mod is fully developed with few to no bugs being reported, some sort of interaction from the mod author is preferred to ensure the mod isn't abandoned.

Step 4 - Documentation

Examine the documentation included with the mod.

Read the Readme and any other documentation that comes with the mod. Note any lack of clarity, installation/uninstall directions, and general completeness. If the mod does not include documentation, please make note of this as well for the final review.

Step 5 - Validation

Validate the archive package, naming scheme, and directory structure using MO2 or 7zip.

Observe if the downloaded mod package is properly structured and configured for installation or not. Mod Organizer can also use BAIN type installations; however, FOMODs are preferred which use XML. FOMODs can be validated using an online validation tool, XML Validator, or users can use FOMOD Validator.

Step 6 - xEdit

Inspect the mod in xEdit to determine quality.

Note any issues of the mod not carrying over changes from DLCs and/or from the Unofficial Patches. Also note any conflicts with mods already in the Step Modifications Guide. Knowledge of xEdit and conflict resolution will be required for this step. Add relevant information to the OP on the forums.

Step 7 - Installation/Uninstall

• Validate the installation procedure listed on the Nexus page.
• Validate the uninstallation procedure listed on the Nexus page.

Step 8 - Inspection

1. Inspect in MO2

   Using Mod Organizer, take note of any asset conflicts that appear against vanilla files and other mods in the Guide. Also extract any BSAs to ensure the conflicts are being read. BSAs should not remain extracted unless comparisons are needed for things such as textures. Otherwise, remove the extracted BSA contents before continuing with testing.

2. Run LOOT

   Unless the mod is new on the scene, LOOT will recognize it and provide some valuable information about the mod such as if the mod is clean or dirty, requires other mods, etc. Include this information, if any, in reviews.

Step 10 - Mandate

After reviewing the mod and before in-game testing occurs, the final step is to pit the mod against the Step Modification Mandate. To do this, review what Core and Extended are and are not about. Ensure the mod does not fall into the "...not about" sections. If it does, no more testing is required. Review and post your findings why the mod doesn't fit the Mandate. If the mod passes this step, continue on to testing the mod in-game.

In Game Testing Procedures

This section is the most important and will detail the steps required for testing a mod in a consistent way so it can be recommended (or not recommended) for a Guide. It will be as simple and streamlined as possible, however, with the complexity, breadth, and depth that mods can be, this step will never be able to cover all mod testing scenarios. Below we provide the most common of scenarios, however, when the scenario isn't covered best-judgment must be used.

Screen & Video Captures

NOTE

When screenshots or videos are needed for comparisons, provide them from in-game sources. Do not use "studio" applications to provide shots and/or video. In-game lighting conditions and rendering can change the appearance of many textures compared to studio applications. Therefore, it is very important to capture these comparisons from within Skyrim itself.

FRAPS is an good program for both screen and video captures in-game, as well as, capturing FPS data; however, feel free to use the preferred program for captures. The paid version of FRAPS provides more functionality and is recommended. PNG is the preferred format for screen captures due to its accuracy in capturing correct colors, saturation, tints, etc. JPEG is not recommended because some of the image's originality can be lost; however, it can be used if PNG isn't an option. Do not use GIF format for screen captures!

For video captures, AVI, MKV, and MP4 file formats (containers) are best. Using one of these three formats or a higher quality one for capturing video is very important for proper captures in high definition. Use the H.264 codec, if possible, and MPEG-4 as a second option. Video for true HD should be captured in at least 1080p. Audio for video captures, if it can be set, should be no less than a 48khz sample rate and no less than a 128kbps bit rate (96khz sample/384kbps bit rate is recommended for true HD audio). Use this information for encoding edited videos for compares as well. Adobe Premiere Elements is excellent for this, but rather expensive.

When uploading captures for posting compares on the forums, please use a 3rd party service to host captures. Do not store your captures on the STEP wiki! Imgur and Postimage have proven to be an excellent free image hosting sites for uploading screen captures to. Please use the editor functions on the forums to post compares. For posting video captures, please use YouTube. Other services have proven themselves annoying for community members to use; most requiring an account to view the videos.

Quick tip: Unless the monitor resolution is 1920x1080 or higher (and capturing at that resolution), do not encode your videos in 1080p. The result will be blurry due to upscaling. If below 1920x1080, 1600x900 for example, encode videos in 720p.

In-Game Mod Assessment

Pretesting Setup

Enable the testing profile
Testing for Step Modification Guides should be done on a profile set up for that Guide in Mod Organizer.

Testing Locations
Below are some recommended testing locations for Skyrim. Locations for other games should cover the majority of the games assets within them:

• QASmoke - Testing Hall (great for gathering needed items, but not great for screenshots or video due to lighting)
• Riften
• Whiterun
• Riverwood
• Windhelm
• Mod Specific Location

Testing Step 1 - The Vanilla Experience

It's important to be familiar with how the original content works whether it be a texture, a quest, a game mechanic, etc. Therefore, if the user is not familiar with the mod changes being tested, they should experience the original content first. This is also necessary for texture comparisons, which must have a vanilla shot for reference. To do this:

1. Load the vanilla profile created earlier and launch the game.
2. Fast travel or COC to the area closest to the mod's in-game location will be or find a good example of the assets being altered.
3. Play the game for a period of time or make a save for comparison shots to experience or view the vanilla content.
4. Close the game and review the findings for the next step.

Testing Step 2 - Experience the Mod

1. Relaunch Mod Organizer and select the Step Modification Guide profile for further testing.
2. Activate the mod being tested.
3. Relaunch the game and load the vanilla save again or the save mod for comparison shots.
4. Allow the mods to initialize.
5. Play the game for a period of time assessing the mod or take your shot for the compare.
   • Is it working as intended?
   • Does it look as intended?
   • Does the mod fit in with the game's content/ambiance/environment?
   • Are there any issues?
   • etc...
6. Save the game (new save) and reload it. Have any issues appeared?
7. Close the game and review the findings.

Final Step - Review

1. Gather the findings from all testing sources including the external testing.
2. Summarize them into a review and post this review on the mod's topic on the forums. Things to include are, but not limited to:
   • Issues with file structure and installation
   • Conflicts with vanilla or Guide content (and solutions, if any)
   • Whether the mod met or fell below expectations
   • Any in-game issues
   • Personal assessment of whether mod should or should not be included in the Step Modifications Guide

NOTE

Be mindful to remain objective. Do not write anything that will reflect upon the mod's author in a personal manner. Only review the craftsmanship, reliability, operation, etc of the mod itself. We're assessing mods for Guide inclusion, not judging authors and their skills!

Denied Mods (optional)

At times there will be mods that do not make it pass evaluations for reasons other than Mandate violations or being less preferred over another mod. For these mods, consider contacting the mod's author with the findings so that they may improve their mod, or potentially provide solutions which will work for Guide inclusion. This is entirely optional. Some mod authors will appreciate the contact; however, others will not so use best judgement.

Capturing Texture Comparison Sets

When testing mesh/texture based mods, comparison screen shots should to be taken and posted for the community. These shots are not for the staff to use for evaluation, but rather for the community's benefit and our reference. These shots need to include a Vanilla shot, current Guide content shot, and a shot of the content of the mod being tested. To capture screen shots that work best for these comparisons use the best practices outlined below. This outline will allow the shots to be of the exact same screen, in each shot, with the only differences being the asset replacements.

1. Active Immersive HUD, if not already, for all MO profiles used in the testing, including the vanilla profile. This will auto hide the HUD so that it doesn't have to be done manually from the console.
2. Load the vanilla profile and vanilla save file. The mod being tested should not be active.
3. While in-game, locate the vanilla texture(s) that need to be captured. Explore until the texture(s) are found or fast travel or COC to the texture's location, if known.
4. Once the texture is found, line the shot up in the frame. Pay attention to lighting and angles to make sure the shot will be good for compares.
5. Ensure there is nothing that will interfere in the shot such as an NPC walking into the frame.
6. Save the game with a new save, if not loading from a previous save.
7. Load the save and Do NOT touch that mouse!
8. Once the game loads, make sure there is no text on the screen and Immersive HUD has hidden the HUD elements. When all is good, press the screen capture key for the program being used to capture.
9. Now exit the game.
10. Switch to the Guide profile.
11. Repeat steps 7-9 above for the Guide shots.
12. Now in the current Guide profile, active the mod being tested and ensure it is overwriting the desired assets.
13. Repeat steps 7-9 to capture the shots from the mod being tested.

If completed correctly, a set of three compares should exist: a vanilla shot, a Guide shot, and a shot with the mod included. These can be used to post comparison shots on the forums. Simply repeat this process for all assets that require a comparison set for any given mod. When only one texture is being tested, it is nice to provide several compares from various angles/locations/lighting situations for a more complete comparison set.

Tools, Hints, and Utilities

Helpful Console Commands

Below is a list of helpful console commands to use while testing. Use these to expedite the testing while in-game. They are specific to Skyrim and other Bethesda game. To open the in-game console press the [ ~ ] (tilde) key, which is normally located just below the ESC key on a standard keyboard. Press it again to close the console.

Toggles

• tmm 1

  This will toggle all map markers on; thus allowing you to fast travel to any location quickly. Enter 0 in place of 1, to reset to default.

• tgm

  Toggles God Mode on/off making you invincible. Health, magicka and stamina will not run out either with enabled.

• tcl

  Toggles collisions on/off. Don't use while falling or you may crash to desktop.

• tfc

  Toggles Free Camera Mode on/off so you can fly around the environment. 'tfc 1' will pause the camera.

• tm

  Toggles menus on/off. Useful when taking screenshots. Immersive HUD/Immersive HUD - iHUD Special Edition can be used to achieve this without having to use the console.

• tai

  Toggles Artificial Intelligence so characters will not react to the player.

• tcai

  Toggle Combat Artificial Intelligence so NPCs will not attack the player.

Player Commands

player.additem formID ###

Adds the item to the player's inventory. Replace 'formID' with the item code of the item. Codes can be found here. Replace the '###' with the number of items to add. For example, to add gold: player.additem f 200 (this adds 200 gold to the player's inventory.)
Tip: leading zeroes can be dropped in the item code.

player.addspell formID

Adds a specified spell to the player's spell list. Spell codes can be found here.

Other Commands

• coc locationName

  Transports the player to a specified location. Replace the locationName with the name of the location. A list of location names can be found here and here.

• coc qasmoke

  Transports the player to the developer testing hall.

• unlock

  Unlocks the targeted object (doors, chests, etc). To target an item/object click on it while the console is open.

• Kill

  Kills a targeted enemy.

• killall

  Kill all non-essential NPCs within the player's vicinity.

• Resurrect

  Resurrects a dead target.

• set timescale to ##

  Changes the timescale of the game. 20 is the default setting. Setting this below 10 can cause issues.

External Tools

Intel Texture Works Plugin - allows you to open, edit, save DDS files in Adobe Photoshop using the latest formats (including BC7).

Gimp users can download a DDS plugin. For BC7 support, Gimp users can see these instructions.