STEP:Mod Testing

Template:Warning Template:Notice Mod Testing for S.T.E.P. -- by the S.T.E.P. Team and Wiki Editors

Updated: 20:22:49 7 November 2012 (UTC)

GUIDE FORUM THREAD

Introduction

Computer Stability & Settings

Skyrim Stability & Settings

Skyrim Testing Profiles

Mod Organizing Programs

In Game Testing Procedure

Save Game Archive

In Game Tools

External Tools

Mod Review

Notes/ Ideas

[edit]

General Information

This guide will attempt to lay out good practices for testing mods to recommend for S.T.E.P. as well as for testing mods should this be asked for or required by TC and S.T.E.P. team. With a guide as complex as S.T.E.P., it is beneficial to reduce as much variation as possible, since complexity can breed a host of issues. This guide will try to recommend standardized testing techniques. The approach will be bottom-up, meaning recommendations will begin at the hardware level, progressing to software and then to mods.

Outline

• Examine the documentation of the mod
• Validate the archive package, naming scheme, directory structure, and installation procedure.
• Use TES5edit to determine quality of edits and scope of mod.
• DDSopt used to determine quality of textures.
• Examine files in game (to accomplish this we need to research a quick and easy way of identifying the actual name of the (texture, mesh, etc) file corresponding to what we see in game.
• Examine script source of mods for reliability and compatibility
• All mods with scripts must include methods for installing and uninstalling in game or at the very least stopping scripts with a console command.
• Review Papyrus logs for script issues
• Each mod is examined in multiple ways in game to confirm functionality and compatibility.
  • Test in various key spots of the world using savegames
  • Test in various key points of quest development using savegames
  • Test as a solo mod and with a full current STEP installation
  • Note qualitative changes to the game Appearance, Gameplay, etc
  • Note quantitative changes like VRAM, GPU, CPU, RAM, and FPS usage
• Contact mod authors with hard evidence and constructive criticism to improve any area's in need of improvement before becoming a STEP recommend.

[edit]

General Information

A stable computer is crucial to testing a new mod. Without a stable computer, it will much harder to determine if the mod itself is incompatible or buggy with other mods or Skyrim, or if the mod is incompatible with your computer. Here are some tips to increase stability.

Specifics

Device Driver Updates

A driver is a computer program that allows other computer programs to interact with hardware (device). Most drivers are update relatively frequently, especially drivers for graphic devices such as AMD's or Nvidia's graphic cards. Driver updates fix bugs and offer performance improvements. As such, it is quite important to have an up to date set of drivers for all your computer's hardware devices.

If your computer is pre-built, for example, you bought an off the shelf computer under a brand (vendor) name such as HP or ACER, a program(s) is usually included that assists in vendor specific driver updates. If this is the case it is important to use this program(s), as many brand name off the shelf computers have drivers specifically designed for their hardware devices, rather than generic drivers. Consult the manual or the website for information on how to do this on your computer.

If your computer is custom made then you probably dont need to read this, then there are several excellent programs to assist with this. A great tool is Driver Genius, which will download up to date list of drivers for almost all hardware devices, scan your computer with that list, and specify which need to be updated. The freeware version will do all of this. If you upgrade to the paid version, then the program will also download and install the drivers for you.

Overclocking

If you are currently overclocking your computer it is recommended setting everything to its default stock settings. This goes for the CPU, GPU, and RAM. Although the chances are low, it is possible for a new mod that is resource intensive to instigate stability issues, even if you previous overclock settings were stable. CPU and RAM overclocks can be managed from your motherboard's BIOS, and GPU overclocks are managed by a variety of programs. A recommended program for GPU overclock management is MSI Afterburner.

Ghosts in the Machine

Over time a computer will generate many redundant files that at the minimum will do nothing harmful, and at worst will cause stability issues. These files can include temporary files or redundant registry keys. It is recommended to use a program such as CCleaner to clear these files, as well as to fix registry issues.

To ensure proper registry key cleaning, it is recommended to restart your computer after program installation and uninstallation, and any computer driver related changes. It is important to restart your computer because Windows will do some registry cleaning of its own when the computer restarts. Once you have logged in, you can safely clean the registry as well as remove redundant files.

Disk Fragmentation

On hard disk drive (HDD), over time files that are regularly used by the computer will fragment, increasing read times and potentially causing stability issues. It is good practice to make sure the computers drives are defragmented before testing a mod. It is a good idea to defrag your computer after installing a new mod, especially with large mods such as texture overhauls. A recommended program is Defraggler, which allows full drive defragmenting or specific file and folder defragmenting.

If you use a solid state drive (SSD), please disregard the defragmentation advice detailed above. Defragmenting a SSD will have adverse affects and may damage the drive.

Background Processes

Background processes will take up resources, also potentially introducing stability issues when playing or testing Skyrim. A great program for eliminating as many of these background processes as possible is Game Booster. This will close redundant programs as well as services, and is optimized towards games. Another program that is useful is Process Lasso, which acts as a much expanded task manager. This will allow you to set Skyrim's CPU and I/O priority to high.

TESV.exe Properties

These settings will ensure that when Skyrim runs, Windows Aero Theme will be disabled, freeing up VRAM, since Aero may use between 64 to 128MB of VRAM. To access these settings navigate to your skyrim folder, right click on TESV.exe, and then click on the Compatibility tab. Tick the Disable Visual Themes and Disable Desktop Composition boxes, as shown in the image below.

[edit]

General Information

This section will describe best practices to ensure a stable Skyrim platform to work from. It will also detail various tips for debugging, error tracking, etc.

Specifics

Launcher Settings

Choose Medium or High settings in the Launcher. Then select a screen resolution. If you can, try to select 1920x1080, as this may be the most common, at 25%, screen resolution at the moment. However, if you are unable to do so, do not worry. Full HD (1920x1080) users are by no means a large majority. Other common screen resolutions to test at are 1680x1050, 1366x768, and 1280x1024. AA should be set to 4x, and AF to 8x. All view distances can be maxed out. Distant object detail should be high or medium, FXAA disabled (unchecked), and all water reflection options should be disabled (unchecked).

INI Settings

It is recommended to test a mod on medium or high settings, and once verified, test on ultra if possible. Testing on ultra may skew the results of the testing, especially if others may want to use the mod. The majority of Skyrim users will not have high end computer setups which are able to run on ultra settings.

The easiest way to achieve this is to use Wrye Bash's INI Edits function. Make sure you generate a default Skyrim.ini and SkyrimPrefs.ini. This can be done by deleting your current ini's (remember to back them up first) in your Documents folder. Once deleted, generate new inis by changing your settings to either medium or high in the Skyrim Launcher, as mentioned above. Once these default inis have been generated, back them up. Navigate to the INI Tweaks folder in Skyrim's Data folder. This folder will only be present if you have installed Wrye Bash. Create two new text files and name one Default Medium (or High) Mod Test Skyrim.ini, and the other Default Medium (or High) Mod Test SkyrimPrefs.ini. Make sure that you two text files change from .txt to .ini. Once these have been created, copy the contents of the newly generated ini's into their respective ini files you just created. The same should be done for you normal setting that you play Skyrim with, but name the resulting ini's differently, such as My Skyrim.ini, or My SkyrimPref.ini

This will allow you to easily switch between mod testing ini's and your normal ini settings. To apply the ini settings, right click on the ini file in Wrye Bash and chose apply. Be warned that if you would like to remove the applied ini settings, you cannot do so from Wrye Bash. For example, if you apply your normal ini settings to your mod testing ini setting, you will overwrite many ini settings. However, if you mod testing ini settings contain entries that do not exist in your normal inis, then these entries will remain. Only enteries that exist in both will be overwritten. It is easier just to copy your settings from your normal ini files, delete the text inside your Skyrim.ini and SkyrimPrefs.ini, and paste the copied text inside if you wish to revert back to your personal settings. This can also be done from your backups.

uGrids

This ini setting MUST be set to the default value of 5. For those of you who use a value of 7 and above, be warned that these higher values are 100% inherently unstable. It may be stable as a rock for you, but please don't test a mod with anything but the default value of 5. The chance that a new mod may cause stability issues with higher uGrids values is quite high, and again, most Skyrim players will be using the default value. To easily switch between uGrids values, create the relevant ini files in the INI Tweaks folder for WB as mentioned in the section above.

Should you attempt to test a mod on a save game that used to have a higher uGrids value, you will be unable to load it. There are three solutions. The first is to start a new game, which is highly preferred. The second is to use a Bat file to revert to the default value. These can be obtained from the DNA INI Generator site. Once there, navigate to the green panel called Downloads, and follow the instructions there. The third option is to use a selection of vanilla saves and STEP complete saves that can be downloaded under the save game section.

Default uGrids Settings (Skyrim.ini)

[General]

uGrids=5

uExterior Cell Buffer=32

Papyrus Logging

These ini tweaks will allow you to see if and mod issues are caused by errors in the mods scripts. Not all mods will contain scripts though. Again the easiest way to apply these is create a sperate ini file in Wrye Bash's INI Tweaks folder. This tweak is applied to the Skyrim.ini file. The log files can be found where your save games are, in the Scripts folder.

Papyrus Logging (Skyrim.ini)

[Papyrus]

bEnableLogging=1

bEnableTrace=1

bLoadDebugInformation = 1

Mod Tracing

At the moment, in game mod tracing is very basic. Currently for textures, you can open the command console by pressing the ~ key, and click on the texture that you are interested in. In the console command screen you will see its name and reference ID (eight hexadecimal characters, ie "001C8F02"). The name and/ or the reference ID may be useful in tracing the object or texture of interest.

[edit]

Skyrim Installation Swapper

This is an essential utility for mod testers.

To quote the author; "Skyrim Installation Swapper (SIS) let's you create up to ten individual skyrim installation "profiles" which can be swapped in and out easily, via GUI or hotkey. (Kinda like MOM/mTES4 do for Oblivion, but not copying all the vanilla game files, the addition of hotkeys to swap, pick your storage location, et al.)

A "profile" consists of any files in the skyrim game folder that are NOT a vanilla game or CK file, as well as the contents of the mydocs\my games\skyrim folder and the appdata\local\skyrim folder." - RedStapler/ Kristijn

The use of this utility is important regarding STEP, because the mod must be tested against a vanilla skyrim setup (no other mods at all), and a full STEP modded skyrim setup. This program makes it much easier to switch between the two without the hassle of multiple re installs.

SIS at Skyrim Nexus

Basic Testing Installation

Skyrim installed in a non windows directory. Preferably something like E:\Games\Steam\steamapps\common\skyrim All DLC's installed and cleaned with TES5edit. All Unofficial Patches installed All ini's with Vanilla Ultra settings.

[edit]

General Information

The mod should be first inspected by a mod organizer program such as Wrye Bash (WB) or Mod Organizer (MO). The Nexus Mod Manager (NMM) will not be enough due to inability of the program to show the internal structure of the mod package. Using these programs will let you know if the mod is suitably constructed, and in the case of WB, whether it conflicts with other mods.

Wrye Bash

For testing mods, WB is recommended. Regarding testing, the major advantage of WB is the ability to observe if two more mods conflict. The S.T.E.P. guide recommends installing mods in particular order. This is because the mod installed last will overwrite mods installed before if these mods modify the same file. With WB, it is very easy to observe which STEP mods conflict with the mod being testing. This can then be reported to TC or the S.T.E.P. team. The S.T.E.P. wiki has an excellent WB guide which covers everything from the programs installation to mod management.

In particular, when analyzing the zip or rar file for structure, refer to the "Package Structure & Interpretation" subsection in the BAIN Installers section.

Mod Organizer

MO can also be recommended, but for a different reason than WB. MO creates virtual folders when installing mods, which means it will keep the skyrim directory clean. This can help in troubleshooting. It can also create profiles, similar to SIS. If you use MO as your default mod program, then you can choose between SIS and MO.

[edit]

General Information

This section is the most important and will detail the steps required to test a mod in such a way that it can be recommended for test. It will be as simple and streamlined as possible. However, regarding the complexity, breadth and depth of mods for skyrim, this guide will never be able to cover all testing scenarios. Ultimately, this guide should cover the majority of mod testing scenarios.

Testing Procedure

To begin with, the mod you have chosen to test will be examined using several tools to asses viability.

Next, the mod will be tested on a vanilla skyrim profile.

Thirdly, the mod will be tested on a STEP complete skyrim profile.

Lastly, the mod testers is encouraged to write a small review.

External Assesment

Mod Package Structure Assessment

Using WB or 7zip, observe if the downloaded mod package is properly structured or configured for installation. Detailed information on proper mod package structure can be found here.

Conflicts & Overwrites Assessment

Using WB, take note of any conflicts that appear against vanilla skyrim files. Information on how to do so can be found under the "Package Installation" subsection of the BAIN - Installing Mods section of the WB guide.

For true conflict and overwrite information, it is best to extract the contents of the BSA archives, and use the extracted files instead of the BSA archives. Instructions on how to do this can be found here in the DDSOpt guide.

Plugin Assessment

Using WB, asses the plugin. Right-click on the .esp or .esm plugin and highlight "Mod Cleaning". In the drop down box, select "Scan for UDR's". This will indicate if the mod is clean or dirty, which should be noted in the review. Once additional programs such as TESVEdit, TESVJedit, and TESVGecko are released a more detailed explanation will be added.

Enable a Vanilla Skyrim Profile

Enable your vanilla version of skyrim using SIS or MO. Do not start the game yet.

Generate Default Skyrim.ini & SkyrimPrefs.ini Files

If you have not already done so using SIS or MO, generate default ini files on medium or high settings with the relevant modifications by following the instructions in the Skyrim Stability and Settings.

Papyrus Logging

During all stages of mod testing, papyrus modding should be enabled. Activate it by changing the relevant ini values in the Skyrim.ini file under Documents/My Games/Skyrim/ (for Win7).

Testing Locations

Testing Hall

Riften

Whiterun

Riverwood

Mod Specific

Console Commands

coc

tgm

tcl

tfc

tfc 1

[edit]

Vanilla Saves

S.T.E.P. Complete Saves

[edit]

General Information

Several in-game tools will be specified here. These tools will for the most part be mods that are particularly useful when it comes to mod testing.

Elys MemInfo

Elys MemInfo is a SKSE plugin that is very useful for displaying in game resource usage. Currently it is able to show RAM, Pagefile, Virtual Manager, Handles, and VRAM use and utilization by Skyrim. This is most useful when testing mods that affect graphics, such as texture mods, FXAA Injectors, ENB presets, and lighting and shadow tweaks and mods. For example, when testing out texture omptimizer, such as DDSOpt, it is possible to see how much VRAM use has decreased in certain areas. If you are hitting your graphic cards VRAM limit, it is useful to know that the optimizations have lowered VRAM use. Since Elys MemInfo displays this information in game, there is no need to minimize the game to bring up alternative resource use tools such as GPUZ or the Task Manager.

Requires SKSE

Alternative Start - Live Another Life

Alternative Start - Live Another Life is an extremely useful in game testing mod. As explained before, mod testing is best done by starting a new game. One of the issues with testing out one, or many, mods quickly when starting out a new game, is the requirement by Skyrim to watch and play the opening sequence of the game. This mod allows mod testers to simply skip the rather long introduction sequence. For reducing the time required for testing mods, Life Another Life is an invaluable addition. It is also a safe and clean mod, and will not break any quest lines.

No Boring Sleep Wait Menu

Skyrim requires a 31 day in game wait time before as many cells as possible can be reset. Cell resets are useful when mod testing for several reasons. A cell may be badly generated, or you wish to install a mod without starting a new game for testing. In both ceases a cell reset will potentially generate a proper cell, and the new mod may be integrated properly, especially if the mod affects spawns. This mod allows the tester to wait 31 days quickly and easily, which is not possible using the vanilla waiting system.

[edit]

General Information

This page contains a list of programs that will be recommended for use during the testing procedure.

Generic Programs

GPU-Z

GPU-Z is an amazing lightweight program that offers a plethora of ifnormation regarding any type of gpu you have, as well as allowing logging and graphing of gpu related processes such as vram load, temperature, clock speeds, and far more. This will be the main tool for testing graphic related mods, as its logging ability is very useful here.

Nvidia Inspector

Nvidia Inspector (NI), is a program that allows for in depth gpu information of Nvidia graphics cards. It allows for very fine profile tuning for skyrim, as well as monitors for a range of process such as gpu temperature and vram use.

ATI Tools

Skyrim Focused Programs

Skyrim Performance Monitor

TES5Edit http://forums.bethsoft.com/topic/1392239-wipz-tes5dumpfuture-tes5edit/

TES5Gecko http://www.darkcreations.org/forums/topic/1536-wipz-tesvgecko/

TES5Jedit http://forums.bethsoft.com/topic/1376366-wipz-jtes5edit-java-tes5edit-using-skyproc/

TES5Dump http://forums.bethsoft.com/topic/1392239-wipz-tes5dumpfuture-tes5edit/

esEdit http://code.google.com/p/esedit/

[edit]

Images

Videos

[edit]

- Camera mod

- DDSopt if testing texture mods, link to DDSopt guide.

- videos i. fraps ii. youtube

- gifs

-image galleries i. hosting sites

- compatebility checking i. tes5edit, tes5gecko when released ii. ck workarounds

- wrye bash i. esmfy esps to fix ctds/ tesvsnip.

- INI tweaks / WB ini tweaks i. medium default settings for stability a. maybe upload WB ini wteaks files. ii. papyrus logging iii. mod tracing (need to test)

- saves i. multiple save files from vanilla and STEP setup, heavy exterior areas, heavy interior. ii. link to total cell clear in troubleshooting guide iii. z92's suggestions for 60s run throughs.

- in game tools i. elysmeminfo ii. camera mod

- software tools i. nvidia inspector, skyrim performance monitor, gpu-z, FPS background booster.