User:Kelmych/Benchmark Testing

< User:Kelmych

Kelmych's System Specs

Benchmark conditions: STEP 2.2.1 Guide

The following table provides means ± standard deviations for 4 parameters being used for system benchmarking with STEP mods.:

• GPU temperature
• GPU load
• Dedicated VRAM (on-card memory) - This caps at 2 Gb on my system.
• System VRAM (swapped-to-system memory) - lower is better, I am still determing the value at which moderate/severe stuttering on my system
• FPS

In addition, the table includes two additional parameters:

• maximum dedicated VRAM use, and
• maximum dynamic VRAM use.

The last two reported values are the maximum of a five second running average of dedicated and dynamic VRAM, and are provided since they assist in building conclusions about the test.

Prior to running the benchmark tests all of the Vanilla textures (textures in Skyrim - Textures.bsa) were optimized using DDSopt as discussed in the DDSopt Guide. Selected STEP mods were also optimized prior to the test. The selection used the STEP Mod Optimization table from the DDSopt guide. Any mod with at least one Yes in the first 4 columns was optimized using DDSopt. Note that this criteria excluded all the large environment mods in the Conflicting Graphics section of the STEP guide; the authors of these mods have already done substantial optimization of their textures. The mods used were those in the Extreme STEP list with one exception. Instead of using only Skyrim HD and Serious HD in the Conflicting Graphics section of the list, a mix of textures selected from 5 mods were used to provide the textures that these two mods would have provided. The additional mods used are Skyrim Realistic Overhaul, Langleys Wood Metal and Stone Workshop, and Riften HQ. The specific textures included were the ones Cestral chose from these 5 mods for Texture Pack Combiner.

Being fairly impatient, for the benchmark tests I was initially interested in the results from benchmark tests before mods were loaded and after all mods were loaded. I could then determine roughly how much work remained in reducing VRAM load while running the actual Skyrim game. I also wanted to investigate two options to get some reduction in VRAM usage:

• compressing normal maps on a selected set of textures that z suggested would likely have a major effect on VRAM use - for this experiment I chose the normal maps from the environmental mod mix mentioned above along with those in Skyrim Floral Overhaul, and
• the effect, which I expected to be fairly small, in using cleaned up versions of the Skyrim bsa files other than just the Skyrim - Textures.bsa which had already been optimized. The description on the BSAopt nexus page shows that processing these other bsa files with BSAopt provides noticeable file size reduction through its removal of duplicates, and better read times since BSAopt uses better compression that takes less time in Skyrim to expand than the compression used by Bethesda. I did not use BSAopt on Skyrim - Sounds.bsa since there is a known problem with changing the compression on this file.

(z's conclusions - mine are not here yet) Notice below where the standard deviation (SD) is rather large for Dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. As Dedicated VRAM SD decreases, this should correspond to an increase in swapped (System) VRAM, which should beget stuttering or lag in gameplay.

As the GPU becomes VRAM taxed:

1. FPS goes down
2. Mean-dedicated VRAM consumption increases
3. Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity
4. Mean-system VRAM increases
5. Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU)

At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency.] Template:Notice Small

Assessment	GPU Temp. [?C]	GPU Load [%]	Dedicated VRAM [MB]	System VRAM [MB]	FPS	Max GP VRAM [MB]	Max Sys VRAM [MB]
Windows background	43 ± 0.2	0 ± 0.2	88 ± 0	89 ± 0	n/a	88	89
(1) Bench 1.F.2 INI Tweaks	58.5 ± 3.7	76.3 ± 17	692.2 ± 63.2	114.3 ± 0.5	59.9 ± 0.7	782	115
(2) Bench 2.M Gameplay	61.1 ± 2.0	89.9 ± 11.9	1647.2 ± 169.5	115.8 ± 1.4	55.9 ± 4.4	1908	117
(3) Bench 2.M Gameplay	58.9 ± 3.6	86.6 ± 18	1618.0 ± 179.1	116.5 ± 0.5	55.9 ± 4.6	1924.2	117
(4) Bench 2.M Gameplay	58.7 ± 3.9	87.9 ± 13.9	1,445.6 ± 177.8	118.1 ± 0.8	55.7 ± 4.7	1736	119
(4) Bench 2.M Gameplay	58.7 ± 3.9	88.3 ± 13.0	1,412.4 ± 182.1	112.7 ± 0.5	55.7 ± 4.8	1720.4	113

Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.

1. Bench 1.F.2 INI Tweaks - Performed after 1.F.2, 2.2.1 Guide, Optimized vanilla Skyrim - Textures (not HD), no modifications
2. Bench 2.M Gameplay - Performed after 2.M Gameplay , Optimized vanilla textures including HRDLC, Optimized selected mods, 2k-constrained
3. Bench 2.M Gameplay - Performed after 2.M Gameplay , Optimized vanilla textures including HRDLC, BSAopt optimized Skyrim vanilla resources, Optimized selected mods, 2k-constrained,
4. Bench 2.M Gameplay - Performed after 2.M Gameplay , Optimized vanilla textures including HRDLC, Optimized selected mods, 2k-constrained, normal maps reduced to 50% for environment mod mix plus Skyrim Flora Overhaul
5. Bench 2.M Gameplay - Performed after 2.M Gameplay , Optimized vanilla textures including HRDLC, BSAopt optimized Skyrim vanilla resources, Optimized selected mods, 2k-constrained, normal maps reduced to 50% for environment mod mix plus Skyrim Flora Overhaul