User:Kelmych/Benchmark Testing

From Step Mods | Change The Game
< User:Kelmych
Revision as of 23:54, January 27, 2013 by Kelmych (talk | contribs)

< User:Kelmych

Kelmych's System Specs

Benchmark conditions: STEP 2.2.1 Guide The following table provides means ± standard deviations for 4 parameters being used for system benchmarking with STEP mods.:

GPU temperature GPU load Dedicated VRAM (on-card memory) - This caps at 2 Gb on my system. System VRAM (swapped-to-system memory) - lower is better, > 300 MB produces moderate/severe stuttering (on my system) FPS

In addition, the table includes two additional parameters, the maximum dedicated VRAM use and maximum dynamic VRAM use during the benchmark test. These values are the maximum of a five second running average of dedicated and dynamic VRAM, and are provided since they assist in building conclusions about the test.

The data below are from my system, which is set up with Crossfire2X. This is important, because GPU-z includes RAM from both GPUs, which inflates the reported Dedicated VRAM by a factor of 2. Thus, it can be easily seen below that my card must be a 1 GB card, since my max VRAM below is nearly 2 GB. Notice below where the standard deviation (SD) is rather large for Dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. As Dedicated VRAM SD decreases, this should correspond to an increase in swapped (System) VRAM, which should beget stuttering or lag in gameplay.

However, this is not strictly true, as the data below indicate. I don't begin to see stuttering until System VRAM begins to consistently hold around 300 MB. Consistency is important here, because it is the frequency of swaps between Dedicated VRAM and System VRAM that will ultimately contribute to stuttering. The System VRAM SD is a handy indicator for assessing the swap frequency. As the mean increases, the standard deviation should steadily increase, which is exactly what can be seen happening below. Now, when even more VRAM is demanded even more frequently, the Mean System VRAM should peak and stableize at some max value, and the System VRAM SD will begin to decrease as this value is approached.

As the GPU becomes VRAM taxed:

FPS goes down Mean-dedicated VRAM consumption increases Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity Mean-system VRAM increases Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU) At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency.

     Notice:The numbers in the table represent [mean] ± [standard deviation]. This means that 67% of all measures are within the range of M ± SD, and 95% of all measures are within M ± 2SD, and 99% are within M ± 3SD.

Assessment GPU Temp. [?C] GPU Load [%] Dedicated VRAM [MB] System VRAM [MB] FPS Windows background 52.5 ± 0.7 0.5 ± 0.9 85.4 ± 8.6 32.6 ± 0.5 n/a (1) Bench 1.F.2 INI Tweaks 60.4 ± 1.6 65.9 ± 7.5 1,961.3 ± 21.6 89.7 ± 12.6 51.5 ± 8.6 (2) Bench 2.D Fixes 1 61.1 ± 1.4 66.6 ± 7.6 1,876.6 ± 64.6 85.2 ± 8.0 52.7 ± 7.7 (3) Bench 2.D Fixes 2 60.2 ± 1.7 66.1 ± 7.6 1,854.1 ± 78.0 91.2 ± 10.3 52.5 ± 8.1 (4) Bench 2.E Interface 61.0 ± 1.5 65.7 ± 7.7 1,868.4 ± 75.8 88.9 ± 7.7 52.6 ± 7.5 (5) Bench 2.F Conflicting Graphics 1 61.8 ± 1.3 67.5 ± 5.9 1,918.3 ± 37.8 83.7 ± 6.7 49.9 ± 8.3 (6) Bench 2.F Conflicting Graphics 2 61.0 ± 1.6 66.8 ± 6.1 1,970.0 ± 18.1 93.8 ± 11.2 49.0 ± 8.5 (7) Bench 2.G Landscape & Environment 1 59.6 ± 1.5 69.1 ± 6.8 1,975.5 ± 14.8 208.0 ± 34.2 35.4 ± 11.2 (8) Bench 2.G Landscape & Environment 2 59.9 ± 1.6 75.7 ± 7.6 1,974.9 ± 18.6 236.2 ± 35.5 32.4 ± 11.3 (9) Bench 2.G Landscape & Environment 3 60.0 ± 1.7 76.1 ± 7.1 1,976.7 ± 16.2 231.3 ± 32.7 33.6 ± 10.6 (10) Bench 2.G Landscape & Environment 4 59.9 ± 1.5 76.4 ± 6.6 1,972.8 ± 20.2 203.0 ± 32.8 36.7 ± 10.7 (11) Bench 2.H Characters & Creatures 59.5 ± 1.6 71.0 ± 8.4 1,982.8 ± 11.1 273.8 ± 62.6 33.5 ± 12.0 (12) Bench 2.I Clothing & Equipment 59.4 ± 1.7 75.2 ± 7.6 1,979.8 ± 12.6 308.9 ± 75.2 30.8 ± 10.6 (13) Bench 2.J Animation & Effects 59.3 ± 1.6 69.5 ± 8.8 1,979.7 ± 12.0 323.4 ± 94.2 29.8 ± 11.3 Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.

Bench 1.F.2 INI Tweaks - Performed after 1.F.2, 2.2.1 Guide, Pure Vanilla HD, no modifications Bench 2.D Fixes 1 - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, no "Optimized HRDLC" (Vano89) Bench 2.D Fixes 2 - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) Bench 2.E Interface - Performed after 2.E Interface , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) Bench 2.F Conflicting Graphics 1 - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) Bench 2.F Conflicting Graphics 2 - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) (also includes my own tweaked 2k overhauls, which is the 2k DDSOpted version of both Skyrim HD & Serious HD with 1k-constrained normal maps for non-dungeion textures). Bench 2.G Landscape & Environment 1 - Identical to Bench (6) but including L&E Bench 2.G Landscape & Environment 2 - Identical to Bench (7), but SFO is DDSoptimized, without constraints Bench 2.G Landscape & Environment 3 - Identical to Bench (8), but SFO is DDSoptimized and 1k-constrained NM Bench 2.G Landscape & Environment 4 - Identical to Bench (8), but SFO is DDSoptimized and 1k-constrained NM (disabled Diverse Trees) Bench 2.H Characters & Creatures 4 - Consistent with previous Bench 2.I Clothing & Equipment - Consistent with previous, Armor and weapons packages DDSoptimized with 1k-constrained NM. Bench 2.J Animation & Effects - Consistent with previous