User:Kelmych/Benchmark Testing

The following table provides means ± standard deviations for 4 parameters being used for system benchmarking with STEP mods.:

GPU temperature
GPU load
Dedicated VRAM (on-card memory) - This caps at 2 Gb on my system.
System VRAM (swapped-to-system memory) - lower is better, I am still determing the value at which moderate/severe stuttering on my system
FPS

In addition, the table includes two additional parameters, the maximum dedicated VRAM use and maximum dynamic VRAM use during the benchmark test. The reported values are the maximum of a five second running average of dedicated and dynamic VRAM, and are provided since they assist in building conclusions about the test.

Prior to running the benchmark tests all of the Vanilla textures (textures in Skyrim - Textures.bsa) were optimized using DDSopt as discussed in the DDSopt Guide. Selected STEP mods were also optimized prior to the test. The selection used the STEP Mod table from the DDSopt guide. Any mod with at least one Yes in the first 4 columns was optimized using DDSopt. Note that this did not include any of the large environment mods in the Conflicting Graphics section of the STEP guide as the authors of these mods have already done substantial optimization of their textures. The mods used were those in the STEP list with one exception. Instead of using only Skyrim HD and Serious HD, a mix of textures selected from 5 mods were used to provide the textures that these two mods would have provided. The additional mods used are Skyrim Realistic Overhaul, Langleys Wood Metal and Stone Workshop, and Riften HQ. The specific textures included were the ones Cestral chose from these 5 mods for Texture Pack Combiner.

(z's conclusions - mine are not here yet) Notice below where the standard deviation (SD) is rather large for Dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. As Dedicated VRAM SD decreases, this should correspond to an increase in swapped (System) VRAM, which should beget stuttering or lag in gameplay.

As the GPU becomes VRAM taxed:

FPS goes down
Mean-dedicated VRAM consumption increases
Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity
Mean-system VRAM increases
Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU)

At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency.] Template:Notice Small

Assessment	GPU Temp. [?C]	GPU Load [%]	Dedicated VRAM [MB]	System VRAM [MB]	FPS	Max Dedicated VRAM [MB]	Max System VRAM [MB]
Windows background	43 ± 0.2	0 ± 0.2	88 ± 0	89 ± 0	n/a	88	89
(1) Bench 1.F.2 INI Tweaks	58.5 ± 3.7	76.3 ± 17	692.2 ± 63.2	114.3 ± 0.5	59.9 ± 0.7	782	115
(2) Bench 2.D Fixes 1	61.1 ± 1.4	66.6 ± 7.6	1,876.6 ± 64.6	85.2 ± 8.0	52.7 ± 7.7
(3) Bench 2.D Fixes 2	60.2 ± 1.7	66.1 ± 7.6	1,854.1 ± 78.0	91.2 ± 10.3	52.5 ± 8.1
(4) Bench 2.E Interface	61.0 ± 1.5	65.7 ± 7.7	1,868.4 ± 75.8	88.9 ± 7.7	52.6 ± 7.5
(5) Bench 2.F Conflicting Graphics 1	61.8 ± 1.3	67.5 ± 5.9	1,918.3 ± 37.8	83.7 ± 6.7	49.9 ± 8.3
(6) Bench 2.F Conflicting Graphics 2	61.0 ± 1.6	66.8 ± 6.1	1,970.0 ± 18.1	93.8 ± 11.2	49.0 ± 8.5
(7) Bench 2.G Landscape & Environment 1	59.6 ± 1.5	69.1 ± 6.8	1,975.5 ± 14.8	208.0 ± 34.2	35.4 ± 11.2
(8) Bench 2.G Landscape & Environment 2	59.9 ± 1.6	75.7 ± 7.6	1,974.9 ± 18.6	236.2 ± 35.5	32.4 ± 11.3
(9) Bench 2.G Landscape & Environment 3	60.0 ± 1.7	76.1 ± 7.1	1,976.7 ± 16.2	231.3 ± 32.7	33.6 ± 10.6
(10) Bench 2.G Landscape & Environment 4	59.9 ± 1.5	76.4 ± 6.6	1,972.8 ± 20.2	203.0 ± 32.8	36.7 ± 10.7
(11) Bench 2.H Characters & Creatures	59.5 ± 1.6	71.0 ± 8.4	1,982.8 ± 11.1	273.8 ± 62.6	33.5 ± 12.0
(12) Bench 2.I Clothing & Equipment	59.4 ± 1.7	75.2 ± 7.6	1,979.8 ± 12.6	308.9 ± 75.2	30.8 ± 10.6
(13) Bench 2.J Animation & Effects	59.3 ± 1.6	69.5 ± 8.8	1,979.7 ± 12.0	323.4 ± 94.2	29.8 ± 11.3

Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.

Bench 1.F.2 INI Tweaks - Performed after 1.F.2, 2.2.1 Guide, Pure Vanilla HD, no modifications
Bench 2.D Fixes 1 - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, no "Optimized HRDLC" (Vano89)
Bench 2.D Fixes 2 - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
Bench 2.E Interface - Performed after 2.E Interface , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
Bench 2.F Conflicting Graphics 1 - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
Bench 2.F Conflicting Graphics 2 - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) (also includes my own tweaked 2k overhauls, which is the 2k DDSOpted version of both Skyrim HD & Serious HD with 1k-constrained normal maps for non-dungeion textures).
Bench 2.G Landscape & Environment 1 - Identical to Bench (6) but including L&E
Bench 2.G Landscape & Environment 2 - Identical to Bench (7), but SFO is DDSoptimized, without constraints
Bench 2.G Landscape & Environment 3 - Identical to Bench (8), but SFO is DDSoptimized and 1k-constrained NM
Bench 2.G Landscape & Environment 4 - Identical to Bench (8), but SFO is DDSoptimized and 1k-constrained NM (disabled Diverse Trees)
Bench 2.H Characters & Creatures 4 - Consistent with previous
Bench 2.I Clothing & Equipment - Consistent with previous, Armor and weapons packages DDSoptimized with 1k-constrained NM.
Bench 2.J Animation & Effects - Consistent with previous