User:Z929669/Benchmarks

STEP:Core Benchmarks

z929669's System Specs

The following benchmarks were performed under a pure vanilla HRDLC install with varying degrees of STEP:Core v2.2.6 installed by Section. INI settings were maxed, and STEP INI Tweaks were all applied according to the Guide. Each benchmark run represents 202 seconds of the scripted new-game carriage ride and is highly consistent between equal runs (i.e., we should assume a very small margin of error).

STEP:Core & DDSopt Benchmarks - Since SD remains relatively flat, SD does not factor into the interpretation, because it is not a true SD of the mean. This is because the VRAM measures are continuous and independent, changing in response to gameplay. A true SD can only be calculated from many samples of like-averages (which is impractical, since each bench would need to be calculated a minimum of 3x). Therefore, if we assume that the mean dedicated VRAM is very accurate, we can disregard any notions of margins of error and take the diffs at face value (relatively small diffs are probably not important, but moderate to large diffs probably are). ("vanilla" = HRDLC)

NOTE:

Mean FPS was constant at just under 60 FPS for all runs

Original Benchmarks

z929669's System Specs

The following benchmarks were performed on the system linked above but for two exceptions:

My display adapters were Radeon HD-6850 in CrossfireX (2 GPUs)
I was using my old RAID0 HDD

Benchmark conditions: STEP 2.2.1 Guide

The following table provides means ± standard deviations for:

GPU temperature
GPU load
Dedicated VRAM (on-card memory) - This caps at 1,996 MB on my system due to crossfireX, which adds memory used on BOTH cards. Effective VRAM is still 1 GB.
System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering (on my system)
FPS

The data below are from my system, which is set up with Crossfire2X. This is important, because GPU-z includes RAM from both GPUs, which inflates the reported Dedicated VRAM by a factor of 2. Thus, it can be easily seen below that my card must be a 1 GB card, since my max VRAM below is nearly 2 GB. Notice below where the standard deviation (SD) is rather large for Dedicated VRAM when there the VRAM usage is at peak efficiency (operating below maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to spike upwards due to moments of high demand. As Dedicated VRAM SD decreases, this should correspond to an increase in swapped (System) VRAM, which should beget stuttering or lag during gameplay.

I don't begin to see stuttering until System VRAM begins to consistently hold around 300 MB. Consistency is important here, because it is the frequency of swaps between Dedicated VRAM and System VRAM that will ultimately contribute to stuttering. Although swap frequency is not directly measurable, the System VRAM standard deviation is a handy indicator for assessing the swap frequency. As the mean System-swapped VRAM increases, the standard deviation should steadily increase, which is exactly what can be seen happening below. Note that ideally, the System VRAM should remain near its initial value when running vanilla Skyrim (in my case, approx 85 MB). Specifically, if System VRAM mean and standard deviation remain fairly stable and low, then this indicates that VRAM demands are within the limits of Dedicated VRAM. As demand is increased, VRAM begins to swap with System, and Sytem VRAM mean and standard deviation both begin to increase. This marks the point where VRAM-induced stuttering can happen.

As the GPU becomes VRAM taxed:

FPS goes down
Mean-dedicated VRAM consumption increases
Dedicated VRAM SD increases at first and then decreases as it fills to capacity
Mean-system VRAM increases
System VRAM SD increases rapidly as it is relied upon in varying degrees and will theoretically decrease when a ceiling is hit (but stuttering occurs before this point, so it is not of interest here.)

At the point that the mean System VRAM SD begins to noticeably increase, stuttering becomes significant, indicating a high swap frequency.

The relationship between the summary statistics (M & SD) of VRAM measures and FPS is depicted in the following charts. Chart 1 shows trends for VRAM and FPS (the data have been transformed to exaggerate the relationships) with increasing Dedicated VRAM standard deviation, and Chart 2 shows the same trends with increasing System VRAM. This shows the inverse relationship between System VRAM and FPS. Note that GPU load and temperature seem to follow no predictable pattern. In my testing, noticeable stutter began occurring just after installing Section J, and the point along the benchmarking sequence is indicated by the black vertical line.