User:TechAngel85/Benchmarks: Difference between revisions

From Step Mods | Change The Game
No edit summary
No edit summary
Line 1: Line 1:
Benchmark conditions: [[STEP:2.2.1#1.D._Performance_Benchmarking|STEP 2.2.1 Guide]]
[[User:Techangel85/SystemSpecs|Techangel85's System Specs]]
[[User:Techangel85/SystemSpecs|System Specs]]


The following table provides means +/- standard deviations for:
Benchmark conditions: [[STEP:2.2.1#1.D._Performance_Benchmarking|STEP 2.2.1 Guide]]<br>
 
 
The following table provides means ± standard deviations for:
* GPU temperature
* GPU temperature
* GPU load
* GPU load
* Dedicated VRAM (on-card memory) - Cap is 1GB on my system.
* Dedicated VRAM (on-card memory) - This caps at 1,024 MB on my system  
* System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering
* System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering (not available on my system; no senor to monitor this)
* FPS
* FPS


Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim. The build is full Baseline STEP 2.2.1. Benchmark 1 (though STEP 1.F.2 in the Guide) shows great performance, as expected. Benchmark 2 also shows good performance, even though there is some bleed-over of dedicated VRAM into system. No noticeable stuttering occurs at this point.
The table below represents the data from my system. Notice below where the standard deviation (SD) is rather large for dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. When SD decreases, this should correspond to an increase in swapped VRAM, which should beget stuttering or lag in gameplay. I typically start experiencing stutter when my VRAM exceeds 1000MB, so the goal for my system would be to keep the VRAM maximum between 900MB and 1000MB for smooth gameplay.


Benchmarks 3 - 5 all show a marked increase in VRAM usage and a decrease in FPS. The frame drop is a mininal issue, as it still remains close to 30 FPS on average. The VRAM, however, is problematic, as noticeable moderate stuttering occurred in these runs. Texture optimization using DDSopt did not alleviate the problem, and neither did ATTK. The final solution was to run the normal maps (*_n.dds) through DDSopt with big texture overhauls, clamping their size to 1k. Reducing the VRAM bleed-over by the additional 100 MB was enough to noticeably reduce stuttering to an acceptably minimal level.
As the GPU becomes VRAM taxed:
# FPS goes down
# Mean-dedicated VRAM consumption increases
# Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity
# Mean-system VRAM increases
# Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU)


At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency.
{{Notice Small|The numbers in the table represent [mean] ± [standard deviation]. This means that 67% of all measures are within the range of M ± SD, and 95% of all measures are within M ± 2SD, and 99% are within M ± 3SD.}}
{| class="sitetable"
{| class="sitetable"
! Run
! Assessment
! Stat
! GPU Temp. [?C]
! GPU Temp. [?C]
! GPU Load [%]
! GPU Load [%]
Line 21: Line 29:
! System VRAM [MB]
! System VRAM [MB]
! FPS
! FPS
|- style="text-align:right;" |
! style="text-align:left;" rowspan="2"|Windows background
! style="text-align:left;"| Mean
| 53.0||0.0||75.0||32.0||n/a
|-  
|-  
! style="text-align:left;"| Std Dev
! style="text-align:left; background-color:#567;"| Windows background
| 0.5||1.0||1.6||0.0||n/a
| style="background-color:#aaa;"| 45.6 ± 1.0
| style="background-color:#aaa;"| 0.07 ± 0.48
| style="background-color:#aaa;"| 122.0 ± 0.0
| style="background-color:#aaa;"| n/a
| style="background-color:#aaa;"| n/a
|-
! style="color:#ccc; text-align:left;"| (1) Bench 1.F.2 INI Tweaks
| 60.4 ± 1.6||65.9 ± 7.5||1,961.3 ± 21.6||89.7 ± 12.6||51.5 ± 8.6
|-
|-
! style="text-align:left;" rowspan="2"|Bench1 <br>After INI Tweaks(2.2.1)
! style="color:#ccc; text-align:left;"| (2) Bench 2.D Fixes 1
! style="text-align:left;"| Mean
| 61.1 ± 1.4||66.6 ± 7.6||1,876.6 ± 8.8||85.2 ± 64.6||52.7 ± 7.7
| 60.0||66.0||1,837.0||87.0||49.0
 
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (3) Bench 2.D Fixes 2
| 1.4||7.0||89.2||8.6||9.3
| 60.2 ± 1.7||66.1 ± 7.6||1,854.1 ± 78.0||91.2 ± 10.3||52.5 ± 8.1
 
|-
|-
! style="text-align:left;" rowspan="2"|Bench2 <br>After Step 2.F (2.2.1)
! style="color:#ccc; text-align:left;"| (4) Bench 2.E Interface
! style="text-align:left;"| Mean
| 61.0 ± 1.5||65.7 ± 7.7||1,868.4 ± 75.8||88.9 ± 7.7||52.6 ± 7.5
| 61.0||77.0||1,970.0||177.0||38.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (5) Bench 2.F Conflicting Graphics 1
| 1.1||7.0||15.8||7.9||10.2
| 61.8 ± 1.3||67.5 ± 5.9||1,918.3 ± 37.8||83.7 ± 6.7||49.9 ± 8.3
|-
|-
! style="text-align:left;" rowspan="2"|Bench3 <br>After Full 2.2.1 install
! style="color:#ccc; text-align:left;"| (6) Bench 2.F Conflicting Graphics 2
! style="text-align:left;"| Mean
| 61.0 ± 1.6||66.8 ± 6.1||1,970.0 ± 18.1||93.8 ± 11.2||49.0 ± 8.5
| 59.0||74.0||1,979.0||553.0||27.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (7) Bench 2.G Landscape & Environment 1
| 1.5||9.3||7.4||143.2||9.4
| 59.6 ± 1.5||69.1 ± 6.8||1,975.5 ± 14.8||208.0 ± 34.2||35.4 ± 11.2
|-
|-
! style="text-align:left;" rowspan="2"|Bench4 <br>2.2.1 + DDSop
! style="color:#ccc; text-align:left;"| (8) Bench 2.G Landscape & Environment 2
! style="text-align:left;"| Mean
| 59.9 ± 1.6||75.7 ± 7.6||1,974.9 ± 18.6||236.2 ± 35.5||32.4 ± 11.3
| 59.0||71.0||1,981.0||623.0||27.0
|-
! style="text-align:left;"| Std Dev
| 1.3||7.9||8.1||191.4||9.4
|-
! style="text-align:left;" rowspan="2"|Bench5 <br>2.2.1 + DDSopt + ATTK
! style="text-align:left;"| Mean
| 58.0||66.0||1,987.0||604.0||25.0
|-
! style="text-align:left;"| Std Dev
| 1.2||8.5||7.0||150.9||9.0
|-
! style="text-align:left;" rowspan="2"|Bench5 <br>2.2.1 + DDSopt (1k-constrained NM)
! style="text-align:left;"| Mean
| 59.2||66.8||1,980.0||465.9||26.7
|-
! style="text-align:left;"| Std Dev
| 1.4||8.1||12.6||141.5||9.7
|-
! style="text-align:left;" rowspan="2"|Bench5 <br>2.2.1 + DDSopt (1k-constrained NM) + ATTK
! style="text-align:left;"| Mean
| 58.9||71.8||1,981.4||468.6||26.9
|-
! style="text-align:left;"| Std Dev
| 1.3||9.0||8.8||156.8||10.1
|}
|}
Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.
# {{Fc|#e6b69e|'''(1) Bench 1.F.2 INI Tweaks'''}} - Performed after 1.F.2, 2.2.1 Guide, Pure Vanilla HD, no modifications
# {{Fc|#e6b69e|'''(2) Bench 2.D Fixes 1'''}} - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, no "Optimized HRDLC" (Vano89)
# {{Fc|#e6b69e|'''(3) Bench 2.D Fixes 2'''}} - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
# {{Fc|#e6b69e|'''(4) Bench 2.E Interface'''}} - Performed after 2.E Interface , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
# {{Fc|#e6b69e|'''(5) Bench 2.F Conflicting Graphics 1'''}} - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
# {{Fc|#e6b69e|'''(6) Bench 2.F Conflicting Graphics 2'''}} - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) (also includes my own tweaked 2k overhauls, which is the 2k DDSOpted version of both Skyrim HD & Serious HD with 1k-constrained normal maps for non-dungeion textures).
# {{Fc|#e6b69e|'''(7) Bench 2.G Landscape & Environment 1'''}} - Identical to Bench (6) but including L&E
# {{Fc|#e6b69e|'''(8) Bench 2.G Landscape & Environment 2'''}} - Identical to Bench (7), but SFO is DDSoptimized, without constraints

Revision as of 09:22, January 26, 2013

Techangel85's System Specs

Benchmark conditions: STEP 2.2.1 Guide


The following table provides means ± standard deviations for:

  • GPU temperature
  • GPU load
  • Dedicated VRAM (on-card memory) - This caps at 1,024 MB on my system
  • System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering (not available on my system; no senor to monitor this)
  • FPS

The table below represents the data from my system. Notice below where the standard deviation (SD) is rather large for dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. When SD decreases, this should correspond to an increase in swapped VRAM, which should beget stuttering or lag in gameplay. I typically start experiencing stutter when my VRAM exceeds 1000MB, so the goal for my system would be to keep the VRAM maximum between 900MB and 1000MB for smooth gameplay.

As the GPU becomes VRAM taxed:

  1. FPS goes down
  2. Mean-dedicated VRAM consumption increases
  3. Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity
  4. Mean-system VRAM increases
  5. Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU)

At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency. Template:Notice Small

Assessment GPU Temp. [?C] GPU Load [%] Dedicated VRAM [MB] System VRAM [MB] FPS
Windows background 45.6 ± 1.0 0.07 ± 0.48 122.0 ± 0.0 n/a n/a
(1) Bench 1.F.2 INI Tweaks 60.4 ± 1.6 65.9 ± 7.5 1,961.3 ± 21.6 89.7 ± 12.6 51.5 ± 8.6
(2) Bench 2.D Fixes 1 61.1 ± 1.4 66.6 ± 7.6 1,876.6 ± 8.8 85.2 ± 64.6 52.7 ± 7.7
(3) Bench 2.D Fixes 2 60.2 ± 1.7 66.1 ± 7.6 1,854.1 ± 78.0 91.2 ± 10.3 52.5 ± 8.1
(4) Bench 2.E Interface 61.0 ± 1.5 65.7 ± 7.7 1,868.4 ± 75.8 88.9 ± 7.7 52.6 ± 7.5
(5) Bench 2.F Conflicting Graphics 1 61.8 ± 1.3 67.5 ± 5.9 1,918.3 ± 37.8 83.7 ± 6.7 49.9 ± 8.3
(6) Bench 2.F Conflicting Graphics 2 61.0 ± 1.6 66.8 ± 6.1 1,970.0 ± 18.1 93.8 ± 11.2 49.0 ± 8.5
(7) Bench 2.G Landscape & Environment 1 59.6 ± 1.5 69.1 ± 6.8 1,975.5 ± 14.8 208.0 ± 34.2 35.4 ± 11.2
(8) Bench 2.G Landscape & Environment 2 59.9 ± 1.6 75.7 ± 7.6 1,974.9 ± 18.6 236.2 ± 35.5 32.4 ± 11.3

Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.

  1. (1) Bench 1.F.2 INI Tweaks - Performed after 1.F.2, 2.2.1 Guide, Pure Vanilla HD, no modifications
  2. (2) Bench 2.D Fixes 1 - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, no "Optimized HRDLC" (Vano89)
  3. (3) Bench 2.D Fixes 2 - Performed after 2.D Fixes , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
  4. (4) Bench 2.E Interface - Performed after 2.E Interface , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
  5. (5) Bench 2.F Conflicting Graphics 1 - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89)
  6. (6) Bench 2.F Conflicting Graphics 2 - Performed after 2.F Conflicting Graphics , DDSopt, Vanilla HD Textures, 1k-constrained, including "Optimized HRDLC" (Vano89) (also includes my own tweaked 2k overhauls, which is the 2k DDSOpted version of both Skyrim HD & Serious HD with 1k-constrained normal maps for non-dungeion textures).
  7. (7) Bench 2.G Landscape & Environment 1 - Identical to Bench (6) but including L&E
  8. (8) Bench 2.G Landscape & Environment 2 - Identical to Bench (7), but SFO is DDSoptimized, without constraints