User:TechAngel85/Benchmarks: Difference between revisions

From Step Mods | Change The Game
(Created page with "Benchmark conditions: STEP 2.2.1 Guide System Specs The following table provides means +/- stand...")
 
m (Text replacement - "{{alert small" to "{{AlertSmall‏‎")
 
(41 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Benchmark conditions: [[STEP:2.2.1#1.D._Performance_Benchmarking|STEP 2.2.1 Guide]]
[[User:Techangel85/SystemSpecs|Techangel85's System Specs]]
[[User:Techangel85/SystemSpecs|System Specs]]


The following table provides means +/- standard deviations for:
Benchmark conditions: [[STEP:2.2.1#1.D._Performance_Benchmarking|STEP 2.2.1 Guide]]<br>
 
 
The following table provides means ± standard deviations for:
* GPU temperature
* GPU temperature
* GPU load
* GPU load
* Dedicated VRAM (on-card memory) - Cap is 1GB on my system.
* Dedicated VRAM (on-card memory) - This caps at 1,024 MB on my system  
* System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering
* System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering (not available on my system; no senor to monitor this)
* FPS
* FPS


Note that the windows background numbers are included in the first rows. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim. The build is full Baseline STEP 2.2.1 with the exception that I used 2k rather than 1k textures for Skyrim HD and Serious HD (which costs a lot in terms of memory). Benchmark 1 (though STEP 1.F.2 in the Guide) shows great performance, as expected. Benchmark 2 also shows good performance, even though there is some bleed-over of dedicated VRAM into system. No noticeable stuttering occurs at this point.
The table below represents the data from my system. Notice below where the standard deviation (SD) is rather large for dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. When SD decreases, this should correspond to an increase in swapped VRAM, which should beget stuttering or lag in gameplay. I typically start experiencing stutter when my VRAM exceeds 1000MB, so the goal for my system would be to keep the VRAM maximum between 900MB and 1000MB for smooth gameplay.


Benchmarks 3 - 5 all show a marked increase in VRAM usage and a decrease in FPS. The frame drop is a mininal issue, as it still remains close to 30 FPS on average. The VRAM, however, is problematic, as noticeable moderate stuttering occurred in these runs. Texture optimization using DDSopt did not alleviate the problem, and neither did ATTK. The final solution was to run the normal maps (*_n.dds) through DDSopt with big texture overhauls, clamping their size to 1k. Reducing the VRAM bleed-over by the additional 100 MB was enough to noticeably reduce stuttering to an acceptably minimal level.
As the GPU becomes VRAM taxed:
# FPS goes down
# Mean-dedicated VRAM consumption increases
# Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity
# Mean-system VRAM increases
# Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU)


At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency.
{{AlertSmall‏‎|type=notice|text=The numbers in the table represent [mean] ± [standard deviation]. This means that 67% of all measures are within the range of M ± SD, and 95% of all measures are within M ± 2SD, and 99% are within M ± 3SD.}}
{| class="sitetable"
{| class="sitetable"
! Run
! Assessment
! Stat
! GPU Temp. [?C]
! GPU Temp. [?C]
! GPU Load [%]
! GPU Load [%]
Line 21: Line 29:
! System VRAM [MB]
! System VRAM [MB]
! FPS
! FPS
|- style="text-align:right;" |
! style="text-align:left;" rowspan="2"|Windows background
! style="text-align:left;"| Mean
| 53.0||0.0||75.0||32.0||n/a
|-  
|-  
! style="text-align:left;"| Std Dev
! style="text-align:left; background-color:#567;"| Windows background
| 0.5||1.0||1.6||0.0||n/a
| style="background-color:#aaa;"| 45.6 ± 1.0
| style="background-color:#aaa;"| 0.07 ± 0.48
| style="background-color:#aaa;"| 122.0 ± 0.0
| style="background-color:#aaa;"| n/a
| style="background-color:#aaa;"| n/a
|-
! style="color:#ccc; text-align:left;"| (1) Bench 1.F.2 INI Tweaks - Run 1
| 72.1 ± 6.1||97.6 ± 4.2||486.9 ± 38.0|| n/a ||46.8 ± 6.0
|-
! style="color:#ccc; text-align:left;"| (1) Bench 1.F.2 INI Tweaks - Run 2
| 80.7 ± 4.2||94.5 ± 6.0||547.3 ± 34.9|| n/a ||55.4 ± 4.7
|-
! style="color:#ccc; text-align:left;"| (2) Bench 2.D Fixes - Run 1
| 75.8 ± 5.2||97.8 ± 3.8||478.4 ± 40.1|| n/a ||52.0 ± 6.1
|-
! style="color:#ccc; text-align:left;"| (2) Bench 2.D Fixes - Run 2
| 79.9 ± 6.1||94.8 ± 5.7||465.5 ± 34.2|| n/a ||54.9 ± 5.1
|-
! style="color:#ccc; text-align:left;"| (3) Bench 2.E Interface - Run 1
| 73.2 ± 6.9||97.7 ± 4.7||481.2 ± 38.5|| n/a ||47.0 ± 5.9
|-
! style="color:#ccc; text-align:left;"| (3) Bench 2.E Interface - Run 2
| 80.4 ± 6.0||94.7 ± 5.8||464.4 ± 37.6|| n/a ||55.3 ± 4.9
|-
! style="color:#ccc; text-align:left;"| (4) Bench 2.F Conflicting - Run 1
| 75.6 ± 6.3||98.1 ± 6.1||589.3 ± 49.0|| n/a ||42.8 ± 6.0
|-
! style="color:#ccc; text-align:left;"| (4) Bench 2.F Conflicting - Run 2
| 81.5 ± 4.6||96.3 ± 4.7||572.2 ± 57.6|| n/a ||53.4 ± 5.1
|-
|-
! style="text-align:left;" rowspan="2"|Bench1 <br>After INI Tweaks(2.2.1)
! style="color:#ccc; text-align:left;"| (5) Bench 2.G Landscape & Environment - Run 1
! style="text-align:left;"| Mean
| 75.3 ± 6.5||98.5 ± 4.5||711.7 ± 60.5|| n/a ||38.9 ± 7.3
| 60.0||66.0||1,837.0||87.0||49.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (5) Bench 2.G Landscape & Environment - Run 2
| 1.4||7.0||89.2||8.6||9.3
| 81.4 ± 5.1||97.2 ± 7.6||677.0 ± 56.1|| n/a ||51.2 ± 5.2
|-
|-
! style="text-align:left;" rowspan="2"|Bench2 <br>After Step 2.F (2.2.1)
! style="color:#ccc; text-align:left;"| (6) Bench 2.H Characters & Creatures - Run 1
! style="text-align:left;"| Mean
| 76.0 ± 6.3||98.8 ± 0.9||834.4 ± 84.0|| n/a ||38.8 ± 7.4
| 61.0||77.0||1,970.0||177.0||38.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (6) Bench 2.H Characters & Creatures - Run 2
| 1.1||7.0||15.8||7.9||10.2
| 82.0 ± 4.5||97.2 ± 4.1||770.4 ± 71.7|| n/a ||50.3 ± 5.6
|-
|-
! style="text-align:left;" rowspan="2"|Bench3 <br>After Full 2.2.1 install
! style="color:#ccc; text-align:left;"| (7) Bench 2.I Clothing & Equipment - Run 1
! style="text-align:left;"| Mean
| 76.6 ± 6.2||98.7 ± 1.4||853.2 ± 98.3|| n/a ||38.6 ± 7.2
| 59.0||74.0||1,979.0||553.0||27.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (7) Bench 2.I Clothing & Equipment - Run 2
| 1.5||9.3||7.4||143.2||9.4
| 81.9 ± 4.6||97.0 ± 4.1||789.1 ± 103.8|| n/a ||50.5 ± 5.8
|-
|-
! style="text-align:left;" rowspan="2"|Bench4 <br>2.2.1 + DDSop
! style="color:#ccc; text-align:left;"| (8) Bench 2.J Animations & Effects - Run 1
! style="text-align:left;"| Mean
| 77.0 ± 5.8||98.6 ± 2.4||846.3 ± 103.6|| n/a ||38.6 ± 7.3
| 59.0||71.0||1,981.0||623.0||27.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (8) Bench 2.J Animations & Effects - Run 2
| 1.3||7.9||8.1||191.4||9.4
| 81.6 ± 4.5||97.2 ± 4.0||809.7 ± 89.1|| n/a ||50.3 ± 5.5
|-
|-
! style="text-align:left;" rowspan="2"|Bench5 <br>2.2.1 + DDSopt + ATTK
! style="color:#ccc; text-align:left;"| (9) Bench 2.K Clutter & Miscellaneous - Run 1
! style="text-align:left;"| Mean
| 76.7 ± 5.3||98.7 ± 1.8||871.3 ± 98.7|| n/a ||38.9 ± 7.3
| 58.0||66.0||1,987.0||604.0||25.0
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (9) Bench 2.K Clutter & Miscellaneous - Run 2
| 1.2||8.5||7.0||150.9||9.0
| 81.6 ± 4.5||97.2 ± 4.0||809.7 ± 89.1|| n/a ||50.5 ± 5.8
|-
|-
! style="text-align:left;" rowspan="2"|Bench5 <br>2.2.1 + DDSopt (1k-constrained NM)
! style="color:#ccc; text-align:left;"| (10) Bench 2.L&M Sounds & Gameplay - Run 1
! style="text-align:left;"| Mean
| 79.0 ± 4.4||98.6 ± 2.7||864.4 ± 107.1|| n/a ||38.7 ± 7.3
| 59.2||66.8||1,980.0||465.9||26.7
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (10) Bench 2.L&M Sounds & Gameplay - Run 2
| 1.4||8.1||12.6||141.5||9.7
| 82.0 ± 4.0||97.0 ± 5.1||818.0 ± 104.5|| n/a ||50.7 ± 5.8
|-
|-
! style="text-align:left;" rowspan="2"|Bench5 <br>2.2.1 + DDSopt (1k-constrained NM) + ATTK
! style="color:#ccc; text-align:left;"| (11) Bench Full Basline + ENB - Run 1
! style="text-align:left;"| Mean
| 79.6 ± 5.0||98.6 ± 1.7||912.8 ± 92.4|| n/a ||28.0 ± 3.8
| 58.9||71.8||1,981.4||468.6||26.9
|-
|-
! style="text-align:left;"| Std Dev
! style="color:#ccc; text-align:left;"| (11) Bench Full Baseline + ENB - Run 2
| 1.3||9.0||8.8||156.8||10.1
| 82.8 ± 4.4||98.4 ± 2.7||847.0 ± 108.7|| n/a ||34.0 ± 3.0
|}
|}
Note that the windows background numbers are included in the first row. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.
* {{Fc|#e6b69e|'''Run 1'''}} - New game opening sequence according to the benchmark on the STEP Guide.
* {{Fc|#e6b69e|'''Run 2'''}} - Using the mod, [https://skyrim.nexusmods.com/mods/15094 ScenicCarriages]. Benchmark from Whiterun to Riften in first person on jog speed. It's impossible to tell where stutter happens with the nature of this mod. It's jittery by default, but the numbers still come back solid.
# {{Fc|#e6b69e|'''(1) Bench 1.F.2 INI Tweaks'''}} - Performed after 1.F.2, 2.2.1 Guide, DDSopt Vanilla Textures plus Dawnguard, no modifications
# {{Fc|#e6b69e|'''(2) Bench 2.D Fixes'''}} - Performed after 2.D Fixes, DDSopt Vanilla Textures, no-constraints, no HRDLC, "Optimized HRDLC" only (Vano89)
# {{Fc|#e6b69e|'''(3) Bench 2.E Interface'''}} - Performed after 2.E Interface , DDSopt Vanilla Textures, DDSopt Interface textures, no-constraints, includes "Optimized HRDLC" (Vano89)
# {{Fc|#e6b69e|'''(4) Bench 2.F Conflicting Graphics'''}} - Performed after 2.F Conflicting Graphics , DDSopt Vanilla and Mod Textures, no-constraints Vanilla, 1k-constraints Mods
# {{Fc|#e6b69e|'''(5) Bench 2.G Landscape & Environment'''}} - Identical to Bench (4) but including L&E; SFO not optimized; Diverse Trees no included
# {{Fc|#e6b69e|'''(6) Bench 2.H Characters & Creatures'''}} - Identical to Bench (5) but including C&C
# {{Fc|#e6b69e|'''(7) Bench 2.I Clothing & Equipment'''}} - Identical to Bench (6) but including C&E; Run 1 is the first time VRAM when over 1000MB, Run 2 did not.
# {{Fc|#e6b69e|'''(8) Bench 2.J Animations & Effects'''}} - Identical to Bench (7) but including A&E
# {{Fc|#e6b69e|'''(9) Bench 2.K Clutter & Miscellaneous'''}} - Identical to Bench (8) but including C&M; Run 1 is running 1/6th of the time over 1000MB. Run 2 has yet to breach 1000MB.
# {{Fc|#e6b69e|'''(10) Bench 2.L&M Sounds & Gameplay'''}} - Identical to Bench (9) but including Sounds and Gameplay mods. This is the full Baseline install. This is the first time Run 2 has reached over 1000MB but for only two lines in the GPU-Z log.
# {{Fc|#e6b69e|'''(11) Bench Full Baseline + ENB'''}} - Full Baseline install but including SkyRealism ENB with Cinematic preset; DoF, SSAO & ENB AA off.

Latest revision as of 00:15, February 17, 2021

Techangel85's System Specs

Benchmark conditions: STEP 2.2.1 Guide


The following table provides means ± standard deviations for:

  • GPU temperature
  • GPU load
  • Dedicated VRAM (on-card memory) - This caps at 1,024 MB on my system
  • System VRAM (swapped-to-system memory) - lower is better, > 500 MB produces moderate/severe stuttering (not available on my system; no senor to monitor this)
  • FPS

The table below represents the data from my system. Notice below where the standard deviation (SD) is rather large for dedicated VRAM when there the VRAM usage is at peak efficiency (operating at near maximum capacity). However, the SD quickly goes down as available VRAM shrinks. This is because VRAM from second to second has nowhere to go but down, when it previously had room to get much higher OR much lower. When SD decreases, this should correspond to an increase in swapped VRAM, which should beget stuttering or lag in gameplay. I typically start experiencing stutter when my VRAM exceeds 1000MB, so the goal for my system would be to keep the VRAM maximum between 900MB and 1000MB for smooth gameplay.

As the GPU becomes VRAM taxed:

  1. FPS goes down
  2. Mean-dedicated VRAM consumption increases
  3. Mean-dedicated VRAM SD increases at first and then decreases as if fills to capacity
  4. Mean-system VRAM increases
  5. Mean-system VRAM SD increases rapidly as it is relied upon in varying degrees and then decreases as it fills to capacity (or the swap capacity of the GPU)

At the point that the System VRAM SD begins to noticeably decrease, stuttering should become significant, as this indicates a very high swap frequency.

Info-Logo.png
NOTE:
The numbers in the table represent [mean] ± [standard deviation]. This means that 67% of all measures are within the range of M ± SD, and 95% of all measures are within M ± 2SD, and 99% are within M ± 3SD.
Assessment GPU Temp. [?C] GPU Load [%] Dedicated VRAM [MB] System VRAM [MB] FPS
Windows background 45.6 ± 1.0 0.07 ± 0.48 122.0 ± 0.0 n/a n/a
(1) Bench 1.F.2 INI Tweaks - Run 1 72.1 ± 6.1 97.6 ± 4.2 486.9 ± 38.0 n/a 46.8 ± 6.0
(1) Bench 1.F.2 INI Tweaks - Run 2 80.7 ± 4.2 94.5 ± 6.0 547.3 ± 34.9 n/a 55.4 ± 4.7
(2) Bench 2.D Fixes - Run 1 75.8 ± 5.2 97.8 ± 3.8 478.4 ± 40.1 n/a 52.0 ± 6.1
(2) Bench 2.D Fixes - Run 2 79.9 ± 6.1 94.8 ± 5.7 465.5 ± 34.2 n/a 54.9 ± 5.1
(3) Bench 2.E Interface - Run 1 73.2 ± 6.9 97.7 ± 4.7 481.2 ± 38.5 n/a 47.0 ± 5.9
(3) Bench 2.E Interface - Run 2 80.4 ± 6.0 94.7 ± 5.8 464.4 ± 37.6 n/a 55.3 ± 4.9
(4) Bench 2.F Conflicting - Run 1 75.6 ± 6.3 98.1 ± 6.1 589.3 ± 49.0 n/a 42.8 ± 6.0
(4) Bench 2.F Conflicting - Run 2 81.5 ± 4.6 96.3 ± 4.7 572.2 ± 57.6 n/a 53.4 ± 5.1
(5) Bench 2.G Landscape & Environment - Run 1 75.3 ± 6.5 98.5 ± 4.5 711.7 ± 60.5 n/a 38.9 ± 7.3
(5) Bench 2.G Landscape & Environment - Run 2 81.4 ± 5.1 97.2 ± 7.6 677.0 ± 56.1 n/a 51.2 ± 5.2
(6) Bench 2.H Characters & Creatures - Run 1 76.0 ± 6.3 98.8 ± 0.9 834.4 ± 84.0 n/a 38.8 ± 7.4
(6) Bench 2.H Characters & Creatures - Run 2 82.0 ± 4.5 97.2 ± 4.1 770.4 ± 71.7 n/a 50.3 ± 5.6
(7) Bench 2.I Clothing & Equipment - Run 1 76.6 ± 6.2 98.7 ± 1.4 853.2 ± 98.3 n/a 38.6 ± 7.2
(7) Bench 2.I Clothing & Equipment - Run 2 81.9 ± 4.6 97.0 ± 4.1 789.1 ± 103.8 n/a 50.5 ± 5.8
(8) Bench 2.J Animations & Effects - Run 1 77.0 ± 5.8 98.6 ± 2.4 846.3 ± 103.6 n/a 38.6 ± 7.3
(8) Bench 2.J Animations & Effects - Run 2 81.6 ± 4.5 97.2 ± 4.0 809.7 ± 89.1 n/a 50.3 ± 5.5
(9) Bench 2.K Clutter & Miscellaneous - Run 1 76.7 ± 5.3 98.7 ± 1.8 871.3 ± 98.7 n/a 38.9 ± 7.3
(9) Bench 2.K Clutter & Miscellaneous - Run 2 81.6 ± 4.5 97.2 ± 4.0 809.7 ± 89.1 n/a 50.5 ± 5.8
(10) Bench 2.L&M Sounds & Gameplay - Run 1 79.0 ± 4.4 98.6 ± 2.7 864.4 ± 107.1 n/a 38.7 ± 7.3
(10) Bench 2.L&M Sounds & Gameplay - Run 2 82.0 ± 4.0 97.0 ± 5.1 818.0 ± 104.5 n/a 50.7 ± 5.8
(11) Bench Full Basline + ENB - Run 1 79.6 ± 5.0 98.6 ± 1.7 912.8 ± 92.4 n/a 28.0 ± 3.8
(11) Bench Full Baseline + ENB - Run 2 82.8 ± 4.4 98.4 ± 2.7 847.0 ± 108.7 n/a 34.0 ± 3.0

Note that the windows background numbers are included in the first row. This should be subtracted from all of the data following as a proxy for resources consumed only by Skyrim.

  • Run 1 - New game opening sequence according to the benchmark on the STEP Guide.
  • Run 2 - Using the mod, ScenicCarriages. Benchmark from Whiterun to Riften in first person on jog speed. It's impossible to tell where stutter happens with the nature of this mod. It's jittery by default, but the numbers still come back solid.
  1. (1) Bench 1.F.2 INI Tweaks - Performed after 1.F.2, 2.2.1 Guide, DDSopt Vanilla Textures plus Dawnguard, no modifications
  2. (2) Bench 2.D Fixes - Performed after 2.D Fixes, DDSopt Vanilla Textures, no-constraints, no HRDLC, "Optimized HRDLC" only (Vano89)
  3. (3) Bench 2.E Interface - Performed after 2.E Interface , DDSopt Vanilla Textures, DDSopt Interface textures, no-constraints, includes "Optimized HRDLC" (Vano89)
  4. (4) Bench 2.F Conflicting Graphics - Performed after 2.F Conflicting Graphics , DDSopt Vanilla and Mod Textures, no-constraints Vanilla, 1k-constraints Mods
  5. (5) Bench 2.G Landscape & Environment - Identical to Bench (4) but including L&E; SFO not optimized; Diverse Trees no included
  6. (6) Bench 2.H Characters & Creatures - Identical to Bench (5) but including C&C
  7. (7) Bench 2.I Clothing & Equipment - Identical to Bench (6) but including C&E; Run 1 is the first time VRAM when over 1000MB, Run 2 did not.
  8. (8) Bench 2.J Animations & Effects - Identical to Bench (7) but including A&E
  9. (9) Bench 2.K Clutter & Miscellaneous - Identical to Bench (8) but including C&M; Run 1 is running 1/6th of the time over 1000MB. Run 2 has yet to breach 1000MB.
  10. (10) Bench 2.L&M Sounds & Gameplay - Identical to Bench (9) but including Sounds and Gameplay mods. This is the full Baseline install. This is the first time Run 2 has reached over 1000MB but for only two lines in the GPU-Z log.
  11. (11) Bench Full Baseline + ENB - Full Baseline install but including SkyRealism ENB with Cinematic preset; DoF, SSAO & ENB AA off.