CTD and Performance patch ENBoost (by Boris Vorontsov)

[redirishlord]

KeithInHanoi: thank you very much for making the pilgrimage to the Boracle of Delphi and bringing more clarity to the ongoing discussion.

I feel much more informed regarding editing my enblocal file with regard to performance management. 

[keithinhanoi]

  On 3/5/2014 at 2:35 AM, DoYouEvenModBro said:

Can you please help write/update the STEP ENB Guide, if you don't contribute already. 

 

  On 3/5/2014 at 2:43 AM, redirishlord said:

KeithInHanoi: thank you very much for making the pilgrimage to the Boracle of Delphi and bringing more clarity to the ongoing discussion.

I feel much more informed regarding editing my enblocal file with regard to performance management. 

I will see how I can contribute to the wiki, but fully knowing that there is a process to how it is updated that requires a number of people to agree on any edits.

 

Nevertheless, I'm getting very close to being ready to share the document explaining all the ENBoost memory performance features, which I'd like to call The Elder Boris Scrolls - ENBoost Demystified (tip of the hat to redirishlord / EProctor on that name).

 

But before that happens, I just have a few more questions to ask on the mountaintop, if I can get away with it...

Edited March 5, 2014 by keithinhanoi

[Octopuss]

I started experiencing awful amount of CTDs and couldn't get rid of them. After starting several new games, changing inis and whatnot, I remembered I edited enblocal.ini earlier today and changed ReservedMemorySizeMb from 256 (which I just had there since... the beginning?) to 64, because I read somewhere it should be low with lots of VRAM. Could this be the cause? Does anyone have any experience with this particular setting?

[EssArrBee]

That setting has been changed a few times. It used to be better at 128/256 and now it seems to be better at 64. With Boris you never know about these things, I doubt it would be the cause of CTD. Lots of crashes are probably the result of bad saves or to many mod conflicts.

[Octopuss]

I don't know, this is another new game with mods sorted with BOSS. Really frustrating stuff. I just CTDed again and there was still almost a GB of VRAM free, so you're probably right, must be something with mods. Good luck troubleshooting that to me.

[Kuldebar]

I have increased CTD's when that setting is too low. ReservedMemorySizeMb has been optimized supposedly to allow for a smaller value setting, but it is still largely dependent on the individual PC scenario. When set too small there's less room for background GPU memory support and so insta CTD when something activates in the background while you are playing the game.

[Octopuss]

Well, I still have a few things to try, like setting AF back to being managed by the game, going back to fullscreen mode, setting ipresentinterval back to 0 (I swear the game set it to 0 when it created the ini files) - but seeing how I didn't run even close to filling the VRAM, it must be some obscure mod problem. It's really weird, because the previous game crashed maybe 5 times in 30 hours.

[Kuldebar]

You can have all the VRAM in the world, but if you don't have a big enough amount of it reserved for non-game purposes, you will CTD.

[Octopuss]

I don't know, I tried 64, then 512, and also 256, and still got CTDs. They were extremely frequent with 64, but then again, maybe it wasn't related to this at all.

[Kuldebar]

  On 3/14/2014 at 10:48 PM, Octopuss said:

I don't know, I tried 64, then 512, and also 256, and still got CTDs. They were extremely frequent with 64, but then again, maybe it wasn't related to this at all.

Well, that's the unfortunate complexity of all this, none of the CTD causes are mutually exclusive. I just know one type of CTD can be caused by setting reserved memory too low, but there's a hellacious number of other causes prevalent as well, as you well know, lol.

[gollemodder]

The absolute best thing I've found to help is that memmory blocks .log.log.

I can't remember where I got it here but it provided the safe 1st block info to set ssme with. Truthfully though I had to remove or downsize a lot of my textures as well because of the limitations of my card in my sig. Its a really nice laptop but you can't expect miracles with all of the limitations imposed by the game engine itself and add on top all of the different hardware people use.

Either way, the combo of MO, ssme and enb boost {I can't run enb graphics, I use RLO and Imaginator with MCM for it} along with good enough textures and STEP know how has me running several profiles with little to no crashes anymore.

[Spock]

  On 3/5/2014 at 1:37 AM, keithinhanoi said:

ExpandSystemMemoryX64 is useful, but in practical terms it is much less useful than if you enable the main ENBoost memory manager (via speedhack=true & ReduceSystemMemoryUsage=true in enblocal.ini) with ExpandSystemMemoryX64 disabled.

From what I understood he posted ExpandSystemMemoryX64 is still important to enable certain memory management features (obscure again I know, sorry). Speedhack and ReduceSystemMemoryUsage are the main ENBoost features, ExpandSystemMemoryX64 is an addon feature to increase stability.

I did a medium google session on ReservedMemorySizeMB and couldn't find a good answer to what it does. Recommended values are ranging from 1024 to 64. I expect this feature to reserve memory for other processes (like Aero) so you do not run into problems when ENBoost preloads textures, I would like to understand what the difference to setting VideoMemorySizeMb lower then you actual video memory size is though. The default seems to be 64, I left it at that.

 

[edit]After a very brief testing, setting ReservedMemorySizeMB to 1024 gives me more noticable load stutters. From the windows performance monitor ram usage is still the same, plenty left. It could be explained by me hitting the remaining 2976 mb vram faster (got a 4 gb card, VideoMemorySizeMb=4000) or less vram being available for preload. This would require much more testing for confirmation though.

Edited March 27, 2014 by Spock

[keithinhanoi]

  On 3/27/2014 at 3:59 PM, Spock said:

From what I understood he posted ExpandSystemMemoryX64 is still important to enable certain memory management features (obscure again I know, sorry). Speedhack and ReduceSystemMemoryUsage are the main ENBoost features, ExpandSystemMemoryX64 is an addon feature to increase stability.

A preview from my Elder Boris Scrolls document:

 

ExpandSystemMemoryX64

true 

= this changes some memory allocation code in Skyrim (TESV.exe) to put some allocated data at the top of the Skyrim’s memory space, which, according to Boris, greatly reduces memory fragmentation and makes better use of all of the memory available. Although it was originally intended for 64-bit Windows users, it also seems to work for many people running 32-bit (x86) Windows, and for some people seems to reduce the frequency of CTDs.

 

false

= the memory allocation program code changes are not active

 

NOTES:

1) This setting does not increase the amount of memory that Skyrim (TESV.exe) can use. On 32-bit systems, Skyrim can use 2GB of memory, and on 64-bit systems it can use 3.1GB of memory. ENBoost, however, can increase the memory that can be used for 3D object geometry and textures by enabling ReduceSystemMemoryUsage.

 

2) As of ENB Skyrim v0.246, the included enblocal.ini file had ExpandSystemMemoryX64 set to false. Boris changed it to false because of too many complaints from users who experienced crashes with it set to true. The reasons for crashes with ExpandSystemMemoryX64=true are still unclear, but Boris suspects there are issues with some anti-virus programs or other programs that inject processes into TESV.exe.

Here's just some of the things Boris has said about it that led me to this explanation for ExpandSystemMemoryX64:

 

 

  Reveal hidden contents

ExpandSystemMemoryX64 - Introduced with ENBSeries 0.193

 

22 Jul 2013 - "ExpandSystemMemoryX64=true is not tested much feature which only work properly with x64 OS to free up more system memory."

 

22 Jul 2013 - "Anyway, i'm interesting is x64 fix enabled allow to reach memory usage above 3 gb, because it's designed for that."

 

24 Aug 2013 - Post by Phinix with explanations of all [MEMORY] settings

 

24 Aug 2013 - Boris' reply to Phinix to correct the explanations

 

20 Nov 2013 - "ExpandSystemMemoryX64 is system memory defragmentation feature, it was developed for Skyrim, but never tested on Fallout. Side effect can be crashes or freezes, this depends from game engine code. Variable is useful, but practically much less than when memory manager enabled completely without it, try without it enabled. It have nothing to do with vram. Why freezes occur in other cases - multithreading deadlocks, why they not happen without my memory manager i don't know and only can guess it's because of longer time to load each texture and mesh as i make their copies, validating them, compressing and pushing to enbhost.exe memory space."

 

21 Nov 2013 - "ReduceSystemMemoryUsage - enable memory manager. ExpandSystemMemoryX64 - enable defragmentation, but defragmentation of ram have absolutely different results compared to defragmentation of hard drive, performance do not change, access time do not change, but process can allocate greater size of memory blocks, if required. It work the most efficient with x64 systems, because on winxp only 3 gb of ram available with some setting in boot.ini."

26 Feb 2014 - "Well, ExpandSystemMemoryX64 is good only for certain things and when memory manager enabled, it almost useless. For me it's very important feature for stability, even with 2gb x86 winxp."

 

 

 

Using the utility VMMap, I have with 100% certainty confirmed that setting ExpandSystemMemoryX64=true results in memory allocations at the top of Skyrim's addressable memory space - and of special note this includes the memory blocks affected by Sheson's memory patch fix.

 

  On 3/27/2014 at 3:59 PM, Spock said:

I did a medium google session on ReservedMemorySizeMB and couldn't find a good answer to what it does. Recommended values are ranging from 1024 to 64. I expect this feature to reserve memory for other processes (like Aero) so you do not run into problems when ENBoost preloads textures, I would like to understand what the difference to setting VideoMemorySizeMb lower then you actual video memory size is though. The default seems to be 64, I left it at that.

 

[edit]After a very brief testing, setting ReservedMemorySizeMB to 1024 gives me more noticable load stutters. From the windows performance monitor ram usage is still the same, plenty left. It could be explained by me hitting the remaining 2976 mb vram faster (got a 4 gb card, VideoMemorySizeMb=4000) or less vram being available for preload. This would require much more testing for confirmation though.

I don't have a complete explanation for ReservedMemorySizeMB yet, but basically my understanding is that this is used to set aside an amount of VRAM that will not be touched by Boris' memory static allocation routines, and instead be used for dynamic allocation, which means the moving of resources between RAM & VRAM (and also possibly between VRAM and the RAM used by enbhost.exe). However, the purpose of the setting has been brought up in the ENB Forum thread on the new 0.252 binary release, so I'm asking more about it.

 

What is certain is that ReservedMemorySizeMB cannot be set higher than 1024 (1GB), and that Boris has recommended it be set to 512 if your video card only has 512MB of VRAM.

 

He has also just this week stated that setting it to 512 is a good idea for 64-bit Windows users who have a relatively low amount of VRAM compared to the resolution of the textures you have installed, but if your running 32-bit Windows, then ReservedMemorySizeMB should be set smaller to avoid CTDs.

 

As for VideoMemorySizeMb, it sets the amount of VRAM + RAM that is available for the ENBoost memory allocation. So, if it's set to be the same as your VRAM or lower, then presumably enbhost.exe won't be used, or not used much. If VideoMemorySizeMb is set higher than VRAM, then RAM will also be used (through instances of enbhost.exe) to cache textures and object geometry that's not currently being rendered. Although pulling cached data from RAM is obviously faster than getting it from your HDD or SSD, it's still slower than VRAM, and has to moved into VRAM to be used, so that's when you would get stuttering / lag.

 

I think this is why Boris normally recommends setting VideoMemorySizeMb to be equal or a little under your actual amount of VRAM. Still, it's worth doing some testing to see what the net effect is of setting it at the same as or lower / higher than your VRAM.

Edited November 5, 2014 by phazer11

[Kuldebar]

  On 3/28/2014 at 5:30 AM, keithinhanoi said:

 

He has also just this week stated that setting it to 512 is a good idea for 64-bit Windows users who have a relatively low amount of VRAM compared to the resolution of the textures you have installed, but if your running 32-bit Windows, then ReservedMemorySizeMB should be set smaller to avoid CTDs.

 

I completely missed that remark! If I had seen it I would have been all over that. It completely jives with my own experience in regard to that setting and my system. That's why the advice that had been given in some of the guides to set it "low" like 64 seemed to fly in the face of my own experience with the setting.

 

On the other hand, the last piece of information:

  Quote

I think this is why Boris normally recommends setting VideoMemorySizeMb to be equal or a little under your actual amount of VRAM. Still, it's worth doing some testing to see what the net effect is of setting it at the same as or lower / higher than your VRAM.

This causes me too freeze frame/slideshow in cities when I have it set to my Physical/Chip VRAM 1GB. Now if I set it at 3968 which is slightly lower than the DXDiag reported Total Vram of 4095, everything is smooth for me.

Edited March 28, 2014 by Kuldebar

[keithinhanoi]

  On 3/28/2014 at 5:44 AM, Kuldebar said:

I completely missed that remark! If I had seen it I would have been all over that. It completely jives with my own experience in regard to that setting and my system. That's why the advice that had been given in some of the guides to set it "low" like 64 seemed to fly in the face of my own experience with the setting.

 

On the other hand, the last piece of information:

This causes me too freeze frame/slideshow in cities when I have it set to my Physical/Chip VRAM 1GB. Now if I set it at 3968 which is slightly lower than the DXDiag reported Total Vram of 4095, everything is smooth for me.

 

Sometimes I feel like a vulture that has been circling in closer and closer for a while around a dying horse - but I can't beat a dead horse in this situation because it's not dead yet!

 

Regarding the whole DXDiag report values on video memory - I turns out that my theory about how ENBoost determines VideoMemorySizeMb when AutodetectVideoMemorySize=true was wrong.

 

What I should have been doing is enabling ENB graphics so I could take a look at the reported value for VideoMemorySizeMb in the ENB GUI. On my particular system, if I set AutodetectVideoMemorySize=true then the video memory size as shown in the ENB GUI is over 16GB! So it's clearly not based on the Windows video driver's definition of total video memory.

 

Based on your experience though, I'd have to guess that when you set VideoMemorySizeMb to 1024, the major drops of FPS in cities was indicative of constant textures / geometry having being unloaded from VRAM, and new data being loaded and cached from your SSD to VRAM. In other words, there wasn't enough "video memory" (VRAM + RAM) available to handle the load - and this is probably related to the resolutions of your textures and fact that you've got a 1GB card.

 

Then I have to wonder why it wouldn't help to increase your VideoMemorySizeMb above 4GB, since setting it above 1GB helped a lot, unless it's not actually getting used beyond a certain point. What happens when you set it above 4GB - any change in performance / stability?

Edited March 28, 2014 by keithinhanoi