SKYRIMLE Memory Blocks Log (by Sheson)

[hishutup]

This is a situation I had....

I was told that block1 shouldn't need to be raised because it goes to block 2 and block2 didn't need to be raised because skyrim will just create another block 2

Lets if someone has a good answer

[vlog]

I already have the line

DefaultHeapInitialAllocMB=768

in my skse.ini, located under ...Skyrim - Legendary Edition\Data\SKSE\
The MemoryBlocksLog still shows the maximum of 256 for the first block before the game crashes. I launch the game from Mod Organizer. The MO has no patch, where it could overwrite my skse.ini-settings. SKSE version is 1.7.1, so this fix should be included.

How to determine, which skse-settings do I use ingame?

[EssArrBee]

Check that your MO doesn't have the skse.ini file included. You can look at the virtual directory on the Data tab of the right pane.

[vlog]

No, I have another problem: I have to use -forcesteamloader option with skse to work properly. But my Skyrim isn't a steam game. Never used steam at all.

[GrantSP]

According to Sheson you shouldn't raise either blocks above what is needed. If your memory allocation is going past 512, then the 2nd block takes care of the overflow.

 

 

 

There is no need to pre-allocate a much larger block than the game will actually need. In fact doing so can have a negative impact, because other parts of the game need memory as well and the more is pre-allocated to this specific block the less is available to the other processes.

The memory fix is NOT allocating MORE memory it is:

 

 

... we are telling it to pre-allocate the first block in one big continuous chunk instead of the default small chunk and later using additional smaller chunks somewhere else.

Lastly, to back up @hishutup's statement, Sheson counsels this:

 

 

It is only recommended to raise the first block to 512MB and leave the second block alone.

[Nozzer66]

If your Skyrim isn't a Steam game, then it's obviously not a legit edition, since Skyrim required Steam to validate.

 

That's your problem. Where exactly did you get your Skyrim from?

[hishutup]

As Nozzer66 said, Skyrim is a steamworks DRM game.
I'm sure you have your reasons but piracy isn't tolerated in the modding community

 

GrantSP, I know he said that and that is what is accepted but I have seen my game at one point hit 512 and then crash, I compared the timestamps 

That previous situation was back around the time of its discovery though and I didn't really know what was going on.

 

Since this is no longer related to the memory blocks log as a mod please move the discussion about the memory tweaks over to this topic

please read the OP before posting as posting there may not apply to everyone.

If you have a question or you want to share something that directly related to memory blocks log as a mod then feel free to post away. :D

[vlog]

Answered in my main thread. Sorry for off-topic.

[keithinhanoi]

According to Sheson you shouldn't raise either blocks above what is needed. If your memory allocation is going past 512, then the 2nd block takes care of the overflow.

 

I'd like to clarify / qualify the statements quoted by Sheson.

 

As explained in Sheson's OP for his Skyrim Memory Patch on the ENB forums, it involves two blocks of memory (heap cache to be precise) that Skyrim reserves for use of certain kinds of cached data (mainly interior / exterior cell data.) 

 

In an unmodified TESV.exe, both blocks, which Sheson refers to Block 1 and Block 2, are initially set to be 256MB. This 512MB of memory is reserved out of the total RAM that TESV.exe has access to, and being a 32-bit Large Address Aware (LAA) executable, that is either 2GB on a 32-bit Windows system, or about 3.1GB if running on 64-bit Windows.

 

The game's code includes a routine to automatically increase the allocated size of these two blocks of memory when they are full, and in an unmodded Skyrim, this works just fine - most of the time. But when using a good number of mods that add things to the game, the amount of the Block 1 allocation can increase rapidly, and in certain fairly reproducible and consistent situations, the routine to automatically increase the memory reserved for Block 1 fails, causing a memory exception error (ie., CTD.)

 

So, Sheson's fix was to patch the code that sets the size of the two blocks. In his own personal testing, he found that with his load-out of mods, his actual Block 1 usage remained under 512MB, and Block 2 never hit 256MB, so his initial recommendation was as GrantSP quoted - but there was a bit else Sheson said that was missing:

 

It is only recommended to raise the first block to 512MB and leave the second block alone. Like with all new mods this needs to be tested. Extensively.

 

Well, after Sheson announced his fix, people went crazy with testing, myself included, having to depend on a utility called VMmap to monitor how much of Block 1 and 2 were being used while playing in-game. I went as far as to make a batch file that would help create VMMap snapshots at regular intervals so I could look at memory usage whenever a CTD occurred to check if it was related to Block 1 usage being maxed out.

 

From the testing, a few things were discovered.

 

Firstly, it was discovered that Block 2 usage could reach 256MB, without any CTDs. 

 

The next discovery was that for some mod load-outs or a higher uGrids setting, Block 1 usage did reach 512MB, and then the CTDs returned, because the same code to increase the allocated size went into effect. So, people set their Block 1 allocation even higher, to 768MB, and then even 1GB.

 

However, when people set the Block 1 allocation higher than 512, for many of them (myself included) Skyrim would CTD while starting up. Others reported it happening with a Block 1 allocation of 768MB. Testing showed the solution to get around the CTD at startup was to allocate 512MB to Block 2. Setting Block 2 to anything between 256MB and 512MB would not work, though.

 

Sheson does also mention the need to increase the Block 2 allocation in that same OP on the ENB Forums:

 

Only raise the second block if the game doesn't start. It is a workaround at the moment and should only really be needed when testing limits with high uGrids or many mods.

 

EDIT: hishutup has identified the reason why Skyrim crashes for some people when the Block 1 allocation goes over a certain threshold (eg., over 512MB or 768MB). This crash is directly related to the use of ENBoost, when ExpandSystemMemoryX64=true in enblocal.ini. If ExpandSystemMemoryX64 is set to false, then the Block 1 allocation can be set above 512MB, even above 1024MB without Skyrim crashing at startup, and without the need to set the Block 2 allocation to 512MB.

 

Interestingly, it was also found that regardless of what you allocate to it, Block 2 usage never seems go over 256MB.

 

An important final observation is that besides the need to increase the Block 2 allocation when Block 1 is set over a certain size, there is no other directly relationship between the two allocated memory blocks. This means that when the game tries to automatically increase the allocation of Block 1, it does not use Block 2 as "overflow". By all observations and accounts, they are separate and independent.

 

Now again keep in mind as you increase the allocations of these two blocks of memory, you are telling Skyrim to reserve it for these heap caches, and it cannot be used for any other purposes, such as cached textures and 3D geometry data.

 

This is why Sheson recommends using his memory patch fix on a 64-bit Windows system and also using ENBoost. A 64-bit system gives Skyrim more memory to work with, and ENBoost allows some memory usage to be offloaded to your videocard's VRAM or even RAM outside of what TESV.exe can use. But, even if you are running on 32-bit Windows, and can't use ENBoost, you could still potentially benefit from Sheson's memory patch fix with relatively small increase in the Block 1 allocation.

 

So, with all of this in mind, here is what you need to know when using Memory Blocks Log:

• It is the best and most trustworthy method to check that your Block 1 / Block 2 allocations have been changed as you expected.
• It is the best method to monitor your actual Block 1 / Block 2 usage while playing the game, so you can decide if
  • you need to decrease the Block 1 allocation because you never see actual usage go near the amount of memory allocated
  • you need to increase the Block 1 allocation because you saw its usage max out (and followed by a CTD)
• It will show you that when the Block 2 allocation is higher than 256MB, the actual usage never goes over 256MB, and no CTDs happen when it reaches 256MB

Memory Blocks Log will not tell you when you need to increase the Block 2 allocation, but from my explanation above, it's pretty easy to know when you need to do it: If you have just increased your Block 1 allocation, and now Skyrim crashes while starting, then you need to increase the Block 2 allocation to 512MB. Skyrim should start up fine after that, but keep in mind you've just set aside 256MB of memory that the game can't use elsewhere.

 

EDIT: Based on hishutup's discovery mentioned above, there should be no reason to set the Block 2 allocation higher than the default 256MB, as long as ExpandSystemMemoryX64 is set to false in enblocal.ini.

 

Apologies for the very long post, but I hope that clears things up for people.

Edited January 1, 2015 by keithinhanoi

[afonik]

@Keith

As always it's a great pleasure reading your "long" posts.

From my experience I do confirm all of your findings.

Happy new year!

[keithinhanoi]

I was reaching the 512 limit at Block1 so i've setup my memory blocks ini to 1024-512 (game doesn't start with 1024-256)

 

This way my first lines shows:

 

logging of blocks enabled

logging max values only

Timer disabled

Block1      Block2

1024MB    512MB

 

and my final lines (without ctd) are

~

BlocK 1 Block 2

650       256

670       256

678       256

 

So my question is:

why am i stuck with 256 in Block 2?

Although a bit off topic for this thread, I think it's really important to share something I've just learned which will hopefully help a lot of people such as yourself, afonik.

 

hishutup has identified the reason why Skyrim crashes at startup for some people when the Block 1 allocation goes over a certain threshold (eg., over 512MB or 768MB).

 

This crash is directly related to the use of ENBoost, when ExpandSystemMemoryX64=true in enblocal.ini. If ExpandSystemMemoryX64 is set to false, then the Block 1 allocation can be set above 512MB, even above 1024MB without Skyrim crashing at startup, and without the need to set the Block 2 allocation to 512MB.

 

Based on hishutup's discovery, there should be no reason to set the Block 2 allocation higher than the default 256MB, as long as you have set ExpandSystemMemoryX64=false in enblocal.ini. In fact, I think for anyone using the Sheson Memory Patch fix, the advice should be to set ExpandSystemMemoryX64=false if they are also using ENBoost.

 

I have put this information as edits into my above post, just so anyone coming across it will see that as well.

 

I also want to note that setting ExpandSystemMemoryX64=false in enblocal.ini and setting ScrapHeapSizeMB back to the default value of 256 in skse.ini got rid of a Microsoft Visual C++ Runtime Library error I would get every time I tried to quit Skyrim.

 

Kudos and many thanks to hishtup for solving that particular mystery!

[GrantSP]

@keithinhanoi

 

Kudos to you! Brilliant. Concise yet thorough, a very hard combination to produce.

If this can be bookmarked by all then we can all be assured of correct implementation of those settings.

 

Oh, and suppose we can thank @hishutup also, well done to you! Don't want you feeling left out.

[hishutup]

 

Hey, I did a little more than just pm him saying, "HELP"

[sa547]

Just came here to say much thanks for solving what appeared to be a mysterious series of CTDs that nearly derailed my third playthrough in Solstheim. I've been using 1.7.1 (settings are 768/256) and ENB 0.236/0.262 (896mb of VRAM/64mb of Reserved Memory), but I was perplexed about those CTDs that happened often to me, something I never experienced in previous playthroughs.

 

Yeah, I did gone through the week-long process of cleaning up gamesaves, optimizing textures, disabling certain scripted mods, and tried to tweak the enblocal.ini and Papyrus memory settings, did a lot of trial-and-error, reading my Mod Organizer logs, to nail the exact cause of CTDs but to no avail... Until now.

 

I think all the findings presented here should be disseminated as much as possible. I have also bookmarked this thread as valuable reference.

 

EDIT: did used the memory kung-fu as described in this thread -- had the best two hours of Solstheim, with 3 followers and did lots of dungeon raids, culminating with a showdown against Karstaag, and never once experienced a CTD. :D Again, I raise my mug of mead in thanks.

Edited January 17, 2015 by sa547

[Vulgar1]

So, with a heavily modded Skyrim,

3gb VRAM,

16gb RAM,

and a Windows 7 64bit OS,

what would be the ideal settings in my skse and enblocal .ini?

(skse 1.7.2 and latest enb)

Edited May 18, 2015 by Vulgar1