uGridsToLoad Limit: maximum allocated RAM on 64bit < 3GB?

[PatimPatam]

Preface:

 

Hello there, i've got a tricky question (i think) about HW + modding, probably aimed only to people with high-end rigs (8GB or more of RAM + 3GB or more VRAM [non-splitted] + SDD + Windows 7 64bit) that have tried to push Skyrim to the limit.. (for reference i've got a i7-2600K 16GB DDR3 + GTX580 3GB slightly OC + windows n game installed on 128GB SDD).

 

I'm using UGRIDS=9 and about 60+ mods (mostly texture mods from STEP), including a combination of Skyrim HD, Realistic Overhaul and Serious HD, all of them at the maximum quality available. Still running on Skyrim 1.4.27 since NVidia Ambient Occlusion for me is a must and apparently is still not solved.

 

 

Short version:

 

As i understand since v1.3 Skyrim should be able to use up to 4GB of RAM memory (the maximum for a 32bit application on a 64bit OS).. has anyone actually managed to have it working with anything above 3GB?? for instance TESV.exe using something like 3.5GB according to windows task manager? if so did you do anything special to archive that??

 

 

Long version:

 

As we all know there are lots of myths and theories regarding the value ugrids and that can lead to "instability", same thing about using lots of high-res texture mods.. Well i've got the theory that the only real problem when increasing this value or the size of the textures is either lack of VRAM or lack of RAM.. I've got a very stable game using UGRIDS=9 and only recently i found a few specific outside locations where the game would CTD after walking from another outside location (would not CTD if i load a savegame already on the location)..

 

I'm pretty sure VRAM is not a problem (using always less than 2GB - 65%). Total amount of memory is obviously not a problem either (16GB).. However checking my main memory usage i realized CTDs always happens when the game tries to use more than 3GB of RAM. I think that happens when "usable" RAM is already almost filled and when you move to a new area with lots of new resources that have to be loaded there's just no space to put them.

 

I have spend quite some time googling and checking both HW and Skyrim forums with no luck.. have tried using Game Booster 3, changing boot options from the command line "bcdedit /set increaseuserva 4096", changing Skyrim.ini iMaxAllocatedMemoryBytes=4294967295 (4GB) and still the same problem.

 

So is it really an issue of software-limited maximum allocated RAM per process? If so is there a way to use more than 3GB? Or am I totally wrong and it's just a problem of the GPU/CPU not being able to cope with so much information?

 

 

I'm a bit annoyed because i have the feeling that if i could really use the whole 4GB as advertised then all my CTD's would be gone and I could even possibly run with all textures mods + UGRIDS=11.

 

I know there are other ways i could go about this like optimizing the textures, reducing settings to use less memory, etc but that's not the point of the question for now..

 

 

Any ideas?? Thanks in advance!

[z929669]

I am guessing that Windows memory management may be the culprit ....

[stoppingby4now]

There aren't any inherent memory limits placed on applications. If they request memory and it's available, they will get it. Game engines themselves aren't very big, it's all the textures and resources that they load. In general, the only things that will reside in system RAM for a game is pre-cached data, or data that was recently used but being kept around "just in case". I highly doubt that its Windows fault, but more likely a momentary spike of the game needing to load a large amount of textures to the video card and it not managing the process very efficiently to deal with what is already loaded.

[z929669]

I am guessing that Windows memory management may be the culprit ....

 

 

Sorry, I should have been more specific: I mean if Skyrim is never utilizing more than 3 gigs, then it could be Windows memory manager not playing nicely with certain processes and graphics hardware software memory managers. Then throw in the game's memory management, and we have several memory handlers that don't necessarily work efficiently together, resulting in paging out to disk and increasing chances of head-on collisions ... = CTD.

 

 

 

Just speculation though. s4n is probably right, alluding to the issue being with the game's memory management limitations (hence the uGrids bugs with increasing values above 5)

[PatimPatam]

Thanks a lot for the feedback guys, what s4n said makes sense.. i guess it could be just a coincidence that this always happens when it goes exactly above 3GB.

 

Will have to either save often and deal with a CTD from time to time or suck it and cut down on some settings.. will see.

 

Still would be nice to hear if someone else with similar or better specs than mine can confirm this!

[frihyland]

By default a 32 bit windows application will never ask for more than 3GB of memory from the system, memory management above that would have to be specially coded. One of the primary differences between 32 and 64 bit applications is the radical increase in memory supported, from 3GB to 64GB but until consoles go 64bit we won't see any games that do. Since skyrim was coded for consoles and only ported to pc later and it only supported 2GB on launch, its more than a little unlikely that this special above 3GB code has been written.

 

Actually getting some use out of that much ram is rather a problem, one solution is to buy a ramdisk application that turns your unused memory into a virtual harddisk that is roughly 8000 times faster than the fastest ssd. You can copy all of your skyrim directory here and launch skyrim, needless to say load times will be a thing of the past and so should any ctd's or slowdowns involving any type of memory management or disk management. FYI if you encounter any type of drm issues you can probably circumvent them by judicious use of symbolic links, ask again if it arises and many here could help with that.

 

Additionally there are special ini settings which should increase ram usage for specific game uses. Many of these have been tested fairly exhaustively back in November with little to no effect on memory usage or gameplay excepting uGridsToLoad and uExterior Cell Buffer. Posted merely in case you want to try them out of curiosity to see if something has changed since then.

 

In skyrim.ini

 

 

[General]

uGridsToLoad=9

uExterior Cell Buffer=200

uInterior Cell Buffer=18

fMasterFilePreLoadMB=200.0000

iPreloadSizeLimit=419430400

 

[Papyrus]

iMaxMemoryPageSize=16384

iMinMemoryPageSize=4096

 

[budgetCaps]

uActorMemoryBudgetCap=20971520

uLoadedAreaNonActorMemoryBudgetCap=371195904

uWaterMemoryInterior=20971520

uTextureMemoryInterior=20971520

uGeometryMemoryInterior=20971520

uWaterMemory=10485760

uTextureMemory=20971520

uGeometryMemory=10485760

[PatimPatam]

Thank you frihyland, looks like some good advice in there.

 

Will try both things as soon as i can and report back!

[Vypir]

I know this doesn't answer your question, but Tom's Hardware had a good piece about using large amounts of RAM for gaming. It looked at using RAM disks for swap and temp files. You might want to give that a try and see if you can eke out some more performance with all that extra RAM available.

[stoppingby4now]

By default a 32 bit windows application will never ask for more than 3GB of memory from the system' date=' memory management above that would have to be specially coded. One of the primary differences between 32 and 64 bit applications is the radical increase in memory supported, from 3GB to 64GB but until consoles go 64bit we won't see any games that do. Since skyrim was coded for consoles and only ported to pc later [b']and [/b]it only supported 2GB on launch, its more than a little unlikely that this special above 3GB code has been written.

 

A 32 bit windows application can reference a full 4GB of RAM, and thus it can also request more than 3GB. But requesting and actually being granted are two different things. The supposed 3GB limit on Windows 32-bit systems is constantly misunderstood and thus misrepresented and has to do with hardware addresses being "memory mapped IO reservations" in Windows. It's a horrible design.

 

To make matters worse, 32-bit Windows allocates up to 2GB of system memory for the kernel, leaving the rest for user space. Applications requesting more memory than is available in user space can still be granted the extra memory that is reserved by the kernel, but it's not guaranteed. Microsoft introduced the /3GB switch in order to help special cases by forcing the 32-bit kernel to only reserve 1GB, leaving 3GB available for applications (more like ~2.3GB after hardware addresses eat up what's left).

 

Point being, on a 64-bit system with sufficient RAM, a 32-bit application should have no problem using 4GB of RAM.

[frihyland]

Correct, I simplified the problem slightly for readability. Since we are getting techie 64 bit windows also reserves memory for OS and hardware but since it can address more than 4GB as long as you have at least 6GB RAM on your motherboard your 32 bit applications should be able to access their full 4GB alottment (on a 32 bit system this can never happen without special software), in practice this doesn't really ever happen though that I have seen.

[stoppingby4now]

Exactly. Even in games with much higher resolution textures, I never see any large amount of RAM usage approaching anywhere near 4GB. The majority of the textures should be in VRAM except when cached by the application. Even then, there shouldn't be any necessary case where it would need to cache that much data.

[Alister]

i have a amd phenom x2 555 unlocked to a quad at 3.2 a vertex 2 ssd and a radeon hd 6970 2G running at 930mhz interested in trying to get the most of skyrim was the other persons issues due to I/O speed of the ssd and would a ramdisk help with higher ugrids? would 16 Gb or ram be enough for step most high res textures and mods ect my pc using about 1GB of 4 after boot with win 7 x64 any constructive input appreciated

 

im sorry that you dont read posts without punctuation yet do take the time to make posts that dont serve much point there was a point and it was kinda rude but i think staying on a high horse isnt a good thing but you might disagree with me

[z929669]

Sorry Alister. I just don't read posts without punctuation.

 

This is a BB after all and not a chat ;)

[Alister]

thought better of it and removed

[Vond]

Well your post was very hard to read due to the fact that you didn't use puncuation. And you did it again in the reply aswell. Please, just edit the original post with punctuation so that it's easier to read and I'm sure you'll get constructive input.

[Alister]

thought better of it and removed