uGridsToLoad Limit: maximum allocated RAM on 64bit < 3GB?

[PatimPatam]

Preface:

 

Hello there, i've got a tricky question (i think) about HW + modding, probably aimed only to people with high-end rigs (8GB or more of RAM + 3GB or more VRAM [non-splitted] + SDD + Windows 7 64bit) that have tried to push Skyrim to the limit.. (for reference i've got a i7-2600K 16GB DDR3 + GTX580 3GB slightly OC + windows n game installed on 128GB SDD).

 

I'm using UGRIDS=9 and about 60+ mods (mostly texture mods from STEP), including a combination of Skyrim HD, Realistic Overhaul and Serious HD, all of them at the maximum quality available. Still running on Skyrim 1.4.27 since NVidia Ambient Occlusion for me is a must and apparently is still not solved.

 

 

Short version:

 

As i understand since v1.3 Skyrim should be able to use up to 4GB of RAM memory (the maximum for a 32bit application on a 64bit OS).. has anyone actually managed to have it working with anything above 3GB?? for instance TESV.exe using something like 3.5GB according to windows task manager? if so did you do anything special to archive that??

 

 

Long version:

 

As we all know there are lots of myths and theories regarding the value ugrids and that can lead to "instability", same thing about using lots of high-res texture mods.. Well i've got the theory that the only real problem when increasing this value or the size of the textures is either lack of VRAM or lack of RAM.. I've got a very stable game using UGRIDS=9 and only recently i found a few specific outside locations where the game would CTD after walking from another outside location (would not CTD if i load a savegame already on the location)..

 

I'm pretty sure VRAM is not a problem (using always less than 2GB - 65%). Total amount of memory is obviously not a problem either (16GB).. However checking my main memory usage i realized CTDs always happens when the game tries to use more than 3GB of RAM. I think that happens when "usable" RAM is already almost filled and when you move to a new area with lots of new resources that have to be loaded there's just no space to put them.

 

I have spend quite some time googling and checking both HW and Skyrim forums with no luck.. have tried using Game Booster 3, changing boot options from the command line "bcdedit /set increaseuserva 4096", changing Skyrim.ini iMaxAllocatedMemoryBytes=4294967295 (4GB) and still the same problem.

 

So is it really an issue of software-limited maximum allocated RAM per process? If so is there a way to use more than 3GB? Or am I totally wrong and it's just a problem of the GPU/CPU not being able to cope with so much information?

 

 

I'm a bit annoyed because i have the feeling that if i could really use the whole 4GB as advertised then all my CTD's would be gone and I could even possibly run with all textures mods + UGRIDS=11.

 

I know there are other ways i could go about this like optimizing the textures, reducing settings to use less memory, etc but that's not the point of the question for now..

 

 

Any ideas?? Thanks in advance!

[Vond]

I doubt there are any rules that specifically states it. It's just common courtesy that if you want someone to read your post, you put in an effort to actually make it readable. Even a short 4-row post like your last one is hard to read due to lack of punctuation, but I did it this time. When it comes to long posts it's way too straining on the eyes to do so I'm afraid.

[Alister]

yeah im well aware with long posts it becomes taxing on the eyes i can see why needless to say and common sense here would be my punctuation sucks i can randomly guess and make things worse dunno if people would enjoy that sure the person who was saying they dont read anything without would make fun of incorrect punctuation as well i certainly made my post shorter i guess that means i may have subsequent questions later on but such as life i guess and about common courtesy theres lots of things common courtesy could be applied to in life not everyone actions are always so courteous

[MadWizard25]

No excuses, if you can write a forum post you are certainly capable of some punctuation intellectually. Your grammar does not "suck", you just willfully choose not to apply it, even when politely asked to do so.

 

No one expects perfect spelling or grammar of you. You are correct that not every action in life is courteous. However, when courtesy is asked for and still rejected, as you have done, especially when regarding an easy action such as pressing a few more extra keys, some may take that as an insult and/ or impolite behavior.

 

So get punctuating :)

 

Edit: Must remind myself, do not feed the trolls!

[Alister]

i have a grandma omg just what ive always wanted lol haha in the end its all a matter of opinion i see the point your making the way everyone interprets rude actions behavior is subjectively based on individuality i guess i have a confession to make tho i suck at maths to oh boy here we go lol :P

[ivactheseeker]

From the S.T.E.P Community Citizenship

Welcome to the STEP-Community Support Website! As of this moment, we have many relevant forums and are in the process of building out a wiki that will articulate and integrate to some degree with these. Within the site, you will find many helpful resources voluntarily provided. Many of the questions that you may have will likely have already been answered by others.

 

We humans are composed of personalities that fall along a continuum of positive to negative, and fortunately, the majority tend to associate with the more positive pole of this continuum. Unfortunately, there is a minority that aligns with the more negative --not necessarily pessimistic, but more truly negative. While these individuals are not necessarily "bad" people, they can nonetheless be a damper to the rest, and like a wild yeast, can have a tendency to sour an otherwise perfectly good batch of home-brewed mead. This monologue and the guidelines that follow are intended for all, but necessitated by these less "enlightened" of us.

 

TheCompiler and the other STEP-Community site administrators ask that you, as a registered member of this site, participate and/or contribute in a manner of good spirit and common courtesy. Adherence to this simple ethos will head off any unpleasantness before they are manifest. Let the guidelines outlined herein serve as a reference for our members, so that the community may evolve to its maximum potential.

 

 

 

 

and form General Points of Etiquette

 

- The use of "Leet-Speak" and chat lingo is very unfriendly to foreign-language speakers, or those unfamiliar with these conventions. Such conventions are not in the spirit of this community, and offenders will be warned.

- Please don't use [Caps Lock] or all caps for thread titles of any substantial part of a post.

- Wait a reasonable amount of time (more than 24 hours) before "bumping" your topic.

- Refrain from "Double Posting". Please use the [Edit] button to correct this when done accidentally.

- When editing or modifying a post, please add your new message on a new line starting with "EDIT:". It can be very confusing and possibly even unfair if a post to which another user has responded mysteriously changes. (The exception is minor spelling or punctuation corrections, no EDIT note necessary in such cases).

- Debate and discussion are always encouraged, but rudeness and personal insults aren't. This includes starting flame wars, or posting purposeless inflammatory posts. Offending posters will be handled accordingly.

 

 

@ z929669 and other moderators

 

I think you should put the S.T.E.P community citizenship and such on a thread in the front page. Maybe it's just me, but it took me a bit to find them.

[z929669]

I can include a link in the posting guidelines and an announcement I suppose

[frihyland]

I've put it on the wiki front page, but it should definitely be in the account signup process, and in the bbs somewhere as well.

[Bealdwine]

I agree with Fri.

 

If anyone becomes obnoxious enough to ban you must be able to point to the specifics of what they have transgressed in some readily available document so that other users don't percieve our actions as arbitrary. Of course if people don't read the stuff, well... *shrug* :D

[PatimPatam]

Ok back on topic..

 

As promised, after trying a few different things I'm reporting my findings!

 

First of all, the modifications on the ini file suggested did not seem to make any difference.

 

So after that i decided to try the RAM disk route and got myself a full version of Dataram RAMDisk (about 20$, free version limited to 4GB). I thought that even if it didn't fix my problem i could put to good use my extra RAM and get a nice boost, even on other games or apps. I can say that I'm quite happy with the product and that i works as advertised, but the first problem i stumbled upon is that i don't have nowhere near enough RAM to load my entire skyrim folder + HD textures (14.2 GB) taking into account i need about 2GB for windows + 4GB free to run skyrim.

 

Using a symbolic link (thanks for the tip frihyland) i tried to load into RAM only the textures folder (about 5.3 GB) and after that i also tried loading the whole skyrim but without the textures folder (about 9 GB, loosing visual fidelity of course). Unfortunately none of these prevented the CTDs from happening.

 

[sIDE NOTE ON RAMDISK PERFORMANCE:]

Surprisingly load times where not greatly reduced in comparison to loading from a SSD (a typical savegame went down from 12 to 10 sec aprox), i guess that apart from merely loading the assets into memory there must be some decompression or other stuff going on that slows down the process.. Also I don't usually get much stuttering but i think preloading on RAM helped to reduce it a tiny bit in some instances, although i can't give any real proof.

 

Anyway all this made me realise of something important though, if i was not loading the textures folder (and/or the HD texture pack BSAs) the RAM usage would be MUCH lower, which means 2 things:

1- textures ARE being cached on RAM

2- RAM usage (in particular 3GB on 64bit) does not have anything to do with the CTDs i was having.

 

Just out of curiosity tried a couple more things:

Overclocking CPU + RAM -> no difference.

Compressing all textures and BSAs to 1024 -> no difference.

Lowering resolution, disabling AA, disabling AO -> no difference.

 

Finally I tried to revert back to ugrids=7 (which was much easier than i thought it would be) and voilà, as expected no CTDs at all. Only one warning, when reverting ugrids do NOT use the nvidia "The Top Five Elder Scrolls V: Skyrim Tweaks" article, it's bad, outdated advice, and will not only NOT work but create even more problems. Both the official NVIDIA tweak guide, or the following forum post are correct:

https://www.geforce.com/Optimize/Guides/the-elder-scrolls-v-skyrim-tweak-guide/#14

https://forums.nexusmods.com/index.php?/topic/490666-patch-13-vs-ugridstoload/page__st__30

 

So in the end i must admit i was wrong with my initial theory and i guess there must be an intrinsic limit on the amount of resources that this engine can handle, and that no matter how much CPU, GPU, RAM or VRAM you throw at it you will hit a wall sooner or later. Or maybe it’s related to some other hardware limitation like memory bandwith.. Anyway I'm pretty sure this could be solved if bethesda looked into it and patched the main engine, but for now it is what it is! I think it's a pity because when you get used to playing with ugrids=11 or ugrids=9 it's really hard to go back, even to 7 (can't imagine trying the default 5). I find it really immersion breaking having the whole landscape changing or popping up in front of me.. honestly i would rather stay with vanilla textures if i could keep ugrids=9, but as reported this is not a choice.. so i will just have to get used to it!

 

The only doubt that i have left is that i've seen in some forums people claiming to have played skyrim for months with ugrids=9 and no CTD's, so maybe there's still something else i could try but really running out of ideas now. One example is this one:

https://forums.nexusmods.com/index.php?/topic/473949-draw-distance-ugridstoload/

 

Porroone said it could be because his GTX 470 has a 384bits memory interface, which could make sense, but I also have 384bits on my 580 so I don’t know..

 

 

To finish off, and in spite of all my troubles, i would like to suggest to those with some spare VRAM and still playing with the default ugrids to give uGridsToLoad=7 a try, it's dead easy to go back to the default at any time in case of trouble.

 

Skyrim.ini:

 

[General]

uGridsToLoad=7

uExterior Cell Buffer=64

 

SkyrimPrefs.ini:

 

[backgroundLoad]

bSelectivePurgeUnusedOnFastTravel=1

[z929669]

Thanks for the detail Patim. very useful info!

[z929669]

I agree with Fri.

 

If anyone becomes obnoxious enough to ban you must be able to point to the specifics of what they have transgressed in some readily available document so that other users don't percieve our actions as arbitrary. Of course if people don't read the stuff, well... *shrug* :D

 

 

That has been implemented for awhile now I think

[MadWizard25]

@ PatimPatam

 

Yep, CTD with higher uGrids values is most certainly linked to how much VRAM you have. Actually just thrashed this out (again), with some other people, a week or 2 ago on another thread, here, https://forums.bethsoft.com/topic/1380897-ugridstoload-7-freezing/page__fromsearch__1

 

Short story was trade off between hi-res texture mods and higher ugrids values. I can run ugrids 9 on vanilla, with occasional ctds, and 7 is rock solid. However, using step mods, and even 1024 versions, i cannot use anything but the default ugrids value of 5. CTDs are always around areas that require a lot of vram. Removing all large hi-res texture packs (SRO and 2KHD) let me run on 7 again. And thats with a 2GB VRAM card.

 

With 3GB you should be able to run hi-res texture mods and ugrids 7, lucky you :)

 

One of my personal projects for skyrim it somehow achieve hi-res textures with and ugrids value of 7, but no time atm. Thats why your comment helps so much, I dont have to test so many variables in the future :)

[PatimPatam]

@MadWizard

 

Yes i guess i'm quite lucky as ugrids=7 with almost all STEP texture mods (at max quality) is rock-solid for me. But you know, you always want more than what you have :-) As i found out, even after you remove the VRAM problem you still face other ones, which haven't been able to fix so far..

 

Anyway here's something i didn't mention before that might help you stabilizing ugrids=7 and that you could test quite easily, another theory of mine :-P

 

Probably you've heard about the rule where exterior_cell_buffer value must be (ugrids+1)².. Well that didn't make much sense to me.. i believe ugridstoload is the number of cells that fit in one side of a "square" around you, so if for instance ugridstoload is 7 that means you are loading a matrix of 49 cells in total (the cell where you are + 48 cells around you).

 

So in my tests with ugrids=9 i tried the equation ugrids²+1 and found that i had less CTD's than with the other one (i'm using 82 = 9²+1). Haven't tested it with ugrids=7 but you could try.. instead of the usual 64 check exterior_cell_buffer = 50 (7² + 1), it will use much less memory and should be enough to buffer all your cells.

 

Also make sure you use the value i mentioned bSelectivePurgeUnusedOnFastTravel=1 on SkyrimPrefs.ini, that helped me stabilize ugrids=9 a lot. Finally when testing don't load savegames saved with different ugrids values (even though in theory you can load savegames with lesser values than your ini). Skyrim doesn't like that much.

 

 

@Admins

 

do you think it could be possible to change the name of the thread and replace the word skyrim with ugrids or ugridstoload? (thought about changing it after creating it to make the subject clearer, but couldn't find out how). Thanx!

[ivactheseeker]

You can do it yourself: go to the "Support/troubleshooting" section-> find your thread -> don't open it but click and hold the title to edit.

[PatimPatam]

Ok thank you ivactheseeker