CTD at 3.1GB

[viking]

Some key posts on this and related threads (experts feel free to note any errors or insights):

• Post #11
• Post #27
• Post #40 (this is where the 500/430 ratio seems to have originated.)
• Post #66
• Post#278
• Post #313
• Post #1 (adjacent thread: Caution about memory optimisers)
• Post #64 (adjacent thread: Caution about memory optimisers)
• Post #68 (adjacent thread: Caution about memory optimisers)
• Post #70 (adjacent thread: Caution about memory optimisers)

Wiki article (draft)

 

Thanks,

STEP

 

OP follows

---

First of all I wanted to thank you all for the great work you have done with STEP. Skyrim is the first game I installed on this computer and you guys have made it an AMAZING game. That being said, I have an issue that I hope you can help me solve.

 

My setup:

Vanilla Skyrim

gtx670 w/ 4GB @1080p w/ latest driver

16GB Memory

3770K at about 4GHz

Windows 8 64bit

ENB 149

Ultra settings

Highest available texture/quality

Mod Organizer

Step 2.2.1 + Skyrim Revisited + others

 

I have noticed a post here and there saying that Skyrim can't really address more than about 3.1GB of memory without issues. This seems to jive with my experience, meaning I CTD every time my memory hits that mark, but I couldn't really find anything definitive on the topic. The issue with googling the topic is the pre 1.3 skyrim that couldn't address more than 2gb of memory.

 

My mod list is mostly based on STEP which is why I came here for help, with about ten mods added onto the end (Interesting NPCs, Detailed Cities, Economics, COT, and a couple others). The reason I haven't included my mod list is that it doesn't seem to matter. As long as I keep the memory usage below 3GB I can have pretty much any combination of mods.

 

What I have tried so far (in no particular order):

• resetting ini files
• removing enb
• not using attklt
• only using a new game
• removing all mods and adding one by one until issue crops up
• running as admin
• watching the papyrus log - it seems relatively clean, no obvious errors right before CTD

Yes, I can run STEP just fine without any issues, but I also never get near 3GB of memory. I have tracked VRAM usage as well and have seen a max of 2.7GB/4GB.

 

As an example of where I might run into issues: I start a new character with Alternate Start. I start with Breezehome. Run out of Whiterun, past the Brewry, up the hill to the bandits. Enter the cave (watching memory usage with Elys MemInfo), and it dies right after I see 3GB. I have this same issue not using AS, sitting through the intro, and then running over to whiterun.

 

I'm sorry if this post is all over the place. I have spent more than a week trying to solve this issue, and the only solution I have found is to reduce memory usage. I have got to the point where I can exchange two texture packs and get into the cave without a CTD, but with both I get a CTD. I didn't even think texture packs should even affect CTDs, but I'm relatively new to Skyrim on the PC, so I could be wrong. I also found I could get a bit further with ENB turned off, but would still crash once I got above 3GB of memory. Finally, if I reload a game after a CTD, I can play just fine...until I reach 3GB of memory.

 

I really hope you guys can help. I more than willing to try anything at this point, besides just disabling all of the mods.

[MontyMM]

There's no question that there's a 4GB limit, but I think there are still many open questions to answer here.

[TechAngel85]

 

Seems like the 4GB limit has also be confirmed by others on Nexus: https://forums.nexusmods.com/index.php?/topic/787596-skyrim-ram-usage-increases/

 

EDIT:

After Googling and reading some more, I'm convinced that this CTD issue is caused by Skyrim reaching it's RAM limit. We only need to confirm by a few more testing. Viking and Neovalen have both already confirmed this issue at around 3GB of usage already. The fix for this would only be to decrease your mod load.

Unfortunately so yes. Going to go through and see who the big culprits are. Even with all my hardware I apparently can't use all 2048 textures without hitting cap.

 

There's no question that there's a 4GB limit but I think there are still many open questions to answer here.

On a side note, has anyone install Extreme STEP? I'm curious if it hits this limit as well. We need to go through and gather all the questions up and try to answer them.

[torminater]

I have. I can't say for sure, but I don't think extreme step (unless maybe at high display resolutions) hits this cap.

[viking]

If anyone comes up with an easy/quick way to test this, please post it here.

 

I'll second torminater, at least for 2.2.0 with all the max options I didn't have this issue.

[Besidilo]

The problem is that high resolution textures causing increased VRAM usage are actually only half of the problem.

 

Since the 4GB memory limit will be linked to both system memory and that part of video memory that's actually "lockable", which I suspect is the large part of Skyrim, given its outdated engine, increasing VRAM usage would mirror the increase in the Skyrim's RAM usage and therefore make it more prompt to crashes. But let's not forget oth

[z929669]

I'm very curious about your mod setup and how you can get that high physical RAM usage with default uGrids. Loading into Riverwood with every mod I have I'm at 1859 RAM usage and 1889 VRAM usage. That's with most textures at 2048+ resolution, 2560x1600 screen resolution, and a large number of script-heavy mods outside of STEP. Even with uGrids=7, those numbers would only jump to ~2300 physical, 1950 VRAM.

I misquoted Neovalen in my previous post ... it was you, JJ, that supported my experience of 1.8-ish GB of system RAM use allocated to TESV.exe

I'm not going to go through all the trouble of doing a spreadsheet but I can show the difference between vanilla and vanilla+TPC as far as VRAM/RAM usage goes. Unfortunately GPU-z doesn't seem to support dynamic memory monitoring on GTX680s. What could I use instead?

You are missing the sensor (or GPU-z does not talk to it). I think it works better for AMD cards, but I'm not positive on that. You should only be bleeding off some relatively stable VRAM to RAM. I would guess something like 5-10% is all ... until you begin to hit your VRAM ceiling, at which point, you begin seeing more VRAM spill over. If you cannot see your dynamic VRAM (VRAM to RAM) spillover, then you cannot know that textures and resolution are costing the TESV.exe process any RAM .... unless my reasoning is flawed (I am no hardware/software expert).

 

Reading through this thread I am not convinced that this is a RAM issue at all (and I echo Neovalen's comment about system RAM allotted to TEXV.exe being higher than 1.8 GB). Something strange is happening if people are getting system RAM usage that high from this single process.

I'm not sure which of Neovalen's comments you are referring to. Also I'm not sure why it would be strange for this single process to use a lot of ram. I would be very happy if this is not a ram issue. z --> Fixed

 

Are we certain about these RAM allocations being specifically tied to TESV.exe?

Yes? I'm not sure where you are coming from. I tracked the RAM (not VRAM) usage using Process Explorer (which you are familiar with); it was tracking only TESV.exe. VRAM, which was system wide, was tracked using GPUz.

z --> But you can also look at GPU dedicated VRAM and dynamic VRAM (System RAM) in Process Explorer too. You have to set this ip under [View] I think.

Skyrim does not allocate much VRAM to RAM until it runs out of VRAM. Even then it is not so much that it could additively boost my Skyrim RAM use to more than 3 GB. My benchmarks are pretty detailed in this regard.

I haven't had a chance to read through your benchmark, but I will. I am more than happy to do any tests you are interested in, because I would much rather find a setup issue than reduce mod count. Unfortunately I won't have a chance until this weekend to do any real tests (meaning redoing step from scratch). I'd like to point out that my numbers, and all of my experiences with ctds, were all using attklt. z --> Bummer. I thought that this would help. Maybe try with it off or using the full version?

 

As for whether the "bleed over", as you called it, goes into the same address space, it would seem that it does; however we have no real "proof" beyond some anecdotal evidence and the microsoft directx patch details linked to earlier in this thread.

 

---

Did a run from Helgen cave to Riverwood. Not the most accurate thing in the world, but it correctly depicts what I've found in my testing.

 

So with ONLY Texture Pack Combiner being added, I see an increase of ~500MB VRAM and 430MB physical RAM.

But this will be the very biggest chunk. The rest of STEP should not cost enough to get you past 1.8ish, right?

---

I'm going to try to get the thread back on track:

 

Besidilo, I'm not sure who got your name wrong. If it was me, sorry about that. I know very little about VRAM. A link with more info would be more helpful. The more we understand about the behavior of VRAM the more likely we are to understand this issue. I read in some forum somewhere (highly reliable...) that Skyrim was partially directx10. Don't trust me on that. It makes far more sense for it to be directx9, especially since xbox is limited to that.

 

Salvador, I agree that the references is Besidilo's links were excellent. I just had a hard time following the first one. I didn't notice anything incorrect in the links, there were just a few times when things like the maximum ram in a system was taken to be a constant across all systems. Again, the information was correct, just difficult to follow and left some key pieces of information out IMO. 

 

z recommended several test we can run to test the memory/vram correlation, and I will run them this weekend. Are there any other ideas on how we can get to the root of this issue?

Have you read through this?--> https://wiki.step-project.com/User:Z929669/Benchmarks

 

Also, it is entirely possible I suppose that my and many others' VRAM is the majority of the "Private Bytes" (if I understand correctly),; however, us regular folks (... who might wear tennis shoes or an occasional python boot ...) are trying to keep our VRAM usage below 1 GB, so the 1.8 I see at max is very likely 1 part VRAM and 0.8 parts RAM all loaded onto TESV.exe I suppose .. duh :O_o:

 

Anyway, read the benchmark page I created and take a careful look at the charts. This is all about how VRAM and RAM play together, and it is an interesting little dance.

 

However, sorry for leading the thread astray, as I see now that this is probably a process-RAM problem that I don't have the VRAM to experience :P

[viking]

Z, I am really glad you have been posting here, and your benchmark is excellent and informative. Unfortunately I can't seem to get a couple of your graphs on your benchmark to enlarge...not sure if that is just my issue or not, but even still, the chart gives a pretty good idea of what you mean. I do think you have enough vram to experience what we are talking about, just not in the normal play of the game. If you are interested, you could try the "cheese wedge effect" discussed earlier in the thread and see if you get a ctd at 3.1GB.

 

I am pretty sure you are right about Process Explorer listing dynamic and static vram usage, but I'll have to look closer when I'm at a windows computer.

 

I have tried running it without Attk. It does lower the ram usage slightly, but not significantly. I will have to try the full version and see if it makes a difference. I'll have to be honest, I haven't read the description of Attk, so I don't know exactly what it does (besides help manage memory somehow). 

[Salvador]

Z, I am really glad you have been posting here, and your benchmark is excellent and informative. Unfortunately I can't seem to get a couple of your graphs on your benchmark to enlarge...not sure if that is just my issue or not, but even still, the chart gives a pretty good idea of what you mean. I do think you have enough vram to experience what we are talking about, just not in the normal play of the game. If you are interested, you could try the "cheese wedge effect" discussed earlier in the thread and see if you get a ctd at 3.1GB.

 

I am pretty sure you are right about Process Explorer listing dynamic and static vram usage, but I'll have to look closer when I'm at a windows computer.

 

I have tried running it without Attk. It does lower the ram usage slightly, but not significantly. I will have to try the full version and see if it makes a difference. I'll have to be honest, I haven't read the description of Attk, so I don't know exactly what it does (besides help manage memory somehow). 

Hi Viking and Z.

 

To start with, I tried the Cheese wedge things on vanilla skyrim with texture packs.

Only increases memory usage by couple mb. It does gradually increase CPU usage though. There is no difference at all in VRAM, as expected. So it does not load any RAM clearly. You can however freeze your game, just try 800 cheese wedges 3 times in a row (I was on high hrothgar). I wonder why ATTK helps so much (you can do many more cheese wedges) if it uses relatively little memory?...

 

Therefore you need some other memory intensive mods that do not touch textures, to see if it really is textures, or something else that makes it cap at ~3GB Perhaps some large sound files attach to a certain location?

 

My interpretation.

 

Apparently only part of the VRAM you will see in process explorer (or GPU-Z) is actually part of the virtual address space that is used by Skyrim. At least that is how I interpret the documentation from Microsoft that MontyMM linked to (and this, although perhaps not fully reliable). Most of the memory is allocated to the driver for AA etc.

 

"The game does not need to know about the memory used for AA calculations. This happens in memory space that is not a part of the games 32 bit address space. How can we be certain of this? Well, you can force AA at the driver level, and the game will never know it's running AA. This is handled by a different application, and while I suspect this memory space would be subject to the address space limit of a 32 bit OS, it should not be an issue for a 32 bit app in a 64 bit OS." 

 

This would suggest you can save some memory by using driver AA above Skyrim or ENB AA etc.

 

Now, from the link MontyMM provided it is clear that the data loaded into VRAM by a process was copied in RAM, and the copy was maintained, taking up address space and essentially doubling memory allocation. However, this was fixed for DX10 and above. This is however partially fixed for DX9.

 

You need to understand which part of that memory is allocated as locked. In other words, Skyrim doesn't allow it to be swapped out. Only this allocated video resource is copied into RAM and part of the allocated virtual memory address space. The remainder is not. Say skyrim allocates 512MB VRAM, it only has 3.5GB allocation left. However, in the past the full used 512MB was copied in RAM, leaving only 3GB. To overcome this (partially) windows makes sure only part of the 512MB is copied (the locked part) say 256MB is allocated as locked pages, that part is copied into RAM, leaving 3.25GB of allocation space.

 

This provides a new problem. Dynamic VRAM (or system VRAM) is RAM allocated as VRAM (hypermemory (AMD) or TurboCache (Nvidea)). Is this addressed by the skyrim engine? if so, than it takes up address space. This is not necessarily loaded with locked allocations, and might therefore be loaded with unlocked pages. If the dynamic VRAM is loaded with lockable pages by Skyrim, it is also duplicated and loaded again within skyrims address space.

 

But I fully agree, to find any evidence of textures taking up RAM space for skyrim, you should measure VRAM spillover and Skyrim memory allocation. you should see a Skyrim memory usage increase similar or proportional to the increase in dynamic VRAM by only increasing texture size.

[viking]

Your point about large sound files being an issue is interesting. I do have two mods with 500mb+ (on disk) of sound files. I'll try to see how that affects memory when I get a chance.

 

As for how enb AA affects memory usage. 119 was the last version of enb to support hardware AA, which would have been outside of the memory space of skyrim. Newer versions, however, are software based. The dll version of enb, I believe, is loaded by skyrim into skyrim's addressable memory (can someone confirm this?). Enb does not use a large amount of ram in my experience, but does affect it some.

 

It is unfortunate that the cheese wedge idea didn't work for our purposes. I am also curious what attk does to allow more wedges if more wedges don't affect memory usage.

[Salvador]

Your point about large sound files being an issue is interesting. I do have two mods with 500mb+ (on disk) of sound files. I'll try to see how that affects memory when I get a chance.

 

As for how enb AA affects memory usage. 119 was the last version of enb to support hardware AA, which would have been outside of the memory space of skyrim. Newer versions, however, are software based. The dll version of enb, I believe, is loaded by skyrim into skyrim's addressable memory (can someone confirm this?). Enb does not use a large amount of ram in my experience, but does affect it some.

 

It is unfortunate that the cheese wedge idea didn't work for our purposes. I am also curious what attk does to allow more wedges if more wedges don't affect memory usage.

Would be interesting to see how the sound mods affect memory.

 

Although ENB does not load much in RAM. How about VRAM? Any VRAM allocation used from software based AA is also part of skyrim address space. AA is a big VRAM user (normally, perhaps Boris did good optimizations). Yes any DLLs in skyrim root are loaded within its address space. Would be interesting to see how much VRAM/RAM ENB or SMAA use, most probably a lot less than the driver (hardware). However if you have plenty of spare VRAM, you would be better off using forced driver settings. I figured you have 4GB card? so VRAM limit is not a problem for you. (This is ofcourse VRAM wise, the performance impact of hardware based AA (or AF) vs ENB or SMAA is not considered here)

 

Yes it is a shame cheese wedges don't work. I don't understand the ATTK stability improvements either, would be good to ask Jason2112, he knows a lot about memmory. 

[Neovalen]

Still curious why some of us have soooo much more RAM/VRAM use than others... I would have no trouble hitting this cap with only 2048 textures...

 

I run my screens in surround but play Sky rim full screen on one monitor (other two go black). Wonder if that could be a contributor.

[JudgmentJay]

As z stated, they limit their VRAM usage to very low numbers. I'm sure if you limited your VRAM to 1-1.5GB you'd never come close to 3GB of physical RAM usage either.

[TechAngel85]

Still curious why some of us have soooo much more RAM/VRAM use than others... I would have no trouble hitting this cap with only 2048 textures...

 

I run my screens in surround but play Sky rim full screen on one monitor (other two go black). Wonder if that could be a contributor.

I don't use anything over 1K textures (unless they're not available), run at 1366x768, and have a very custom set of settings and INI tweaks (shadow and water resolutions are turned down, etc.). I max out my 1GB of VRAM, but to be honest I don't know how much system ram is being used by the TES process. I don't have those sensors on my system for system ram during the benchmarks. I'll have to use Processor Explore later to find out.

 

What sucks is how demanding Skyrim is. I can play other games (Mass Effect 2&3, Dragon Age, Sims 3, Borderlands 2, etc) at 1600x900 (laptop's max resolution) with 4xAA and 16xAF while still maintaining around 35-40 FPS. Skyrim...forget it! No AA, 8xAF, and 1366x768 just to maintain 30-35 FPS.

[Salvador]

Still curious why some of us have soooo much more RAM/VRAM use than others... I would have no trouble hitting this cap with only 2048 textures...

 

I run my screens in surround but play Sky rim full screen on one monitor (other two go black). Wonder if that could be a contributor.

Yes that be interesting to test, I also wonder how much screen resolution plays a role.

 

What I understand from this thread is that you can get pretty low VRAM usage and still crash and have high RAM usage. allocated VRAM is part of address space and as all locked pages in VRAM are copied in RAM, this would suggest to me, that all loaded normals are locked. Because if you reduce normals you hit very high VRAM, but no problems in game. however, leave normals alone, and do simple optimization, or use lower resolution textures, and you can crash with low VRAM usage. We should test this. Load very low textures with high normals and visa versa and measure both VRAM and RAM.

[Salvador]

As z stated they limit their VRAM usage to very low numbers. I'm sure if you limited your VRAM to 1-1.5GB you'd never come close to 3GB of physical RAM usage either.

One of the people in this thread crashes with 1200mb VRAM. Whilst Neovalen can run perfectly fine with 2900VRAM with half sized normals. How much of the VRAM is allocated by Skyrim is impossible to find out. But all that is allocated by skyrim as locked (not to be swapped out in case of need) is duplicated within RAM (allocation, not literally copied, but since their is a 4GB allocation limit, it is still a heavy burden) This duplication is used if you alt-tab out of the game. It copies all locked VRAM to RAM, and back when you alt-tab back in.