CTD at 3.1GB

[viking]

Some key posts on this and related threads (experts feel free to note any errors or insights):

• Post #11
• Post #27
• Post #40 (this is where the 500/430 ratio seems to have originated.)
• Post #66
• Post#278
• Post #313
• Post #1 (adjacent thread: Caution about memory optimisers)
• Post #64 (adjacent thread: Caution about memory optimisers)
• Post #68 (adjacent thread: Caution about memory optimisers)
• Post #70 (adjacent thread: Caution about memory optimisers)

Wiki article (draft)

 

Thanks,

STEP

 

OP follows

---

First of all I wanted to thank you all for the great work you have done with STEP. Skyrim is the first game I installed on this computer and you guys have made it an AMAZING game. That being said, I have an issue that I hope you can help me solve.

 

My setup:

Vanilla Skyrim

gtx670 w/ 4GB @1080p w/ latest driver

16GB Memory

3770K at about 4GHz

Windows 8 64bit

ENB 149

Ultra settings

Highest available texture/quality

Mod Organizer

Step 2.2.1 + Skyrim Revisited + others

 

I have noticed a post here and there saying that Skyrim can't really address more than about 3.1GB of memory without issues. This seems to jive with my experience, meaning I CTD every time my memory hits that mark, but I couldn't really find anything definitive on the topic. The issue with googling the topic is the pre 1.3 skyrim that couldn't address more than 2gb of memory.

 

My mod list is mostly based on STEP which is why I came here for help, with about ten mods added onto the end (Interesting NPCs, Detailed Cities, Economics, COT, and a couple others). The reason I haven't included my mod list is that it doesn't seem to matter. As long as I keep the memory usage below 3GB I can have pretty much any combination of mods.

 

What I have tried so far (in no particular order):

• resetting ini files
• removing enb
• not using attklt
• only using a new game
• removing all mods and adding one by one until issue crops up
• running as admin
• watching the papyrus log - it seems relatively clean, no obvious errors right before CTD

Yes, I can run STEP just fine without any issues, but I also never get near 3GB of memory. I have tracked VRAM usage as well and have seen a max of 2.7GB/4GB.

 

As an example of where I might run into issues: I start a new character with Alternate Start. I start with Breezehome. Run out of Whiterun, past the Brewry, up the hill to the bandits. Enter the cave (watching memory usage with Elys MemInfo), and it dies right after I see 3GB. I have this same issue not using AS, sitting through the intro, and then running over to whiterun.

 

I'm sorry if this post is all over the place. I have spent more than a week trying to solve this issue, and the only solution I have found is to reduce memory usage. I have got to the point where I can exchange two texture packs and get into the cave without a CTD, but with both I get a CTD. I didn't even think texture packs should even affect CTDs, but I'm relatively new to Skyrim on the PC, so I could be wrong. I also found I could get a bit further with ENB turned off, but would still crash once I got above 3GB of memory. Finally, if I reload a game after a CTD, I can play just fine...until I reach 3GB of memory.

 

I really hope you guys can help. I more than willing to try anything at this point, besides just disabling all of the mods.

[viking]

Sorry, I had changed the thread to solved...because at one point it seemed like we weren't going to get any further. This has obviously changed so thanks to whomever changed it back.

[Salvador]

VRAM usage is almost never fixed. It's dynamic + memory buffer.

 

Which is why it's pretty much impossible to measure it easily or accurately for any game. No, MSI Afterburner or GPU-Z won't do the job, as they will only show you the allocated memory, with no regards as to what is actually being buffered to the memory, how much is used by the application, how much by background process or system UI.

 

Oh, and it's Besi·dilo.

 

And no, Skyrim isn't "partially DirectX 9". It's fully DirectX 9 and there are no Dx10/11 executables. So the normal memory limitations apply.

 

Hmmm, how did I got your name wrong?.. apologies.

 

I believed it was DX9 only, yet Viking said it could partially be DX10. If DX9 only this means that all allocated VRAM to skyrim is mirrored in system memory.

Yes I know it is all allocated and not necessarily used. But this should suggest, that any increase in dynamic VRAM (or system VRAM) is mirrored in RAM. Therefore, by just increasing texture sizes when you reached your VRAM cap, you could perhaps get an idea on how much is really mirrored (allocated. The problem occurs with 3.1GB allocated memory).

 

However, I need to find the link, in original windows 7 RAM allocated by sytem was shown, and because of the improved pre-fetch, it had very high memory usage numbers, which upset quite a lot of people, so they changed that. So even the allocation numbers are somewhat incorrect.

 

Edit: fixed names

 

Since there is an absolute allocation cap at 4GB, and since allocation has improved considerably (read most skyrim updates), especially in combination with ATTK. You can argue that when you get CTD, you have used most of the allocated memory. If not, there is still room for improvement. Usually when the process needs to allocate more, it removes blocks (or pages) it believes it no longer requires. If it fails to do so, it is unable to allocate more.

[TechAngel85]

Thread isn't off track from what I've read. The idea I presented was just another possible cause of what is causing the issue and everyone has been discussing it. Z's suggestion is also to measure the VRAM and system RAM using the STEP benchmarks. I think the 4GB limitation could be a very real possibility of the cause of the issue. Now, the only real way to test if this is a RAM limitation issue is to have others heavily mod and use RAM heavy mods to push their systems to and beyond what you have done. If everyone gets a crash between 3 and 3.5GB, then it would be safe to say that we've found the limit to modding Skyrim when it comes to RAM useage.
 

Sorry I had changed the thread to solved...because at one point it seemed like we weren't going to get any further. This has obviously changed so thanks to whomever changed it back.

That was me. We don't mark anything as solved unless it is truly solved. If it remains unclear or unknown what caused the issue, then it will not be marked as solved. Even if it's no longer being discussed.


I'd like to point out, if it hasn't already, for anyone coming into this thread and reading up on it, that this is not a STEP issue as hundreds are running pure STEP without incident.

[Salvador]

Thread isn't off track from what I've read. The idea I presented was just another possible cause of what is causing the issue and everyone has been discussing it. Z's suggestion is also to measure the VRAM and system RAM using the STEP benchmarks. I think the 4GB limitation could be a very real possibility of the cause of the issue. Now, the only real way to test if this is a RAM limitation issue is to have others heavily mod and use RAM heavy mods to push their systems to and beyond what you have done. If everyone gets a crash between 3 and 3.5GB, then it would be safe to say that we've found the limit to modding Skyrim when it comes to RAM useage.

 

I'd like to point out, if it hasn't already, that this isn't a STEP issue as hundreds are running STEP without problems.

I fully agree.

 

Would be nice to see memory hogs that do not increase textures. That way the mirroring can be tested... The benchmark the guy used for testing ATTK was cheese wedges. These are duplications, and therefore require no additional textures or VRAM presumably. But their havoc needs calculations and thus a lot of memory. Is that an idea?

BTW I couldn't find the link about the prefetch stuff I was talking about, I know it was from Mark Russinovich and this comes close. Although it is no longer shown in the task manager, that is still happening!

 

EDIT: Perhaps not the best idea. The graphs that were provided by xeightballx show no or little increase in RAM usage (or CPU?), but he already reached his VRAM max. Also VRAM and RAM usage remained high, even though the game had crashed?

[MontyMM]

... we still are not sure exactly how Skyrim manages its vram. A link earlier in this thread (and actually in your first link) discusses the windows patch that helps manage vram outside of the address space of the program. In versions of directx earlier than 10, the vram is mirrored in the application address space, but I haven't seen any clear indications of skyrim being strictly directx 10 or strictly directx 9 (please link if you can), and so, in my opinion, we are unable to answer with any certainty if all vram is always mirrored in application ram for skyrim.

 

I think this is a key question.  The wording in the KB article (attached to a hotfix), suggests that Windows itself was modified to improve the problematic memory allocation of DX9:

 

If an application creates its own in-memory copy of its video resources, or the application uses DirectX 9 or an earlier version, the virtual address space contains the WDDM video memory manager's virtualized range and the application's copy. Applications that use graphics APIs that are earlier than DirectX 10 and that target GPUs that have large amounts of video memory can easily exhaust their virtual address space.

 

To address this problem, Microsoft is changing the way that the video memory manager maintains the content of video memory resources. This change is being made so that a permanent virtual address range does not have to be used for each virtualized allocation. With the new approach, only allocations that are created as "lockable" consume space in the virtual address space of the application. Allocations that are not created as "lockable" do not consume space. This approach significantly reduces the virtual address space that is used. Therefore, the application can run on large video memory configurations without reaching the limits.

 

This seems to leave open the question of how much of Skyrim's memory texture caching is inherent to its DX9 origins, or a (possibly configurable) feature of the engine.  Perhaps someone knows with certainty the DX9 memory behaviour on later versions of Windows?

[Salvador]

 

... we still are not sure exactly how Skyrim manages its vram. A link earlier in this thread (and actually in your first link) discusses the windows patch that helps manage vram outside of the address space of the program. In versions of directx earlier than 10' date=' the vram is mirrored in the application address space, but I haven't seen any clear indications of skyrim being strictly directx 10 or strictly directx 9 (please link if you can), and so, in my opinion, we are unable to answer with any certainty if all vram is always mirrored in application ram for skyrim.

I think this is a key question.  The wording in the KB article (attached to a hotfix), suggests that Windows itself was modified to improve the problematic memory allocation of DX9:

 

If an application creates its own in-memory copy of its video resources, or the application uses DirectX 9 or an earlier version, the virtual address space contains the WDDM video memory manager's virtualized range and the application's copy. Applications that use graphics APIs that are earlier than DirectX 10 and that target GPUs that have large amounts of video memory can easily exhaust their virtual address space.

 

To address this problem, Microsoft is changing the way that the video memory manager maintains the content of video memory resources. This change is being made so that a permanent virtual address range does not have to be used for each virtualized allocation. With the new approach, only allocations that are created as "lockable" consume space in the virtual address space of the application. Allocations that are not created as "lockable" do not consume space. This approach significantly reduces the virtual address space that is used. Therefore, the application can run on large video memory configurations without reaching the limits.

 

This seems to leave open the question of how much of Skyrim's memory texture caching is inherent to its DX9 origins, or a (possibly configurable) feature of the engine.  Perhaps someone knows with certainty the DX9 memory behaviour on later versions of Windows?

 

Install skyrim on XP x64 with only DX9 installed and see if it runs. If it does, it certainly doesn't use DX10.

 

What is defined by lockable? And is all allocated VRAM allocated by Skyrim? Or part (as I assume) by the driver to process the images.... Would it make a difference to run AA through driver or through skyrim? Would this be a separate process? ENB DLL (and therefore I assume AA through Skyrim) takes up memory in the skyrim process for sure. 

 

EDIT: Says on the box, DirectX 9c video card. So no DX10, although I am sure most people already assumed so. Windows did fix allocation. Now the full 2GB isn't mirrored anymore, but the mirror increases with VRAM demand from the process. But apparently that is dependant on "Lockable" allocation

 

EDIT EDIT: apologies, VRAM is part of the allocated virtual memory space. And all "lockable" allocations take space within the allocated virtual memory. However, VRAM used by the process is mirrored in RAM for DX9. The question now remains. Is all lockable allocated VRAM mirrored? What is lockable?

[MontyMM]

Install skyrim on XP x64 with only DX9 installed and see if it runs. If it does, it certainly doesn't use DX10.

Just to clarify - I'm not questioning whether Skyrim is DX9/10.  The article seems to suggest that Windows was patched to improve the memory handling of legacy DX9 applications, without updating them to DX10. 

[TechAngel85]

The way you're highlighted text reads (and this just might be me), is that it depends on how the DX9 application (game) is written to handle the allocations. ...allocations are "written" as lockable or not. This makes a difference on how they are handled. It's probably going to be impossible to know how the allocations are written for Skyrim.

 

Again, the only test I can think of to check out the theory of the game reaching it's RAM limit is to have several people push it to that limit and see if they all have crashes occur.

[MontyMM]

The way you're highlighted text reads (and this just might be me) is that it depends on how the DX9 application (game) is written to handle the allocations. ...allocations are [i']"written"[/i] as lockable or not. This makes a difference on how they are handled. It's probably going to be impossible to know how the allocations are written for Skyrim.

 

Again, the only test I can think of to check out the theory of the game reaching it's RAM limit is to have several people push it to that limit and see if they all have crashes occur.

I'm assuming for a moment that the RAM limit crash does exist.  What I'm curious about is whether it might be the Skyrim engine deciding to cache the textures in RAM (in which case there is the slight possibility of this being configurable) or whether this is certainly an inescapable behaviour of DX9.  There might be a techie around here who can testify.

[viking]

techangel85: Thanks for the clarification. I'll probably just let a mod change the title from now on.

 

I agree with your testing idea. We need a couple of people, besides Neo and myself, that have 6GB+ RAM, to help test the limit. The cheese wedge idea is interesting, but I'm not sure how to implement that.

 

If we would rather go the mod way, we need to determine several Ram heavy/Vram light mods, such as Interesting NPCs, that we can use. Any other recommendations on how we can run this?

[TechAngel85]

If a test configuration can be put together, I'd be willing to test this. I have 8GB of RAM with 1GB of VRAM.

[Salvador]

 

The way you're highlighted text reads (and this just might be me) is that it depends on how the DX9 application (game) is written to handle the allocations. ...allocations are [i']"written"[/i] as lockable or not. This makes a difference on how they are handled. It's probably going to be impossible to know how the allocations are written for Skyrim.

 

Again, the only test I can think of to check out the theory of the game reaching it's RAM limit is to have several people push it to that limit and see if they all have crashes occur.

I'm assuming for a moment that the RAM limit crash does exist.  What I'm curious about is whether it might be the Skyrim engine deciding to cache the textures in RAM (in which case there is the slight possibility of this being configurable) or whether this is certainly an inescapable behaviour of DX9.  There might be a techie around here who can testify.

 

Yes this is how I read it as well. There used to be a full copy of the allocated VRAM in physical memory. With DirectX10 this is handled by WDDM and thus no longer takes up space in the virtual memory allocation.  If an application creates its own in-memory copy of its video resource OR uses Direct X9, they still produce a copy. Memory is lockable or not, in other words, it cannot or can be swapped out and make space for other allocations. The VRAM that is normally tagged as able to be swapped out (unlocked) (hence real VRAM cap is higher than when all VRAM is allocated), is not copied into virtual memory space. Now we need to figure out what is written as locked. Perhaps Jason2112 can help out?

 

EDIT: I should say written rather than copied into virtual address space. I do wonder how that works, because basically it means that the process can use considerably more VRAM, as it is no longer considered part of the address space....?

[Salvador]

If a test configuration can be put together I'd be willing to test this. I have 8GB of RAM with 1GB of VRAM.

Same here. Gonna test the cheese wedges right now. See how much mem they take up in vanilla skyrim.

[TechAngel85]

Seems like the 4GB limit has also be confirmed by others on Nexus: https://forums.nexusmods.com/index.php?/topic/787596-skyrim-ram-usage-increases/

 

EDIT:

After Googling and reading some more, I'm convinced that this CTD issue is caused by Skyrim reaching it's RAM limit. We only need to confirm by a few more testing. Viking and Neovalen have both already confirmed this issue at around 3GB of usage already. The fix for this would only be to decrease your mod load.

[Neovalen]

Seems like the 4GB limit has also be confirmed by others on Nexus: https://forums.nexusmods.com/index.php?/topic/787596-skyrim-ram-usage-increases/

 

EDIT:

After Googling and reading some more, I'm convinced that this CTD issue is caused by Skyrim reaching it's RAM limit. We only need to confirm by a few more testing. Viking and Neovalen have both already confirmed this issue at around 3GB of usage already. The fix for this would only be to decrease your mod load.

 

Unfortunately so yes. Going to go through and see who the big culprits are. Even with all my hardware I apparently can't use all 2048 textures without hitting cap.