CTD at 3.1GB

[viking]

Some key posts on this and related threads (experts feel free to note any errors or insights):

• Post #11
• Post #27
• Post #40 (this is where the 500/430 ratio seems to have originated.)
• Post #66
• Post#278
• Post #313
• Post #1 (adjacent thread: Caution about memory optimisers)
• Post #64 (adjacent thread: Caution about memory optimisers)
• Post #68 (adjacent thread: Caution about memory optimisers)
• Post #70 (adjacent thread: Caution about memory optimisers)

Wiki article (draft)

 

Thanks,

STEP

 

OP follows

---

First of all I wanted to thank you all for the great work you have done with STEP. Skyrim is the first game I installed on this computer and you guys have made it an AMAZING game. That being said, I have an issue that I hope you can help me solve.

 

My setup:

Vanilla Skyrim

gtx670 w/ 4GB @1080p w/ latest driver

16GB Memory

3770K at about 4GHz

Windows 8 64bit

ENB 149

Ultra settings

Highest available texture/quality

Mod Organizer

Step 2.2.1 + Skyrim Revisited + others

 

I have noticed a post here and there saying that Skyrim can't really address more than about 3.1GB of memory without issues. This seems to jive with my experience, meaning I CTD every time my memory hits that mark, but I couldn't really find anything definitive on the topic. The issue with googling the topic is the pre 1.3 skyrim that couldn't address more than 2gb of memory.

 

My mod list is mostly based on STEP which is why I came here for help, with about ten mods added onto the end (Interesting NPCs, Detailed Cities, Economics, COT, and a couple others). The reason I haven't included my mod list is that it doesn't seem to matter. As long as I keep the memory usage below 3GB I can have pretty much any combination of mods.

 

What I have tried so far (in no particular order):

• resetting ini files
• removing enb
• not using attklt
• only using a new game
• removing all mods and adding one by one until issue crops up
• running as admin
• watching the papyrus log - it seems relatively clean, no obvious errors right before CTD

Yes, I can run STEP just fine without any issues, but I also never get near 3GB of memory. I have tracked VRAM usage as well and have seen a max of 2.7GB/4GB.

 

As an example of where I might run into issues: I start a new character with Alternate Start. I start with Breezehome. Run out of Whiterun, past the Brewry, up the hill to the bandits. Enter the cave (watching memory usage with Elys MemInfo), and it dies right after I see 3GB. I have this same issue not using AS, sitting through the intro, and then running over to whiterun.

 

I'm sorry if this post is all over the place. I have spent more than a week trying to solve this issue, and the only solution I have found is to reduce memory usage. I have got to the point where I can exchange two texture packs and get into the cave without a CTD, but with both I get a CTD. I didn't even think texture packs should even affect CTDs, but I'm relatively new to Skyrim on the PC, so I could be wrong. I also found I could get a bit further with ENB turned off, but would still crash once I got above 3GB of memory. Finally, if I reload a game after a CTD, I can play just fine...until I reach 3GB of memory.

 

I really hope you guys can help. I more than willing to try anything at this point, besides just disabling all of the mods.

[Aiyen]

Very interesting read monty! And makes alot of sense as well!

[torminater]

Well, I can say, that on my laptop I didn't get a single CTD since using cleanmem. Even though it shouldn't work according to your article, it does for me at least sort out the issue with the 3 GB crash. Also, why would I get a regular sound stutter, if this program wouldn't do anything? Is it that the sound task/process/whatever is taken out of the task group?

[DoYouEvenModBro]

Post has been deleted

[MontyMM]

I actually stopped using cleanmem because I started getting freezing that I thought was caused by cleanmem (turns out it wasn't.) I ended up solving the 3 gb CTD issue WITHOUT cleanmem by doing the following:

 

Used HD DLC Optimized Hybrid (1024 + 2048) option

Reduced the three texture packs (2k, serious, SRO) to 1k (except I'm using 4096 SRO mountains) with optimizer textures

Reduced the SFO 8k high res pines optional add-on to 4k with OT

 

This solved my CTD without cleanmem. I'm interested to see if I can put the 3 texture packs back to 2k and still stay below 3 gb with cleanmem but I doubt it. 

 

The reason I got rid of cleanmem was because I was getting freezing every few minutes but it turned out to be caused by save file bloat and after running "CleanInvalidRegistrations" with SKSE, my save file shrank by half the size and the stutter/freezes went away completely. Apparently had 255 invalid registrations.

Makes sense to me.

 

Well I can say, that on my laptop I didn't get a single CTD since using cleanmem. Even though it shouldn't work according to your article, it does for me at least sort out the issue with the 3 GB crash. Also, why would I get a regular sound stutter, if this program wouldn't do anything? Is it that the sound task/process/whatever is taken out of the task group?

If I had to guess, I'd say the stutter is probably a certain number of pages being forced to the page file (and then perhaps page faulting as they are immediately read back into RAM) .  It's also possible that there could be a hiccup when the Memory Manager reclassifies pages to the Standby List. There's tons of articles on how unwanted working set trimming can cause performance issues in certain scenarios. Here's one.

 

It seems very unlikely to me that the 3GB crash is really avoided in this way.  The amount of virtual memory that Skyrim is using cannot be reduced unless Skyrim itself reduces it.  The thing to remember is that Task Manager does not give you an accurate figure for the Skyrim process.  If you go into Task Manager processes tab, click on 'View', and choose 'Select Columns', you'll notice that the default Memory column is just the working set size.  If the working set for the active process has been forcibly trimmed, then this figure can be very misleading - a portion of the Skyrim pages in RAM have been reclassified as being on the Standby List, and is no longer counted in that column.  It's a lot like an accountant who moves expenditure from one column to another to make the books look better - but this doesn't actually change the amount of money you've spent.

 

You can select a few more columns to view other aspects of memory use, but none of them will give you a true figure for what Skyrim is actually using.  The 'Commit' column shows how much virtual memory has been committed to the process, but not necessarily the amount it is actually using.  For any problems related to the memory address limits of a 32bit executable, it's this elusive real total of virtual memory use that matters.  I'd always assumed that Skyrim Monitor profiled this figure accurately, but (and I'm not at home to check) it may also just be quoting that misleading working set figure.

[Neovalen]

I actually stopped using cleanmem because I started getting freezing that I thought was caused by cleanmem (turns out it wasn't.) I ended up solving the 3 gb CTD issue WITHOUT cleanmem by doing the following:

 

Used HD DLC Optimized Hybrid (1024 + 2048) option

Reduced the three texture packs (2k, serious, SRO) to 1k (except I'm using 4096 SRO mountains) with optimizer textures

Reduced the SFO 8k high res pines optional add-on to 4k with OT

 

This solved my CTD without cleanmem. I'm interested to see if I can put the 3 texture packs back to 2k and still stay below 3 gb with cleanmem but I doubt it. 

 

The reason I got rid of cleanmem was because I was getting freezing every few minutes but it turned out to be caused by save file bloat and after running "CleanInvalidRegistrations" with SKSE, my save file shrank by half the size and the stutter/freezes went away completely. Apparently had 255 invalid registrations. 

I had no freezing and then using cleanmem caused me to have problems. Granted as we all know it was on Windows8 but the cleanmem site says it's compatible so... take the incompatible claim with a grain of salt.

 

The only real way to prevent crashes is memory management via reductions in texture sizes, makes perfect sense to me whereas cleanmem never did (but figured as an intrepid tester I would test out).

[MontyMM]

In light of all this chat about working sets and so on, it's worth checking what the figure is that Skyrim Monitor is reporting. Is it an accurate figure reported by Skyrim, or perhaps it's just pulling up the working set figure, same as the task manager? Could someone compare the figures, and also see if the Skyrim Monitor figure drops when Cleanmem trims? If so, we at least have a very likely answer to the odd figure of 3.1gb - it's only reporting the working set, and the real memory usage is simply hitting 4gb.

[Vond]

Skyrim Performance Monitor as well as Elys MemInfo both report the same as taskmanager, you'll see huge dips in the RAM graph on performance monitor when using cleanmem

[MontyMM]

Thanks Vond. At least this removes the mystery, and tallies with the official response from Bethesda, which implies the simple 4gb limit:

 

"Skyrim is a 32-bit program. Even if you're running on a 64-bit operating system, Skyrim was not written to utilize these features. As such, the memory that the game will utilize prior to crashing is extremely limited. We apologize for the inconvenience."

 

I'll update the sticky post.  This means we know that we have a limit for texture use, and we ought to take this into consideration for the future of the guides.

[DoYouEvenModBro]

Post has been deleted

[MontyMM]

Hard to say, josh. It would depend on how good a job you did of compression vs what the author did. You would certainly expect that the author's reduced version would be better, since he would have access to the highest quality original assets, and perhaps better tools, but it is not guaranteed.

[z929669]

Thanks Vond. At least this removes the mystery, and tallies with the official response from Bethesda, which implies the simple 4gb limit:

 

"Skyrim is a 32-bit program. Even if you're running on a 64-bit operating system, Skyrim was not written to utilize these features. As such, the memory that the game will utilize prior to crashing is extremely limited. We apologize for the inconvenience."

 

I'll update the sticky post.  This means we know that we have a limit for texture use, and we ought to take this into consideration for the future of the guides.

Agree. Need to set texture caps for Extreme STEP and correspondingly to Baseline and Performance STEP (only Baseline is currently defined as the existing guide). Extreme STEP should probably not use any textures larger than 2k in any exteriors (accept for dragons arguably). Max should be 1k for Baseline 9with maybe a few exceptions.

 

Texture sizes can pretty much be limitless for Baseline and Extreme stem with regard to interiors ... the trick is identifying all interior textures. Dungeons are obvious. Cluter can be problematic.

 

Extreme STEP will be more apt to handle Packs and instruction-intensive mods.

[DoYouEvenModBro]

Post has been deleted

[Neovalen]

I run 2k major texture packs with half normals.

[DoYouEvenModBro]

Post has been deleted

[Neovalen]

Full 1k will obviously use less VRAM/RAM but 2k textures with 1k normals saves you memory with only minor visual loss compared to going 1k for all.

 

Basically you save 1/4 instead of saving 1/2.