viking

Sorry Nshell, that isn't how it works. There are system functions (called "system calls") that the program (skyrim) calls which expect a 32-bit integer for a 32-bit program, irrespective of whether the OS is 32 or 64 bit. It isn't a matter of being fast or slow, it simply wouldn't work. If you had multiple lists of pointers each with 2^32 entries, each list would point to the same memory locations. This limitation of 32-bit is a large driving factor behind the adoption of 64-bit applications. aleksanderdev, I am surprised you are able to get up to 5.2GB of VRAM, mind sharing your setup, modlist, and location where you reached that?

Because Bethesda. And consoles. And because it was written as a 32-bit program. Bethesda could theoretically rewrite Skyrim as 64-bit, but the only people it would help would be PC modders, so they have little incentive to invest in it. "Requesting multiple address spaces" is not really possible. All of the variables are 32 bit and memory access functions take a single 32 bit integer, so even if you requested "multiple address spaces" you wouldn't be able to tell the OS that you wanted to get to them. EDIT: I should be clear: The maximum size of a variable is 32 bits; an integer (which is used as a memory pointer) is 32 bits, but other types are of different sizes <= 32bits in a 32-bit program.

I'm not entirely sure what you are getting at, Nshell. Yes, device drivers need to do some form of 64-bit arithmetic, but this is because they are in kernel space, not user space. Even if you ran Skyrim as administrator, this would not run in kernel space (thus not do 64-bit arithmetic). Also, you are correct about the registry value that sets how much memory you can use in 32-bit versions of windows vista and windows 7, but again, I'm not sure how it is relelvant. Skyrim is limited to 4GB of memory (with the LAA switch enabled) because it is a 32-bit program. C++ based 32-bit programs, which Skyrim is, uses 32-bit integers to address the memory addresses in their virtual memory tables. This is a limitation of both how the program was written, and the specific compiler used to compile the program (thus you cannot typically just re-compile it).

Just as an FYI, this is not solved in the 1.9 beta.

JudgementJay, could you post your mod list? I have been playing around with this for a good part of the day, and I'm just amazed that you can get ~1.8GB of Ram with TPC at twice my resolution. Just jumping to riverwood with a new character puts me at more than 2.1GB. The worst locations for me are in cities, particularly Windhelm, which jumping there puts me at about 2.6GB...and entering Candlehearth Hall then adds another 500MB and CTD. These numbers are with SRO and Skyrim HD disabled (which seemed to add the most based on my testing).

Your point about large sound files being an issue is interesting. I do have two mods with 500mb+ (on disk) of sound files. I'll try to see how that affects memory when I get a chance. As for how enb AA affects memory usage. 119 was the last version of enb to support hardware AA, which would have been outside of the memory space of skyrim. Newer versions, however, are software based. The dll version of enb, I believe, is loaded by skyrim into skyrim's addressable memory (can someone confirm this?). Enb does not use a large amount of ram in my experience, but does affect it some. It is unfortunate that the cheese wedge idea didn't work for our purposes. I am also curious what attk does to allow more wedges if more wedges don't affect memory usage.

Z, I am really glad you have been posting here, and your benchmark is excellent and informative. Unfortunately I can't seem to get a couple of your graphs on your benchmark to enlarge...not sure if that is just my issue or not, but even still, the chart gives a pretty good idea of what you mean. I do think you have enough vram to experience what we are talking about, just not in the normal play of the game. If you are interested, you could try the "cheese wedge effect" discussed earlier in the thread and see if you get a ctd at 3.1GB. I am pretty sure you are right about Process Explorer listing dynamic and static vram usage, but I'll have to look closer when I'm at a windows computer. I have tried running it without Attk. It does lower the ram usage slightly, but not significantly. I will have to try the full version and see if it makes a difference. I'll have to be honest, I haven't read the description of Attk, so I don't know exactly what it does (besides help manage memory somehow).Â

If anyone comes up with an easy/quick way to test this, please post it here. I'll second torminater, at least for 2.2.0 with all the max options I didn't have this issue.

techangel85: Thanks for the clarification. I'll probably just let a mod change the title from now on. I agree with your testing idea. We need a couple of people, besides Neo and myself, that have 6GB+ RAM, to help test the limit. The cheese wedge idea is interesting, but I'm not sure how to implement that. If we would rather go the mod way, we need to determine several Ram heavy/Vram light mods, such as Interesting NPCs, that we can use. Any other recommendations on how we can run this?

Sorry, I had changed the thread to solved...because at one point it seemed like we weren't going to get any further. This has obviously changed so thanks to whomever changed it back.

I'm going to try to get the thread back on track: Besidilo, I'm not sure who got your name wrong. If it was me, sorry about that. I know very little about VRAM. A link with more info would be more helpful. The more we understand about the behavior of VRAM the more likely we are to understand this issue. I read in some forum somewhere (highly reliable...) that Skyrim was partially directx10. Don't trust me on that. It makes far more sense for it to be directx9, especially since xbox is limited to that. Salvador, I agree that the references is Besidilo's links were excellent. I just had a hard time following the first one. I didn't notice anything incorrect in the links, there were just a few times when things like the maximum ram in a system was taken to be a constant across all systems. Again, the information was correct, just difficult to follow and left some key pieces of information out IMO. z recommended several test we can run to test the memory/vram correlation, and I will run them this weekend. Are there any other ideas on how we can get to the root of this issue?Â

I think that this is a very important point. Assuming memory is the issue (which can't do with complete certainty), the issue very easily could be a limitation of the CreationKit engine (is that what it's called?).

I shouldn't have used the term "pre-cached" since caching memory already implies a certain buffer. My point was to not get confused by the extra memory that would be cached in any case, any application may appear to use more memory than it actually needs, up until the point it runs out completely. You can't measure VRAM usage accurately, and that has been addressed a lot of times already. You're probably thinking about the extra textures that has been buffered to the video RAM, but aren't actually used by the engine. With regards to the memory limitation, Dx9 mirrors VRAM onto the system memory, in turn reducing your actual system RAM limit by the amount of VRAM in use for any process. Microsoft got rid of this limitation in DirectX 10. x86 processes with Large Address Aware patch can use up to 4GB total system memory. In other words, it's complicated. But it can be assumed that a crash would occur when Skyrim with LAA suddenly requires more than 4GB of RAM+VRAM. How to measure that point accurately is another matter. At least that's how I understand it. It is a shame that we keep posting at the same time. I assume that your second paragraph means that virtual memory is different than physical memory, and in that we are in agreement. I'm not sure why you think that you can't measure VRAM usage accurately. I agree it is difficult to pin VRAM usage to a particular application, but you can get pretty good idea. I addressed your other points in my previous post.

Alright. I finally had a chance to read through both of these. Up front, the gamasutra article is quite misleading and I wouldn't recommend it. The links it references are all high quality and the author did know what he was talking about (for the most part), but leaves out some essential bits if you aren't familiar with the area. The second link is accurate, but somewhat dated. I don't really feel like going into more detail here, but can if someone is interested. In response to your comment, we still are not sure exactly how Skyrim manages its vram. A link earlier in this thread (and actually in your first link) discusses the windows patch that helps manage vram outside of the address space of the program. In versions of directx earlier than 10, the vram is mirrored in the application address space, but I haven't seen any clear indications of skyrim being strictly directx 10 or strictly directx 9 (please link if you can), and so, in my opinion, we are unable to answer with any certainty if all vram is always mirrored in application ram for skyrim. Note that this is a different issue than if vram on the graphics card is in the same address space as the application. Based on my experience, this is not the case. My skyrim ram + skyrim vram often exceed 4gb. As for if you should get a 4GB graphics card, I often go above 2GB of VRAM, but have yet to hit 3GB vram. I think you will still find an advantage of having a 3GB+ graphics card.

Besidilo: I have not yet read the articles that you linked to, but they look very interesting. I also am not sure what you mean by "pre-cached" memory, but would be interested if you could explain more. Your comment that LAA applications are limited to 4GB for System Ram + VRam, however, doesn't line up with what I have seen. I often see my VRAM at about 2.7 GB and my Ram at about 3.1 GB...which is obviously more than 4GB. The only way I can reconcile your comment with my experience is if the VRAM that is mirrored in RAM is not counted twice. That is certainly a possibility. I apologize if this is addressed in your articles, and feel free to respond to that effect if that is the case. I would like to point out for Salvador's sake, and for everyone else reading this thread, that we have done only very limited tests about the mirroring of VRAM and RAM. It is not outside the realm of possibility that the VRAM is temporarily loaded into memory before being forwarded to VRAM...or something to that effect. We also don't know for sure if the VRAM is considered as part of the 4GB address space. All we know for certain at this point is that texture files in skyrim DO affect VRAM usage AND Ram usage. I'll update this comment once I have a chance to read Besidilo's links, they look to have a lot of good information in them.

I only have a minute, but I wanted to clear something up before this thread gets too off track. In reference to System Memory (not VRAM): Memory is only paged out to disk if the entirety of physical memory is used up. This is not managed by individual processes, thus this cannot be a limitation of the game engine and is not related to the amount of memory given to a process. To the process, all it sees is the virtual memory addresses. When the process wants to get something from memory, it tells the OS it wants to get (virtual) address, lets say, 57. The OS then translates that into either the physical memory location or the page location on disk. Now on to some of the comments: Disabling paging is a bad idea. It forces, as Salvador points out, all of the memory to be in physical memory and not paged to disk. Unfortunately, if the system runs out of physical memory it simply crashes. No ifs, ands or buts. It won't speed up a program because the OS already is trying to keep everything in memory that it possibly can. On a more technical note, some OSes may page things out to disk before it is necessary; however it keeps it in memory too. This just allows the OS to not have to copy a page from memory to disk in an "emergency". I don't know as much about VRAM usage. From z's post (sorry! I forgot the number's that follow!) it seems that when VRAM runs out, the system allocates some graphics memory in RAM as well. This will increase stuttering, but not cause CTDs. This is described in much more detail in z's performance testing. If the system got to the point where both VRAM and Memory were paged to disk...you're in trouble. If somehow the system had to allocate some page space on disk for VRAM, your game would be so slow it would be unplayable. Don't do it. Buy more ram. Salvador, you mention that this might be a Bethesda issue. That is currently one of the two likely possibilities, and the one we had agreed upon until z came along and suggested the possibility of a setup issue. Not bashing on z, I appreciate help from anyone who can give it. Salvador, you also mentioned the issue of DLLs. You are right, DLLs are allocated in the same address space so they will take up some of the memory. I have found, though I hadn't previously mentioned, that removing ENB does reduce memory usage, but only slightly. I still have the same 3.1GB limit in that case. Also, I don't think our STEP installments are all that similar, though no one else has posted their mod list. Mine, in particular, is not really STEP for more than textures and fixes. In particular, I have several very RAM heavy mods that are not in STEP (nor do I recommend that they be) and it is unlikely that others here have them all installed. Neo, based on his incredible Skyrim Revisited page, has somewhat similar install to mine, but again varies in some very large mods (like Interesting NPCs). I really do not mean to discourage discussion, I just wanted to make sure things stayed on track. I'm not an expert on this topic, by any means, and I am just as baffled by the 3.1GB cap as everyone else. I apologize in advance if I have been rude to anyone...I haven't had my coffee yet...

I'm not sure which of Neovalen's comments you are referring to. Also I'm not sure why it would be strange for this single process to use a lot of ram. I would be very happy if this is not a ram issue. Yes? I'm not sure where you are coming from. I tracked the RAM (not VRAM) usage using Process Explorer (which you are familiar with); it was tracking only TESV.exe. VRAM, which was system wide, was tracked using GPUz. I haven't had a chance to read through your benchmark, but I will. I am more than happy to do any tests you are interested in, because I would much rather find a setup issue than reduce mod count. Unfortunately I won't have a chance until this weekend to do any real tests (meaning redoing step from scratch). I'd like to point out that my numbers, and all of my experiences with ctds, were all using attklt. As for whether the "bleed over", as you called it, goes into the same address space, it would seem that it does; however we have no real "proof" beyond some anecdotal evidence and the microsoft directx patch details linked to earlier in this thread.

I am aware that my count of mods is not that high, but some of them use a ridiculous amount of memory. I wasn't certain before starting this thread that there is a (relatively) hard 3.1 GB ram limit, but now that it seems pretty certain, I may go back and do just what you suggest. I think I will still gain a bit from ddsopting the textures I've got and will start with that, because, unfortunately, I have already cut my non-texture mods just about to a minimum for me.

Mod List: But you should note that I haven't DDSopted anything besides HRDLC. I'm pretty sure the major culprits for me are Interesting NPCs and Detailed Cities, though I haven't spent much time determining memory usage of each since it sounds like a huge pain (and entirely location dependent).

So I ran three tests: All textures, No SRO, and No HRDLC/NO SRO using Process Explorer and GPUz for TESV Ram and system Vram monitoring respectively. I loaded a save game in the same location three times and immediately exited once it loaded. I know it isn't scientific or repeatable, but the results are surprising non the less. Full     1,702 MB VRAM 3,020 MB RAM No SRO   1,454 MB VRAM 2,560 MB RAM No HRDLC 1,357 MB VRAM 2,222 MB RAM So the math on these gives a VRAM to RAM usage of about 1:2 or 1:3. So irrespective of the exact ratio, it seems textures are mirrored in memory as part of the ~3.1GB RAM limit. Thanks to everyone, especially MontyMM, for helping me track this down. I consider the case closed and will move forward with experimenting with DDSopt and some of the ini settings to reduce my memory usage. I'll post to this thread if I find anything interesting. MontyMM, I'm happy to help with any experiments you may decide to do with this in the future, just shoot me a PM.

That's an interesting experiment. I might try reducing interior/exterior cell buffers and loading the data off of a ram disk/ssd or some combination thereof. The biggest ram usage for me seems to be when entering an interior space from a larger interior space (entering breezehome from within Whiterun), because both cells seem to stay resident in memory.

No, there isn't a simple "stop mirroring" command, it is an issue with directx9 and earlier. The url from Monty has more details:

That doesn't seem unrealistic to me. I'll try to run a test tonight and see if I can get some numbers to back it up.

TPC w/ SRO? I was trying to see if we could come up with a ratio just based on that plus disk space (~7gb compressed) of TPC, but the textures that are loaded are so dependent on the scene that I think it has to be ram vs vram, not ram vs disk space.

JudgmentJay - I assume you are basing this on personal experience? Any idea what kind of ratio of VRAM is loaded into RAM? It obviously isn't 1 to 1 (otherwise it couldn't use unlimited VRAM). I might run a test tonight to remove all of my textures from MO and see how Ram and VRAM change. I'll post results.