MO for Oblivion TES4LODGen memory use issues

[Luinithil]

Installing mods for Oblivion through MO, I encountered an issue where TES4LODGen (needed to run Really AEVWD, which is an object LOD mod) consistently runs out of memory when generating LOD files. Memory use as reported by Task Manager in Windows 7 shows the memory use balloons up to > 30000000 K, at which point LODGen usually fails. Nothing was significantly changed in my mod setup between Wrye Bash and MO except the addition of a few house mods, which AFAIK should not have added many (if any) LOD items to be generated.

 

If I run LODGen through MO with only the official plugins and a small handful of mods, the LOD files are successfully generated; however at some point it starts choking on lack of memory. I have 8GB of RAM on my laptop, and an i7 CPU and frankly I don't see why this should even be an issue. I've found no solution to this problem, since it doesn't seem to be any particular mod I've installed and thus have had to switch back to using Wrye Bash for the time being since installation and behaviour overall seem to work better with Oblivion. I would also note that when I've run TESLODGen after installing mods in Wrye Bash, all LOD files are generated with no problems and in less than 5 minutes, and RAM usage for LODGen never breaks 500 - 600K. I'm not sure what's going on here, since LODGen is a 32-bit app.

 

In the absence of anything else, I am wondering if the absence of correct archive invalidation settings might be at fault (see my other thread on the subject in this subforum) ? I'll admit this is reaching, but it was already at the point where Oblivion simply wouldn't start but crashed before the main menu. So I decided to head back to Wrye Bash.

 

This is highly frustrating, because I'd had 90% of my install already configured in MO--INI tweaks and such. In any case I am planning on trying to migrate to MO again, just not quite so soon as I've only just managed to get my Wrye Bash install done.

[Uhuru]

Have you got Archive Invalidation fixed because you will not get anywhere until you do.

See my reply to your 1st post on subject

Please respond to that post with yes or no so others can benefit as well as you.

Also if it was not fixed then more help can be given.

[tomtomtommy]

I have the same issue, need to run hit with RAEVWD and tons of mods, think it's when the TESLODGen.exe process hits around 3.5gb memory usage..

[Luinithil]

@Uhuru: Not that I think it matters now to TESLODGen whether AI is on, since I checked and did have Archive Invalidation working on two of the profiles I tested with previously, have managed to get it up and running properly currently on all profiles as far as I can tell-- and yet LODGen is using insane amounts of memory and running out of it same as usual when called from MO, which it *doesn't* when run on its own. Somewhere somehow something is leaking memory I think.

[nakakita]

I didn't have any problems running TES4LodGen through MO.. until today. Everything had been working well, except my FPS was horrible because of OBGE, so I decided to uninstall that, and mistakenly thought everything would continue working. I had been using version 3.0.15 without any issues. Installed some things (specifically this, this, this and this.) I redid my bashed patch, ran TES4LODgen.. and started getting errors, as well as insane memory usage. I don't have a screenshot of it, but TES4lodgen did try and use 3.6 GB of memory before crashing repeatedly.

 

Reverted to 2.2.2. Same problems. Deleted previous LODs. Same problems. Deleted several mods I'd installed last night which had added towns or villages and their compatibility patches (all the Arthmoor towns except the one near Frostcrag, then Nernie's Bravil and Waterfront) and I still had the hangups. Removed Qarl's DOF. Still problematic. Unchecked the enhanced water mods. Still failing to complete. Finally I unchecked Vanilla Essence, deleted the LODs from overwrite yet again and suddenly it works. I'm using other texture mods without any problems. I have other mods that mess with LODs, and have no problems. I really don't understand why it randomly hangs up. I had Qarl's gigantic texture pack installed at one time and it still worked, but now MO wants to fail over this. 

 

Screenshots of my errors, MO's memory usage at the time of these errors, and what it all looked like when it finally worked:

[Luinithil]

Nakakita's issue looks similar to mine-- I did check though and I am using none of the mods listed above. The only mods I have that add things to \DistantLOD are the UOP and Artifacts. It happens whether using 2.2.2 or 3.0.15.

 

Log snippet from the end of LODGen 3.0.15's process, it usually fails around this point-- either replacing the Toddland LODs or Sheogorath's realm.

[00:19] Background Loader: [D:\Program Files\Steam Games\SteamApps\common\Oblivion\Data\] Setting Resource Path.
[00:19] Background Loader: finished
[00:20] LOD Generator: starting
[00:20] LOD Generator: Replacing Tamriel [WRLD:0000003C]
[02:48] LOD Generator: Replacing Toddland [WRLD:00000858]
[02:52] LOD Generator: Replacing ToddTestLand2 [WRLD:00000879]
[02:52] LOD Generator: Replacing MQ15TestWorld3 [WRLD:0000145B]
[02:52] LOD Generator: Replacing EmptyWorld [WRLD:0000568F]
[02:52] LOD Generator: Replacing SEWorld "Realm of Sheogorath" [WRLD:00009F18]
[02:52] LOD Generator: <Error: Out of memory>

 

 

 

 

 

Screenshots taken during the process:

[nakakita]

I'll also note that I am using the UOP, and Artifacts. Neither of these have caused any problems in the past with the LODGen, and when the program finally worked those two mods were loaded. Since my hangup was Vanilla Essence, I removed folders until LODGen worked. Landscape and LandscapeLod (both in the textures folder) were not the cause of the hangup. Something in the trees folder was. I'm going to pick through that until I see what the problem is. It doesn't really make much sense though.

 

Edit: There are two subfolders in trees, leaves and billboards. Removing billboards resolves the problem. I tried finding the exact problem files, but oddly even one or two of the files in billboards will cause the memory to soar to 1,500,000... then it completes. I even got 6 of these to work together, but when I added two more that had also worked when used alone, it crashed. I only found one file (shrubboxwood.dds) that caused it to crash on it's own. Apparently these are just too intensive for LODgen to deal with while running through MO. Absolutely no problems running the same files through LODgen when they're in the data directory. Memory usage stays below a modest 280,000k, and that's with every file in the folder including the problematic one.

[Mystery2me]

Having the same issue with the memory error.....anyone ever come up with a fix for this?

[nakakita]

I never found a solution other than to remove everything in the Textures\Trees\billboards folder if you happen to have one. Of course it works perfectly if you're running the program outside of MO even with those files, but with them, in my experience at least, LODgen will always hang indefinitely. Rather frustrating.

[hishutup]

ok you can use the 4GB patcher it bring the memory up to 3.7GB but mines been running for almost 8 times longer through MO... Which is absurd. MO is unusable with patchers including SUM 

https://i58.tinypic.com/2hi2s9e.png

CPU maxed out on one core for some reason

[DoubleYou]

I bet if you loaded all your MO mods into data and ran the TES4LODGen it would do the same thing. It's not MO's fault that TES4LODGen requires too much memory to process your modlist.

[hishutup]

nope... Its MO after removing MO from the equation, meaning copying over all the mod folders and mods. It runs without using a ridiculous amount of resources

read my bug report 

especially check out the pics

https://issue.tannin.eu/tbg/modorganizer/issues/759

I tried it three times both ways and it was about the same. With mo it takes longer than 30min followed up with crash. Whereas removing MO it took less than 2min and was just fine. Don't get me wrong I love MO but I surprised you haven't noticed application running slowly

Yes, I moved the mods into the data directory just like manual.

Edited August 10, 2014 by hishutup

[DoubleYou]

How come your picture shows a clear usage of about 3.7 gb of memory, yet your bug report says they can't use more than 1 gb?

 

And then, how can Skyrim be run with well over 1 gb of memory all the time?

 

The only thing I can imagine, is because MO redirects created files to Overwrite, it takes longer writing the files, and any memory leaks in the programs being run would be intensified.

[hishutup]

thats for Skyproc patchers...

Yet again the 1GB thing is for skyproc

if it did then why does it take upwards of 15x the time and then fail... TES4LODGen only used <400MB as ram and didn't move above 7% on a quad core but with MO it just kept increasing until it ran out of memory and using 25% the entire time or 100% on one core.

I doubt its a memory leak otherwise skyrim would hate it. 

 

I generated the tamriel files outside MO and stuck them as a mod and it still takes forever and fails.

 

I changed around the bug report a little also

Edited August 11, 2014 by hishutup

[DoubleYou]

I'm going to guess it has to do with the 5 second delete and guess thing.

[hishutup]

Have you noticed a substantial increase in time when using Skyproc patchers? 

What do you mean 5 second delete and guess thing?

I'm guessing you're also going to call thing General MO weirdness or do you consider this a bug?