Absurb File Size for DnyDOLOD Generation

[Rage]

I was going through the process of generating precache grass, LODs and 3D trees for DynDOLOD, and normally it would have only taken about half an hour, an hour at most, but for some ungodly reason it too 16 hours and 160GB of space to generate... I was following the step modifications guide only.

Any ideas would be appreciated.

[sheson]

8 hours ago, Rage said:

I was going through the process of generating precache grass, LODs and 3D trees for DynDOLOD, and normally it would have only taken about half an hour, an hour at most, but for some ungodly reason it too 16 hours and 160GB of space to generate... I was following the step modifications guide only.

Any ideas would be appreciated.

Read https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs which log and debug log to upload when making posts.

https://dyndolod.info/FAQ#Long-running-time-or-output-several-GB-in-file-size

[DoubleYou]

That's a combo that I know some of the people on Lexy's server are using, so you might check there. I believe they said you can use the Performance HLT 3d LODs instead of the Quality ones.

[DoubleYou]

Ultra is required. Highly recommend using performance 3d lods where available. 

[z929669]

To potentially drastically reduce generation time, disable all of you driver-related software via Task Manager. On AMD, Adrenaline and other related AMD background processes can kill gen time. NVIDIA has its own processes, so you should be able to see them.

[z929669]

Some tree mods are very performance heavy (e.g., Veydogolt and Aspens Ablaze). If the LODs use the base un-optimized crowns of such tree mods, the BTO files can be huge and require many times more draw calls. Compare the file sizes of the LOD trees against the base trees. Most LOD NIFs are about 70% (ish) the size of the base NIFs. Truly optimized LODs will be one tenth or less the size of the base models.

EDIT: 70% is usually just fine when the base models are < 300k, so it's stuff > 500k that gets taxing without crown optimization.

[DoubleYou]

Oh my. Seasons too. You may have difficulty getting below 50gb. Set LOD32 to only use Billboard6 for trees might give you a chance. Might be better off sticking with using lod16 for the map if file size is a concern. 

[DoubleYou]

Consider swapping Level0 to Billboard4 then for trees should greatly reduce size. 

[z929669]

28 minutes ago, Rage said:

Low and behold, after 5 and 1/2 hours, it is finished but now it's taken ages to zip up the files... I hope this isn't stuck. 

EDIT

So. Lovely. It just prompted this error, which is false, then proceeded to delete ALL OF THE FILES IT JUST FINISHED DOWNLOADING.

Use the Performance LODs for AA in the FOMOD as mentioned. The result is not really any different for Level# tree LOD, the processing time and output size will be much reduced, and you will gain 10+ FPS in Riften. Those AA LOD do not support seasons though, so if you are running seasons, you may also be using seasonal AA LODs that I cannot speak to. Regardless, using the Performance LODs for the spring/summer/fall will definitely resolve most of it, since winter LOD crowns should not be as complex as the others. You will need to install the AA LODs for spring/summer/fall (and winter) as separate mods and possibly rename those models as seasons requires ... I don't run seasons and have no experience with them. Seasons will probably add to processing time though.

[DoubleYou]

59 minutes ago, z929669 said:

Use the Performance LODs for AA in the FOMOD as mentioned. The result is not really any different for Level# tree LOD, the processing time and output size will be much reduced, and you will gain 10+ FPS in Riften. Those AA LOD do not support seasons though, so if you are running seasons, you may also be using seasonal AA LODs that I cannot speak to. Regardless, using the Performance LODs for the spring/summer/fall will definitely resolve most of it, since winter LOD crowns should not be as complex as the others. You will need to install the AA LODs for spring/summer/fall (and winter) as separate mods and possibly rename those models as seasons requires ... I don't run seasons and have no experience with them. Seasons will probably add to processing time though.

I know that Fallven on Lexy's server got it working with seasons, but I believe they are modifying textures too. Models are compatible, so just require some additional modding to get right if it isn't working.

[z929669]

1 hour ago, Rage said:

So I finished that generation process, booted up the game with 120GBs of LOD Data, loaded outside a random place in Markarth and noticed my FPS was idling at 40 while sitting in one spot. Exited the game, disabled the plugins, did the same thing at the same spot and it went way back up to 90-100 FPS. This can't be the only solution to generating LODs... tanking 50-60 FPS just for LODs is absurd.

I did install the performance LODs, and I couldn't tell if it helped much but the LOD generation process did seem a bit faster. But my performance says otherwise.

For these, yeah I am concerned for support on SoS. I believe Fabled Forests do have support for them in Seasons as it mentioned SoS support on the mod page using Turn of the Seasons. 

The AA Add-On lods mod has all of the performance LODs for all but the winter season, but I think Seasons requires that they be renamed to match expected strings for each season. If your performance is dropping that much, then you have other UN-optimized LODs in your mod list that could be attributed to one or more specific seasons. It's not a DynDOLOD problem or a problem that the AA Add-On should address. It's simply a matter of using mods that offer proper LOD models for use in object LOD. Even one tree that is not optimized can kill your performance if it is common.

Also, Aspens Ablaze has at least two trees that will kill performance in loaded cells irrespective of LOD, especially if you are using the larger trees AA option. The LODs are optimized though (except for winter LODs, which I cannot speak to).

[z929669]

17 minutes ago, Rage said:

I will have to look more into my LODs then, but I thought I had got all of the performance-based LOD options I could. Not sure what else could be causing it. I will share my full load order after this generation process.

On another note, I generated LODs using specifically the High preset and running just Tamriel with Seasons, and there was no negligible performance difference. This was with the same setup, but just the Tamriel worldspace.

Just install the AA Add-On LOD up to three times into up to three separate mods for spring/summer/fall. I would use green aspens for both spring and summer, and either autumn or autumnal variety for fall. The winder LODs may be provided elsewhere. As I mentioned, the LODs may need to be renamed (but not the CRC32 strings), but I can't provide those details, since I don't run seasons.

EDIT: to verify performance and LOD, I would scrap running Seasons for now to convince yourself that the LODs and DynDOLOD produce a very playable result. It's Seasons and LODGen for Seasons that is most likely making things difficult. post with DynDOLOD logs on the DynDOLOD Alpha topic for assistance with Seasons.

[DoubleYou]

Typically trees. What tree mods? 

[Rage]

4 hours ago, DoubleYou said:

Typically trees. What tree mods? 

Fabled Forests, they're based off of HLT but use a mix of two other three mods.

[Rage]

4 hours ago, DoubleYou said:

That's a combo that I know some of the people on Lexy's server are using, so you might check there. I believe they said you can use the Performance HLT 3d LODs instead of the Quality ones.

Alright, I will give that a shot. I'm regenerating without the Ultra option enabled, because I think that was the reason why it generated so long and caused a huge intake on file size. Previously, before I didn't use that and it was at 30GB for DynDOLODs output.

I have to ask, is Ultra Tres required for 3D Tree LODs? Or can I just set LOD 4 as Level0 and it will generate 3D trees?

[Rage]

5 hours ago, DoubleYou said:

Ultra is required. Highly recommend using performance 3d lods where available. 

Damn, that's going to be super annoying, I won't be able to generate them for a while then. I'll rerun with Ultra later.

On other terms, this generation process is still going on and it's got up to 40GBs. If I use the performance LODs, will this cut down on the overall generation size? I am trying to keep it all under 50GBs.

Or another idea, would utilizing my GPU for generating the LODs be a better process? This is all running on a Ryzen 9 5900X and an RTX 3070.

Edited October 21, 2023 by Rage