xLODGen - Terrain LOD beta 128 for FNV, FO3, FO4, FO4VR, TES5, SSE, TES5VR, ENDERAL, ENDERALSE

[sheson]

10 hours ago, Marcurios said:

i get a consistent CTD in game after i rendered terrain LOD with the extremely experimental version of xLODGen 4.14 for FO4.

The only logfile i have on this crash is from Buffout 4;
E:\Repos\f4se\Buffout4\src\Warnings\CreateTexture2DWarning.cpp(60): [critical] A call to ID3D11Device::CreateTexture2D failed with error code 0x80070057. This will crash the game.

If i remove the textures created for the terrain by putting them temporarily in a separate folder the game doesn't know the game starts fine with the rendered LODChunks from the terrain, ofcourse without the new textures.

Also, the object LOD doesn't use the texture files specified in the lod material and lod meshes that are referred to by my esp file.
the regular version of FO4Editx64 renamed to FO4LODGenx64 does use those specified textures correctly.
But it can't do terrain ofcourse.

Also, the ability to choose plugins that you want to enable/disable in the regular xEdit is very convenient too.
having to do that all in a mod manager before you open up xLODGen is a drag, especially since i use a very rudimentary mod manager, cause all the fancy ones are more a nuisance cause they have badly written mod install routines, so i always install mods by hand, that way i'm sure files get placed in the right directories.
The downside is that the simple mod manager doesn't have mod profiles you can enable/disable.
So if you could implement that (enable/disable plugins on load), that would be cool.

But right now, the extremely experimental version sadly doesn't produce a workable result for me.
It doesn't read the loose texturefiles and materials for objects and treelod (3d treelod) it looks like, it looks like it gets the stuff needed only from the ba2 files.
the regular version uses the loose files first.
So i'll still have to use CK for terrain i guess.

Provide logs, instructions and examples how to reproduce a problem.

To select plugins,  start the tool in Edit mode, e.g. rename to xEdit.exe or use -edit command line. Then start LOD generation via right click, Other, Generate LOD.

Upload a simple example plugin/mod that can be use to reproduce and troubleshoot the problem with the LOD material/LOD Meshes not being used.
What exact version is the regular version?

xEdit/xLODGen loads from BA2 and loose files just like the game. Loose files always win. If a loose file isn't used, it is typically because it doesn't exist at the exact same path. Maybe the OS, antivir or something else interferes with the scanning of files and folders. AFAIK the code in xEdit that loads BA2 and loose files hasn't changed between versions.

If the generated texture cause CTD, then its probably because of an unsupported compression format. Use the default settings for them.
xLODGen generates the textures in 888 format and then uses TexConv to convert them. Maybe the OS, anti or something else interferes in the process of saving/converting files.
 Isolate a texture with binary search and upload it.

[katubug]

I'm having an issue running xLODGen through MO2. I have the 64-bit exe chosen. My arguments line is -sse -o:”D:\Games\SkyrimModding\Tools\xLODGen\Output\". I disable real-time protection during the whole LOD process. I've also ensured that I have the right windows framework, and have tried restarting my pc.

However, when I launch through MO2, it does all the background loading tasks, says "Background Loader: Finished" and then stops. There's a little popup window that tells me to "Wait a few minutes," and its progress bar either finishes very quickly or stops at about 30%. It disappears, and then nothing else happens. The version of lodgen that I got from Nexus is working fine, but the latest beta as of yesterday never produces the settings window that lets me continue with the lodgen.

I'm sure I'm missing something foolish, but I'd appreciate help! Thanks!

Edit: Okay, I seem to have fixed the issue, as per the rules of the universe. I removed the output folder from my arguments line and simply told it to write directly to an empty mod that I created, and now the Options screen appears without issue. Ah, technology.

Edited January 8, 2022 by katubug

[sheson]

40 minutes ago, katubug said:

I'm having an issue running xLODGen through MO2. I have the 64-bit exe chosen. My arguments line is -sse -o:”D:\Games\SkyrimModding\Tools\xLODGen\Output\". I disable real-time protection during the whole LOD process. I've also ensured that I have the right windows framework, and have tried restarting my pc.

However, when I launch through MO2, it does all the background loading tasks, says "Background Loader: Finished" and then stops. There's a little popup window that tells me to "Wait a few minutes," and its progress bar either finishes very quickly or stops at about 30%. It disappears, and then nothing else happens. The version of lodgen that I got from Nexus is working fine, but the latest beta as of yesterday never produces the settings window that lets me continue with the lodgen.

I'm sure I'm missing something foolish, but I'd appreciate help! Thanks!

Edit: Okay, I seem to have fixed the issue, as per the rules of the universe. I removed the output folder from my arguments line and simply told it to write directly to an empty mod that I created, and now the Options screen appears without issue. Ah, technology.

The first double quote of the command line you posted is not actually a straight double quote but a curly one.

” should be "

That can happen if copy pasting stuff from websites that have braindead WYSIWYG Editors automatically "pretty" up stuff.

Always set a dedicated output folder outside of Steam, game and mod manager folders.

[Marcurios]

Thanks for taking the time to reply..

The xLodgen log is too big for the allowed filesize, its almost 5 mb.
It is full of texconv errors like these;
01:00]       0: VID:10DE, PID:2484 - NVIDIA GeForce RTX 3070
[01:00]       1: VID:1414, PID:008C - Microsoft Basic Render Driver
[01:00] Error: Executing Texconv failure 00000001: "E:\Games\Steam\steamapps\common\Fallout 4\Edit Scripts\Texconvx64.exe" -nologo -y -aw 256 -sepalpha  -f DXT1 -bc d -o "e:\games\steam\steamapps\common\fallout 4\data\textures\terrain\commonwealth" -flist "C:\Users\Marcus\AppData\Local\Temp\FO4Edit\Texconv_Commonwealth_112_0.txt"
[01:00] e:\games\steam\steamapps\common\fallout 4\data\textures\terrain\commonwealth\commonwealth.4.92.-68_msn.dds
[01:01] Invalid value specified with -f (DXT1)
[01:01] Microsoft (R) DirectX Texture Converter (DirectXTex version)
[01:01] Copyright (C) Microsoft Corp. All rights reserved.
[01:01] Usage: texconv <options> <files>
Maybe 1/3 of all texture converts have this error, the rest renders ok it looks like.
link to the file in my onedrive; https://1drv.ms/t/s!AnB6a-bT8xxukAP_xvDO2fj4ThmM?e=tJvsCG

With the suggested procedure to load plugins with the xLODGen version it's not possible to generate terrain lod, only object lod.

There definitely seems to be a difference in how the program loads texturefiles depending on how you use it.
If you use it as FO4Editx64 it renders object lod with loose textures fine, and you can select plugins, but it can't render terrain lod.

if used as FO4LODGenx64, you have the option to render terrain lod, and you can't select plugins,
but if used like this without the option to render terrain lod it doesn't load the loose files for my trees (3d object lod).
It even loads completely different incorrect files for the objectlod atlas without changing anything when it comes to referenced files and where they are.
So with terrain lod option enabled it renders unusable terrain dds files that make the game crash, and if you delete the terrain dds files and terrain chunks so you only have the objectlodatlas textures and the objectlod meshes left, they (textureatlas, not meshes) are completely wrong as well.
Cause it does something weird with the objectlod atlas file, it takes the wrong messed up textures from i don't know where, but not from the specified referenced textures in the
lod meshes and their material. (see screenshots)

I have a exception in my antivirus for my gamefolders, so they don't get scanned by the realtime scanner.
And both programs are in my Fallout dir, so none of the two are being handled by antivirus program.
I also checked the programs set up in windows , there is no security settings on them, they have full privileges.
The only thing i haven't done is set them to run as admin, should i perhaps do that ?
Otherwise i can't imagine Windows messing with them programs, since all other programs i install work perfectly fine.
And if i say the regular xEdit, i mean the version this xLODGen is based on, so 4.04 (i said 4.14 in my earlier comment, but i ment 4.04 ofcourse)

Also, I did use the default setting for the textures, right out of the box, tried multiple times with the default settings.

I don't think i made any stupid mistakes here (but you never know, not saying i'm 100% certain about that), cause i rendered LOD for FO3 and FNV hundreds of times when i made WFO, so i do know what to do when it comes to referring to textures from nif files, the only difference here is that there is a material added in FO4 that has a reference to the used textures as well, and it is referenced from within the meshes itself. I used the material with the name unchanged so i didn't have to change the path in the mesh, and also used the same name for the texture, only used them as loose files to override whats in the ba2 file of the Treesredux.esp mod that i wanted to change the leafcolor from.
I also unpacked the meshes and lod meshes to see what the mod author did, but it doesn't matter if i keep tehm in the ba2 file only or also place them as loose files, i already tried that.
I actually tried a bunch of stuff, so i already rendered terrain lod and/or object lod for atleast 30 times now, with the same result no matter what i do.
I'm going to go over all my files once again to see if i did make some mistake somewhere just to be certain.
Otherwise i'll probably forego terrain LOD alltogether.
xEdit mode renders fine object lod, everything correct, xLodgen mode does not looks like, and also refuses to render usable terrain textures, meshes are ok it seems.

maybe there is something wrong with the format of my terrain textures ?
I think i made them al BC5 and BC7 if i'm not mistaken, i'd have to check that..

 

Edited January 8, 2022 by Marcurios
clarified some things and added some

[sheson]

49 minutes ago, Marcurios said:

The xLodgen log is too big for the allowed filesize, its almost 5 mb.
It is full of texconv errors like these;
01:00]       0: VID:10DE, PID:2484 - NVIDIA GeForce RTX 3070
[01:00]       1: VID:1414, PID:008C - Microsoft Basic Render Driver
[01:00] Error: Executing Texconv failure 00000001: "E:\Games\Steam\steamapps\common\Fallout 4\Edit Scripts\Texconvx64.exe" -nologo -y -aw 256 -sepalpha  -f DXT1 -bc d -o "e:\games\steam\steamapps\common\fallout 4\data\textures\terrain\commonwealth" -flist "C:\Users\Marcus\AppData\Local\Temp\FO4Edit\Texconv_Commonwealth_112_0.txt"
[01:00] e:\games\steam\steamapps\common\fallout 4\data\textures\terrain\commonwealth\commonwealth.4.92.-68_msn.dds
[01:01] Invalid value specified with -f (DXT1)
[01:01] Microsoft (R) DirectX Texture Converter (DirectXTex version)
[01:01] Copyright (C) Microsoft Corp. All rights reserved.
[01:01] Usage: texconv <options> <files>
Maybe 1/3 of all texture converts have this error, the rest renders ok it looks like.

With the suggested procedure to load plugins with the xLODGen version it's not possible to generate terrain lod, only object lod.

There is definitely a difference in how the program loads texturefiles depending on how you use it.
If you use it as FO4Editx64 it renders object lod with loose textures fine, and you can select plugins, but it can't render terrain lod.

if used as FO4LODGenx64, you have the option to render terrain lod, and you can't select plugins,
but if used like this without the option to render terrain lod it doesn't load the loose files for my trees (3d object lod).
It even loads completely different incorrect files for the objectlod atlas without changing anything when it comes to referenced files and where they are.
So with terrain lod option enabled it renders unusable terrain dds files that make the game crash, and if you delete the terrain dds files and terrain chunks so you only have the objectlodatlas textures and the objectlod meshes left, they are completely wrong as well.
Cause it does something weird with the objectlod atlas file, it takes the wrong messed up textures from i don't know where, but not from the specified referenced textures in the
lod meshes and their material. (see screenshots)

I have a exception in my antivirus for my gamefolders, so they don't get scanned by the realtime scanner.
And both programs are in my Fallout dir, so none of the two is being handled by antivirus program.
I also checked the programs set up in windows , there is no security settings on them, they have full privileges.
The only thing i haven't done is set them to run as admin, should i perhaps do that ?
Otherwise i can't imagine Windows messing with them programs, since all other programs i install work perfectly fine.
And if i say the regular xEdit, i mean the version this xLODGen is based on, so 4.04 (i said 4.14 in my earlier comment, but i ment 4.04 ofcourse)

Also, I did use the default setting for the textures, right out of the box, tried multiple times with the default settings.

See my signature about using a paste or a file service for uploading log files.
If there are errors in the log, it should be no surprise things are not working as expected...

As explained on the first post, unpack xLODGen into a dedicated, empty new folder outside of Steam, Game and special Windows folders like Program Files x86. 

Use the latest xLODGen terrain LOD beta 85 from the first post. It is based on 4.1.4 Extremely Experimental.

It seems the Texconv.exe that ships with the xLODGen archive got replaced. Do not replace single files or tools from the archive. When unpacking always unpack into a new empty folder. Use 7zip to unpack. Make sure there are no files from other versions.

Since Texconv is also used when reading textures, it possible that the troubles of not using/loading certain textures is automatically solved  as well.

Always set a dedicated output folder with -o that is outside of Steam, game and mod manager folders as explained on the first post.

Starting xLODGen terrain LOD beta in edit mode can generate terrain LOD without any problem. Enable simple records in the options first as indicated by the message in case they are disabled. After a restart, right click on the plugin that adds the worldspace you want to generate LOD for, select Other, then select Generate LOD.

The log prints a list of loaded archives and which data folder is used. Verify that these things are what is expected.

If in Edit mode, use the Asset Browser (CTRL+F3) to enter filenames into the filer field to see which containers are found. The first container listed is the one winning. It is "data" for loose files.

If the Asset Browser behaves the same for the different versions, provide an example plugin and/or the steps how to reproduce the claim xEdit/xLODGen loads archives and/or loose files differently now compared to older versions.

[Marcurios]

OK, i feel completely stupid now, cause i did RTFM, but when it said install in a different directory i thought to myself, that shouldnt make such a big difference.
But when it comes to rendering correct terrain, it obviously does...sigh..banging head against wall..

Thanks for your help Sheson, sorry to take your time needlesly..
 

[Marcurios]

Everything renders OK now, but i have a question, did you ever get shimmering lights on object lod after you rendered it ?

Cause the trees look OK, but at certain angles i get all dancing white lights on the trees, flickering like crazy, i never seen this before in any modded game.
i can't even begin to guess at what is causeing it too. I had working objectlod before without this phenomenon.

You have any idea what it is and how to get it fixed ?

It's not z-fighting, i know what that looks like and how to alleviate that.
I't looks like bloom flashes, but it happens on dark objects (tree canopies) only, so that's not what bloom does, still, it looks like bloom flashing on and off rapidly.

Edited January 8, 2022 by Marcurios

[sheson]

11 hours ago, Marcurios said:

Everything renders OK now, but i have a question, did you ever get shimmering lights on object lod after you rendered it ?

Cause the trees look OK, but at certain angles i get all dancing white lights on the trees, flickering like crazy, i never seen this before in any modded game.
i can't even begin to guess at what is causeing it too. I had working objectlod before without this phenomenon.

You have any idea what it is and how to get it fixed ?

It's not z-fighting, i know what that looks like and how to alleviate that.
I't looks like bloom flashes, but it happens on dark objects (tree canopies) only, so that's not what bloom does, still, it looks like bloom flashing on and off rapidly.

Does this happen with vanilla trees and only vanilla LOD resources and no mods, especially no ENB and/or lighting mods?

Sounds a bit like missing/wrong  _s.dds texture for the LOD asset.

[aethel]

Hi,

I'm getting crashes while generating terrain LOD using the latest version that is currently published: https://i.imgur.com/TPIijnu.png

It never crashes in the same place, or even in the same worldspace every time. If I keep running it over and over and telling it to not do the worldspaces it has already done it (eventually...) completes.

Digging a little with a debugger I found that the crash is due to the following Delphi exception being raised:

EListError

List index out of bounds (65)

Without knowing anything about how the program works I guess it's some threading issue.

Here's the log if it helps: https://pastebin.com/j0LXEvbq

[Marcurios]

2 hours ago, sheson said:

Does this happen with vanilla trees and only vanilla LOD resources and no mods, especially no ENB and/or lighting mods?

Sounds a bit like missing/wrong  _s.dds texture for the LOD asset.

no, it doesn't happen with vanilla trees, even with ENB it doesn't happen.
I determined that ENB makes it less when enabled, if i disable ENB it's like a lightshow.
Since i already suspected it might have something to do with specular or gloss i checked the specular yesterday.

So yes,I had a missing specular texture indeed for the trees, but rezised the original and renamed it to the lod variant and rendered again but that didn't fix it yet.
Just woke up so now i'm going to try to darken my spec texture, cause right now it's as bright as vanilla textures, and i often find vanilla somewhat to bright to look just right.
I'll also darken the gloss layer a bit to narrow the gloss.

[sheson]

57 minutes ago, aethel said:

Hi,

I'm getting crashes while generating terrain LOD using the latest version that is currently published: https://i.imgur.com/TPIijnu.png

It never crashes in the same place, or even in the same worldspace every time. If I keep running it over and over and telling it to not do the worldspaces it has already done it (eventually...) completes.

Digging a little with a debugger I found that the crash is due to the following Delphi exception being raised:

EListError

List index out of bounds (65)

Without knowing anything about how the program works I guess it's some threading issue.

Here's the log if it helps: https://pastebin.com/j0LXEvbq

The log you uploaded is from a successful the terrain LOD meshes generation from LODGen. The screenshot seems to indicate the problem happens while generating terrain LOD textures.

Upload the SSELODGen_log.txt and bugreport.txt.

Use the x64 version in case you are not using it.

11 minutes ago, Marcurios said:

no, it doesn't happen with vanilla trees, even with ENB it doesn't happen.
I determined that ENB makes it less when enabled, if i disable ENB it's like a lightshow.
Since i already suspected it might have something to do with specular or gloss i checked the specular yesterday.

So yes,I had a missing specular texture indeed for the trees, but rezised the original and renamed it to the lod variant and rendered again but that didn't fix it yet.
Just woke up so now i'm going to try to darken my spec texture, cause right now it's as bright as vanilla textures, and i often find vanilla somewhat to bright to look just right.
I'll also darken the gloss layer a bit to narrow the gloss.

Double check the 2 channels of the source _s.dds are somehow modified while being put on the texture atlas like ..\Textures\Terrain\Commonwealth\Objects\Commonwealth.Objects_s.DDS

Also check the mipmaps.

[aethel]

20 minutes ago, sheson said:

The log you uploaded is from a successful the terrain LOD meshes generation from LODGen. The screenshot seems to indicate the problem happens while generating terrain LOD textures.

Upload the SSELODGen_log.txt and bugreport.txt.

Use the x64 version in case you are not using it.

 

The log I uploaded is the only file that is created when it crashes. SSELODGen_log.txt only appears after I close the program whenever it finishes without crashing. bugreport.txt does not ever appear and the madExcept window doesn't show up either.

I'm using the x64 version.

 

Here's the SSELODGen_log.txt from a successful run if you need it, but note that it did not crash this time: https://qu.ax/EDwV.txt

Edited January 9, 2022 by aethel

[sheson]

3 hours ago, aethel said:

The log I uploaded is the only file that is created when it crashes. SSELODGen_log.txt only appears after I close the program whenever it finishes without crashing. bugreport.txt does not ever appear and the madExcept window doesn't show up either.

I'm using the x64 version.

 

Here's the SSELODGen_log.txt from a successful run if you need it, but note that it did not crash this time: https://qu.ax/EDwV.txt

See if anything is different if you use this version

https://mega.nz/file/YN50QDBA#yl5fCJWtr4qIaQwAuVbfXD0e7VwxC2GsvQqNNEW0FV0

[Marcurios]

Ah, now i see what's going on with the spec and gloss, the file had a _ (underscore) too many in it so it didn't get used.
But when xLODGen says it uses a flat replacement, does it than use a white flat replacement for the red and green channel ?
Cause red and green are white, and blue (not used) is black, isn't it better to provide a 2x2pixel dark gray texture bundled with xLODGen and use that for when it misses a spec texture ?

I also noticed that a quite a few vanilla textures have this problem going on, cause i have 6 vanilla blocks in the atlas with the same problem, and 1 actually uses a 128 color grey texture from what i can tell, the rest all have normal spec textures.

i'm going to unpack all the texture archives to see which lod spec textures are missing or have a white flat one, time to fix some stuff.
i have been playing the game up until now but i guess it's time to get modding again cause this kinda annoys me a bit.
There is also a lot of doubled stuff on the atlas so i guess a bunch of textures that are the same have a different name and are referred to by separate meshes,
so i need to fix those meshes too to save some real estate on the texture atlas so i can enhance some other lod with the newly reclaimed space..
EDIT:just noticed the diffuse has different colors, so i guess it's needed the doubled stuff.

Thanks again for your help !

Edited January 9, 2022 by Marcurios

[Marcurios]

It's fixed now, it was indeed the spec from the trees that was missing from the atlas, it's on it now and the shimmers are gone.
Now to find those pesky vanilla textures and fix them.

Thanks Sheson, you saved me a lot of time figuring it out myself.