Linux/Proton: A call to an OS function failed

[itsdnoftheworld]

Hello, I have been having an annoying issue running Dyndolod 3 on my Proton setup related to texconv. While running Dyndolod after successfully generating xlodgen and texgen, I am unable to proceed with generating Dyndolod, as I eventually get a message saying "Could not execute "Z:\home\dillon\Games\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -gpu 0 -y -m 1 -aw 256 -f R8G8B8A8_UNORM -o "C:\users\steamuser\Temp\DynDOLOD_SSE" "C:\users\steamuser\Temp\DynDOLOD_SSE\379794EC272A4EF98BAF93F0B0726085.dds". Then I get a pop-up about Call to an OS Function Failed. No .esp is mentioned.

Since I am on Linux, I don't have UAC or AV on my computer, and after changing permissions of some of the folders, specifically my game folder, mods folder, and the Dyndolod folder I have been able to run it once or twice. However, after changing a texture mod, I have been completely unable to get around this Texconv issue. I believe that Texconv should be running, since I can see it in the task manager and I can see some .dds files being converted. I also was able to run texgen and xlodgen with no issue. In fact I have run Dyndolod successfully earlier today! I have tried placing Dyndolod folder in both my Z: drive (the native Linux system) and my C: drive (the emulated Windows drive), and both run into the same error. I also reinstalled and repaired dotnet and C++, and I was able to run texconv (the official one from Github) with the below output. I also was able to convert images into .dds from the command line. However I could not replicate this with the one downloaded alongside Dyndolod. Is this expected behavior?

I am currently unable to test on Windows since my install is all messed up, but I am just looking to see if there is anything I have missed in troubleshooting. Any advice would be greatly appreciated.
 

Bug Report: https://pastebin.com/B3Te9XG4

SSE Log: https://paste.ee/p/Hhg78

Texconv Test: https://paste.ee/p/6vtVc

[sheson]

  On 5/16/2024 at 1:52 AM, dkfljaslkdfh said:

Hi!

I'm having the same issue, now with Dyndolod. This is strange, because I was having the same issue with xLODGen, but now it's working fine.

It's exactly as described in the OP. The only thing that changes is the mod being processed at the time.

Can you take a look at the logs, please?

bugreport: https://paste.ee/p/z6QJ6
log: https://paste.ee/p/zaiPI

Thank you!

EDIT: I forgot to mention that the tools are on a drive outside of the MO2 and the system folders. Also, I'm using the x64 version. I tried with the x86 and it worked with a clean install of the game, but with the full mods it stopped without giving me an error, it just hanged.

Expand  

Read https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs which DynDOLOD debug log to also upload when making posts.

If you have the exact same issue, then the exact same explanations and tips apply.
Use backslashes only in paths.
Map drive letters to the mounted drives in the winetricks configuration to reduce path lengths.
Do not use -o command line with TexGen/DynDOLOD, let them use their default output folders or set them via the interface option.

The OS reports that it has a problem. This most likely has to do with file access problems, maybe because of file system drivers, their settings or MO2 virtual folders. Hence the repeated tip to test without MO2. Using MO2 works fine here, though.

[Maotok]

I believe I may have solved the "Call to an OS Function Failed" error. I am also on Linux using the latest version of PopOs.

What seems to be going on is some sort of permission error between MO2, the wine/proton PREFIX, and Linux's permissions. Either the permission is revoked after too long of a time or something in the coding between all the levels and programs doesn't have the correct permission. I can not change the code and I'm not digging through it all to figure that part out (I'll leave that to the code GODS).

HOWEVER, I was able to finally get it working after messing with a few other programs like synthesis. What you need to do is put everything within the wine/proton prefix that steam has created for skyrim. For me that path would look like this:

" /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/ "

I install everything skyrim related in that path including any windows programs needed to run whatever tool skyrim mods need. In this case that was MO2, vcredist, and dotnet. Make sure you install all the required windows software needed for Dyndolod (or any other tool) in that steam prefix. You will do this using this little segment of code ran in the terminal adjusted to your prefix location (my path looks like this):

" WINEPREFIX=/home/mrfuji/Games/steamapps/compatdata/489830/pfx winetricks --gui "

you will place your own install location after WINEPREFIX= *your prefix path to compatdata/489830/pfx* winetricks --gui

This will bring up the wintricks gui which you will use to install any windows drivers or software needed for dyndolod (or any other tool, this works for more than just Dyndolod) directly into the steam prefix folder for your skyrim installation. This is a critical step. We are skipping Linux drivers and treating our skyrim Prefix as its own little windows world. We are doing this to avoid any crosstalk of permissions between Linux and Wine/Proton.

You will also need to do this for MO2. Download the latest version and install it into the prefix.

Using my game paths this would look something like this:

1. *open terminal and input* WINEPREFIX=/home/mrfuji/Games/steamapps/compatdata/489830/pfx winetricks --gui   <--- change to your path

2. *Wine should run and open the gui*

3. Make sure "select the default wineprefix" is selected and hit OK

4. another page will open (if needed make sure windows 10 or 11 is selected in the winecfg, you'll know if a windows installer throws a version error at you).

5. on that same page there is a line called "Run an arbitrary executable (.exe/.msi/.msu)" Select that, hit OK

6. Browse to find the .exe for the MO2 installer and select it and run.

7. This will install MO2 into your steam prefix for skyrim.

You will use a very similar setup for every single dependent software installer needed to run some of the skyrim modding tools. Just repeat the steps above replacing "/home/mrfuji/Games" with your path to your steam apps folder. A tip: try to find the offline installers windows has to help for a smoother install of utilities, thats a howto for another day.

After you are done installing all the needed windows software you will install Dyndolod into its own folder inside wherever you installed your skyrims steamapps   

" /steamapps/compatdata/489830/pfx/drive_c/ " folder.

For me that looked like this:

" /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/DynDOLOD "

Now just launch the MO2 thats installed inside your prefix location and follow the DynDOLOD guide on how to use it with MO2.

IDK if this has anything to do with it however, I launch MO2 using a line of code in the terminal in order to skip using the steam gui launcher which keeps trying to update my Skyrim. (I know the workarounds I just don't care to change it and no I didn't pirate it. At least the pirates don't get updates....Thanks Todd.) However it may help anyone else experiencing the call to OS function error. Just modify this little line of code, turn it into a bash script or just copy paste it each time into the terminal doesn't matter to me, but modify the paths to point to your stuff. This points to my stuff following the paths listed above:

STEAM_COMPAT_CLIENT_INSTALL_PATH=/usr/games/steam STEAM_COMPAT_DATA_PATH=/home/mrfuji/Games/steamapps/compatdata/489830/ /home/mrfuji/.steam/root/compatibilitytools.d/GE-Proton9-7/proton  run /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/Modding/MO2/ModOrganizer.exe

That points to my paths and if you copied all my paths above and mimiced them into your own drive this would run on your own side as is copy pasted int the terminal. Please dont do that,  learn your paths so you can better debug your stuff.

a quick breakdown:

STEAM_COMPAT_CLIENT_INSTALL_PATH=(this points to your steam install) STEAM_COMPAT_DATA_PATH=(This points to your compat data folder also know as your Prefix for Wine/Proton) (the next part points to where your proton is installed on your drive typically in your .steam/root/compatibilitytools.d/) (the last part always has "run (space)" and your path to your ModOrganizer.exe which should now be in your compatdata/489830/pfx/drive_c/ folder )

My compatibilitytools.d path points to my GE proton location and not directly to a steam released proton version, find your proton install path. If you put everything into your compat folder you should be able to now point MO2 to Dyndolod as per the DynDOLOD install guide. When selecting a file to run inside MO2 the file browser should now have a "C:" drive that is listed under MyComputer. You should be able to find your Dyndolod and any other .exe tool installed into your Prefix there. If you cant find the tool in the C: drive then you installed something outside the Prefix. You should not have to look outside the C: drive to find your .exe tools. Recheck your install paths and steps and try again. Do not point MO2 outside the Prefix for anything including your mod folder. (Downloads folder is probably fine.)

If all that checks out then DynDolod should run without throwing the Call to OS error. I wish you all well, if I need to clarify something I will try my best but I'm not a programmer and I have just kinda stumbled into half of it. So I will do what I can. I have run it successfully a couple times now for over an hour each time ( too many mods ) and it has completed its run each time. Before it would stop around the 7 to 10 minute mark and throw the call to OS error.

 

 

 

 

[Gazotti]

I've had this before with texconv. Running DynDOLOD in a terminal had it print an error along the lines of "you need .NET 4.X", so I installed it with winetricks and it worked properly.

.NET Core Desktop 7 was installed, but it seems we also need Desktop 6 (for LodGenX64Win) and 4.8 for texconv and LodGenX64.

[sheson]

It seems likely there are several possible causes for this error message as can be seen by the different solutions and tips above.

As explained before, Texconv (and other things) require the latest Microsoft Visual C++ Redistributable to be installed properly:
Start the protonfix GUI, select Skyrim Special Edition: 489830 Steam App, click through to winetricks, select the default wineprefix, select Install a Windows DLL or component, check vcrun2022 and the click OK.

Or open a console window and execute:

WINEPREFIX="/the/path/to/SteamLibrary/SteamApps/compatdata/489830/pfx" winetricks --force vcrun2022

If nothing else helps, try this test version of https://mega.nz/file/4IQlFA7B#nzdR7V95MQxBYmKDZD9agfxHOGeh1w8ZdxoO5oISaM8 and report back successes or failures.

[kontotnok]

  On 12/14/2024 at 10:18 AM, sheson said:

It seems likely there are several possible causes for this error message as can be seen by the different solutions and tips above.

As explained before, Texconv (and other things) require the latest Microsoft Visual C++ Redistributable to be installed properly:
Start the protonfix GUI, select Skyrim Special Edition: 489830 Steam App, click through to winetricks, select the default wineprefix, select Install a Windows DLL or component, check vcrun2022 and the click OK.

Or open a console window and execute:

WINEPREFIX="/the/path/to/SteamLibrary/SteamApps/compatdata/489830/pfx" winetricks --force vcrun2022

If nothing else helps, try this test version of https://mega.nz/file/4IQlFA7B#nzdR7V95MQxBYmKDZD9agfxHOGeh1w8ZdxoO5oISaM8 and report back successes or failures.

Expand  

logs: https://paste.ee/p/un2Lx

I installed xLODGen and ran the x64 version through MO2, I ran it 3 times before finding this post, every error it made some progress and then crashed, I deleted the output folder and then I used the Texconvx64 that you said to test. The process failed at the same point trying to run Texconvx64

[sheson]

  On 12/15/2024 at 5:45 AM, kontotnok said:

logs: https://paste.ee/p/un2Lx

I installed xLODGen and ran the x64 version through MO2, I ran it 3 times before finding this post, every error it made some progress and then crashed, I deleted the output folder and then I used the Texconvx64 that you said to test. The process failed at the same point trying to run Texconvx64

Expand  

Also upload the FNVLODGen_log.txt when making posts.

Also check all the other posts about other possible reasons and solutions being discussed.

[kontotnok]

  On 12/15/2024 at 7:18 AM, sheson said:

Also upload the FNVLODGen_log.txt when making posts.

Also check all the other posts about other possible reasons and solutions being discussed.

Expand  

But I did upload FNVLODGen_log, everything is on the same link

[sheson]

  On 12/15/2024 at 7:43 AM, kontotnok said:

But I did upload FNVLODGen_log, everything is on the same link

Expand  

Ah, sorry about that. The FNVLODGen_log misses the first line including "starting session"

If none of the other solutions above help, try to upload a copy of the txt file mentioned by the message.
[02:14] <Error: Could not execute "C:\Modding\xLODGen\Edit Scripts\Texconvx64.exe" -nologo -y -aw 256 -sepalpha  -f DXT1 -bc d -o "c:\output\textures\landscape\lod\*" -flist "C:\users\migu\Temp\FNVEdit\Texconv_*.txt">
You should check for the file right away before closing anything, though it may still be gone already. If not upload it.

There is also error logged by LODGen while it generates terrain LOD meshes for WastelandNV. Is that repeatable?

[kontotnok]

  On 12/15/2024 at 8:22 AM, sheson said:

Ah, sorry about that. The FNVLODGen_log misses the first line including "starting session"

If none of the other solutions above help, try to upload a copy of the txt file mentioned by the message.
[02:14] <Error: Could not execute "C:\Modding\xLODGen\Edit Scripts\Texconvx64.exe" -nologo -y -aw 256 -sepalpha  -f DXT1 -bc d -o "c:\output\textures\landscape\lod\*" -flist "C:\users\migu\Temp\FNVEdit\Texconv_*.txt">
You should check for the file right away before closing anything, though it may still be gone already. If not upload it.

There is also error logged by LODGen while it generates terrain LOD meshes for WastelandNV. Is that repeatable?

Expand  

The Texconv file is already gone

I ran it again, trying the fixed mentioned about installing dotnet desktop but nothing (I already have vcrun installed)

logs (everything's here): https://paste.ee/p/IlUe3

Edit (Not really because I can't edit it): It doesn't work outside of MO2 either

[sheson]

  On 12/15/2024 at 8:41 AM, kontotnok said:

The Texconv file is already gone

I ran it again, trying the fixed mentioned about installing dotnet desktop but nothing (I already have vcrun installed)

logs (everything's here): https://paste.ee/p/IlUe3

Edit (Not really because I can't edit it): It doesn't work outside of MO2 either

Expand  

Upload new logs with this somewhat more chatty version of texconv https://mega.nz/file/0MAGXLRI#uer6RNAlqLHAdV-bC8xAMfyhZo5RbnGqe0r-0dLSl2A

[kontotnok]

  On 12/15/2024 at 10:15 AM, sheson said:

Upload new logs with this somewhat more chatty version of texconv https://mega.nz/file/0MAGXLRI#uer6RNAlqLHAdV-bC8xAMfyhZo5RbnGqe0r-0dLSl2A

Expand  

This time it ran for longer but didn't give me an fnvlodgen_log.txt file

Logs: https://paste.ee/p/5rca5

[sheson]

  On 12/15/2024 at 11:21 PM, kontotnok said:

This time it ran for longer but didn't give me an fnvlodgen_log.txt file

Logs: https://paste.ee/p/5rca5

Expand  

Get this test version of xLODGenx64.exe https://mega.nz/file/8Y4iULAT#XrL2rY4lHRkXd1x8u_fqXCQwhaBNZMHqd_n49KtY8dI and upload new logs.

[kontotnok]

  On 12/16/2024 at 7:12 AM, sheson said:

Get this test version of xLODGenx64.exe https://mega.nz/file/8Y4iULAT#XrL2rY4lHRkXd1x8u_fqXCQwhaBNZMHqd_n49KtY8dI and upload new logs.

Expand  

It still crashes unfortunately

Logs: https://paste.ee/p/8BTGs

BTW it turns out the previous run did generate an FNVLODGen_log.txt file, but I hadn't closed the app yet so that's probably why it didn't appear in the folder, here it is: https://filebin.net/ewzi1s8yqdak8ian

[sheson]

  On 12/16/2024 at 9:28 AM, kontotnok said:

It still crashes unfortunately

Logs: https://paste.ee/p/8BTGs

BTW it turns out the previous run did generate an FNVLODGen_log.txt file, but I hadn't closed the app yet so that's probably why it didn't appear in the folder, here it is: https://filebin.net/ewzi1s8yqdak8ian

Expand  

The log is saved when the tool closes.

We will have to try to troubleshoot one problem at a time. Delete all old logs. Download this test version of LODGenx64.exe https://mega.nz/file/wVhggDYQ#08l3-DoknmG_Np3A-sseu_GQjDPg77P4rNZzExBOwmg and replace the one in the Edit Scripts folder.

Start xLODGen, only select WastelandNV and select to generate terrain LOD meshes only. Do not check to generate any terrain LOD textures or any object or tree LOD.
Upload new FNVLODGen_log.txt.

[kontotnok]

  On 12/16/2024 at 10:24 AM, sheson said:

The log is saved when the tool closes.

We will have to try to troubleshoot one problem at a time. Delete all old logs. Download this test version of LODGenx64.exe https://mega.nz/file/wVhggDYQ#08l3-DoknmG_Np3A-sseu_GQjDPg77P4rNZzExBOwmg and replace the one in the Edit Scripts folder.

Start xLODGen, only select WastelandNV and select to generate terrain LOD meshes only. Do not check to generate any terrain LOD textures or any object or tree LOD.
Upload new FNVLODGen_log.txt.

Expand  

Here are the logs: https://paste.ee/p/tb8qw