Linux/Proton: A call to an OS function failed

[itsdnoftheworld]

Hello, I have been having an annoying issue running Dyndolod 3 on my Proton setup related to texconv. While running Dyndolod after successfully generating xlodgen and texgen, I am unable to proceed with generating Dyndolod, as I eventually get a message saying "Could not execute "Z:\home\dillon\Games\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -gpu 0 -y -m 1 -aw 256 -f R8G8B8A8_UNORM -o "C:\users\steamuser\Temp\DynDOLOD_SSE" "C:\users\steamuser\Temp\DynDOLOD_SSE\379794EC272A4EF98BAF93F0B0726085.dds". Then I get a pop-up about Call to an OS Function Failed. No .esp is mentioned.

Since I am on Linux, I don't have UAC or AV on my computer, and after changing permissions of some of the folders, specifically my game folder, mods folder, and the Dyndolod folder I have been able to run it once or twice. However, after changing a texture mod, I have been completely unable to get around this Texconv issue. I believe that Texconv should be running, since I can see it in the task manager and I can see some .dds files being converted. I also was able to run texgen and xlodgen with no issue. In fact I have run Dyndolod successfully earlier today! I have tried placing Dyndolod folder in both my Z: drive (the native Linux system) and my C: drive (the emulated Windows drive), and both run into the same error. I also reinstalled and repaired dotnet and C++, and I was able to run texconv (the official one from Github) with the below output. I also was able to convert images into .dds from the command line. However I could not replicate this with the one downloaded alongside Dyndolod. Is this expected behavior?

I am currently unable to test on Windows since my install is all messed up, but I am just looking to see if there is anything I have missed in troubleshooting. Any advice would be greatly appreciated.
 

Bug Report: https://pastebin.com/B3Te9XG4

SSE Log: https://paste.ee/p/Hhg78

Texconv Test: https://paste.ee/p/6vtVc

[Maotok]

I believe I may have solved the "Call to an OS Function Failed" error. I am also on Linux using the latest version of PopOs.

What seems to be going on is some sort of permission error between MO2, the wine/proton PREFIX, and Linux's permissions. Either the permission is revoked after too long of a time or something in the coding between all the levels and programs doesn't have the correct permission. I can not change the code and I'm not digging through it all to figure that part out (I'll leave that to the code GODS).

HOWEVER, I was able to finally get it working after messing with a few other programs like synthesis. What you need to do is put everything within the wine/proton prefix that steam has created for skyrim. For me that path would look like this:

" /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/ "

I install everything skyrim related in that path including any windows programs needed to run whatever tool skyrim mods need. In this case that was MO2, vcredist, and dotnet. Make sure you install all the required windows software needed for Dyndolod (or any other tool) in that steam prefix. You will do this using this little segment of code ran in the terminal adjusted to your prefix location (my path looks like this):

" WINEPREFIX=/home/mrfuji/Games/steamapps/compatdata/489830/pfx winetricks --gui "

you will place your own install location after WINEPREFIX= *your prefix path to compatdata/489830/pfx* winetricks --gui

This will bring up the wintricks gui which you will use to install any windows drivers or software needed for dyndolod (or any other tool, this works for more than just Dyndolod) directly into the steam prefix folder for your skyrim installation. This is a critical step. We are skipping Linux drivers and treating our skyrim Prefix as its own little windows world. We are doing this to avoid any crosstalk of permissions between Linux and Wine/Proton.

You will also need to do this for MO2. Download the latest version and install it into the prefix.

Using my game paths this would look something like this:

1. *open terminal and input* WINEPREFIX=/home/mrfuji/Games/steamapps/compatdata/489830/pfx winetricks --gui   <--- change to your path

2. *Wine should run and open the gui*

3. Make sure "select the default wineprefix" is selected and hit OK

4. another page will open (if needed make sure windows 10 or 11 is selected in the winecfg, you'll know if a windows installer throws a version error at you).

5. on that same page there is a line called "Run an arbitrary executable (.exe/.msi/.msu)" Select that, hit OK

6. Browse to find the .exe for the MO2 installer and select it and run.

7. This will install MO2 into your steam prefix for skyrim.

You will use a very similar setup for every single dependent software installer needed to run some of the skyrim modding tools. Just repeat the steps above replacing "/home/mrfuji/Games" with your path to your steam apps folder. A tip: try to find the offline installers windows has to help for a smoother install of utilities, thats a howto for another day.

After you are done installing all the needed windows software you will install Dyndolod into its own folder inside wherever you installed your skyrims steamapps   

" /steamapps/compatdata/489830/pfx/drive_c/ " folder.

For me that looked like this:

" /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/DynDOLOD "

Now just launch the MO2 thats installed inside your prefix location and follow the DynDOLOD guide on how to use it with MO2.

IDK if this has anything to do with it however, I launch MO2 using a line of code in the terminal in order to skip using the steam gui launcher which keeps trying to update my Skyrim. (I know the workarounds I just don't care to change it and no I didn't pirate it. At least the pirates don't get updates....Thanks Todd.) However it may help anyone else experiencing the call to OS function error. Just modify this little line of code, turn it into a bash script or just copy paste it each time into the terminal doesn't matter to me, but modify the paths to point to your stuff. This points to my stuff following the paths listed above:

STEAM_COMPAT_CLIENT_INSTALL_PATH=/usr/games/steam STEAM_COMPAT_DATA_PATH=/home/mrfuji/Games/steamapps/compatdata/489830/ /home/mrfuji/.steam/root/compatibilitytools.d/GE-Proton9-7/proton  run /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/Modding/MO2/ModOrganizer.exe

That points to my paths and if you copied all my paths above and mimiced them into your own drive this would run on your own side as is copy pasted int the terminal. Please dont do that,  learn your paths so you can better debug your stuff.

a quick breakdown:

STEAM_COMPAT_CLIENT_INSTALL_PATH=(this points to your steam install) STEAM_COMPAT_DATA_PATH=(This points to your compat data folder also know as your Prefix for Wine/Proton) (the next part points to where your proton is installed on your drive typically in your .steam/root/compatibilitytools.d/) (the last part always has "run (space)" and your path to your ModOrganizer.exe which should now be in your compatdata/489830/pfx/drive_c/ folder )

My compatibilitytools.d path points to my GE proton location and not directly to a steam released proton version, find your proton install path. If you put everything into your compat folder you should be able to now point MO2 to Dyndolod as per the DynDOLOD install guide. When selecting a file to run inside MO2 the file browser should now have a "C:" drive that is listed under MyComputer. You should be able to find your Dyndolod and any other .exe tool installed into your Prefix there. If you cant find the tool in the C: drive then you installed something outside the Prefix. You should not have to look outside the C: drive to find your .exe tools. Recheck your install paths and steps and try again. Do not point MO2 outside the Prefix for anything including your mod folder. (Downloads folder is probably fine.)

If all that checks out then DynDolod should run without throwing the Call to OS error. I wish you all well, if I need to clarify something I will try my best but I'm not a programmer and I have just kinda stumbled into half of it. So I will do what I can. I have run it successfully a couple times now for over an hour each time ( too many mods ) and it has completed its run each time. Before it would stop around the 7 to 10 minute mark and throw the call to OS error.

 

 

 

 

[Gazotti]

I've had this before with texconv. Running DynDOLOD in a terminal had it print an error along the lines of "you need .NET 4.X", so I installed it with winetricks and it worked properly.

.NET Core Desktop 7 was installed, but it seems we also need Desktop 6 (for LodGenX64Win) and 4.8 for texconv and LodGenX64.

[sheson]

It seems likely there are several possible causes for this error message as can be seen by the different solutions and tips above.

As explained before, Texconv (and other things) require the latest Microsoft Visual C++ Redistributable to be installed properly:
Start the protonfix GUI, select Skyrim Special Edition: 489830 Steam App, click through to winetricks, select the default wineprefix, select Install a Windows DLL or component, check vcrun2022 and the click OK.

Or open a console window and execute:

WINEPREFIX="/the/path/to/SteamLibrary/SteamApps/compatdata/489830/pfx" winetricks --force vcrun2022

If nothing else helps, try this test version of https://mega.nz/file/4IQlFA7B#nzdR7V95MQxBYmKDZD9agfxHOGeh1w8ZdxoO5oISaM8 and report back successes or failures.

[kontotnok]

19 hours ago, sheson said:

It seems likely there are several possible causes for this error message as can be seen by the different solutions and tips above.

As explained before, Texconv (and other things) require the latest Microsoft Visual C++ Redistributable to be installed properly:
Start the protonfix GUI, select Skyrim Special Edition: 489830 Steam App, click through to winetricks, select the default wineprefix, select Install a Windows DLL or component, check vcrun2022 and the click OK.

Or open a console window and execute:

WINEPREFIX="/the/path/to/SteamLibrary/SteamApps/compatdata/489830/pfx" winetricks --force vcrun2022

If nothing else helps, try this test version of https://mega.nz/file/4IQlFA7B#nzdR7V95MQxBYmKDZD9agfxHOGeh1w8ZdxoO5oISaM8 and report back successes or failures.

logs: https://paste.ee/p/un2Lx

I installed xLODGen and ran the x64 version through MO2, I ran it 3 times before finding this post, every error it made some progress and then crashed, I deleted the output folder and then I used the Texconvx64 that you said to test. The process failed at the same point trying to run Texconvx64

[sheson]

1 hour ago, kontotnok said:

logs: https://paste.ee/p/un2Lx

I installed xLODGen and ran the x64 version through MO2, I ran it 3 times before finding this post, every error it made some progress and then crashed, I deleted the output folder and then I used the Texconvx64 that you said to test. The process failed at the same point trying to run Texconvx64

Also upload the FNVLODGen_log.txt when making posts.

Also check all the other posts about other possible reasons and solutions being discussed.

[kontotnok]

24 minutes ago, sheson said:

Also upload the FNVLODGen_log.txt when making posts.

Also check all the other posts about other possible reasons and solutions being discussed.

But I did upload FNVLODGen_log, everything is on the same link

[sheson]

41 minutes ago, kontotnok said:

But I did upload FNVLODGen_log, everything is on the same link

Ah, sorry about that. The FNVLODGen_log misses the first line including "starting session"

If none of the other solutions above help, try to upload a copy of the txt file mentioned by the message.
[02:14] <Error: Could not execute "C:\Modding\xLODGen\Edit Scripts\Texconvx64.exe" -nologo -y -aw 256 -sepalpha  -f DXT1 -bc d -o "c:\output\textures\landscape\lod\*" -flist "C:\users\migu\Temp\FNVEdit\Texconv_*.txt">
You should check for the file right away before closing anything, though it may still be gone already. If not upload it.

There is also error logged by LODGen while it generates terrain LOD meshes for WastelandNV. Is that repeatable?

[kontotnok]

1 hour ago, sheson said:

Ah, sorry about that. The FNVLODGen_log misses the first line including "starting session"

If none of the other solutions above help, try to upload a copy of the txt file mentioned by the message.
[02:14] <Error: Could not execute "C:\Modding\xLODGen\Edit Scripts\Texconvx64.exe" -nologo -y -aw 256 -sepalpha  -f DXT1 -bc d -o "c:\output\textures\landscape\lod\*" -flist "C:\users\migu\Temp\FNVEdit\Texconv_*.txt">
You should check for the file right away before closing anything, though it may still be gone already. If not upload it.

There is also error logged by LODGen while it generates terrain LOD meshes for WastelandNV. Is that repeatable?

The Texconv file is already gone

I ran it again, trying the fixed mentioned about installing dotnet desktop but nothing (I already have vcrun installed)

logs (everything's here): https://paste.ee/p/IlUe3

Edit (Not really because I can't edit it): It doesn't work outside of MO2 either

[sheson]

1 hour ago, kontotnok said:

The Texconv file is already gone

I ran it again, trying the fixed mentioned about installing dotnet desktop but nothing (I already have vcrun installed)

logs (everything's here): https://paste.ee/p/IlUe3

Edit (Not really because I can't edit it): It doesn't work outside of MO2 either

Upload new logs with this somewhat more chatty version of texconv https://mega.nz/file/0MAGXLRI#uer6RNAlqLHAdV-bC8xAMfyhZo5RbnGqe0r-0dLSl2A

[kontotnok]

13 hours ago, sheson said:

Upload new logs with this somewhat more chatty version of texconv https://mega.nz/file/0MAGXLRI#uer6RNAlqLHAdV-bC8xAMfyhZo5RbnGqe0r-0dLSl2A

This time it ran for longer but didn't give me an fnvlodgen_log.txt file

Logs: https://paste.ee/p/5rca5

[sheson]

7 hours ago, kontotnok said:

This time it ran for longer but didn't give me an fnvlodgen_log.txt file

Logs: https://paste.ee/p/5rca5

Get this test version of xLODGenx64.exe https://mega.nz/file/8Y4iULAT#XrL2rY4lHRkXd1x8u_fqXCQwhaBNZMHqd_n49KtY8dI and upload new logs.

[kontotnok]

2 hours ago, sheson said:

Get this test version of xLODGenx64.exe https://mega.nz/file/8Y4iULAT#XrL2rY4lHRkXd1x8u_fqXCQwhaBNZMHqd_n49KtY8dI and upload new logs.

It still crashes unfortunately

Logs: https://paste.ee/p/8BTGs

BTW it turns out the previous run did generate an FNVLODGen_log.txt file, but I hadn't closed the app yet so that's probably why it didn't appear in the folder, here it is: https://filebin.net/ewzi1s8yqdak8ian

[sheson]

49 minutes ago, kontotnok said:

It still crashes unfortunately

Logs: https://paste.ee/p/8BTGs

BTW it turns out the previous run did generate an FNVLODGen_log.txt file, but I hadn't closed the app yet so that's probably why it didn't appear in the folder, here it is: https://filebin.net/ewzi1s8yqdak8ian

The log is saved when the tool closes.

We will have to try to troubleshoot one problem at a time. Delete all old logs. Download this test version of LODGenx64.exe https://mega.nz/file/wVhggDYQ#08l3-DoknmG_Np3A-sseu_GQjDPg77P4rNZzExBOwmg and replace the one in the Edit Scripts folder.

Start xLODGen, only select WastelandNV and select to generate terrain LOD meshes only. Do not check to generate any terrain LOD textures or any object or tree LOD.
Upload new FNVLODGen_log.txt.

[kontotnok]

1 hour ago, sheson said:

The log is saved when the tool closes.

We will have to try to troubleshoot one problem at a time. Delete all old logs. Download this test version of LODGenx64.exe https://mega.nz/file/wVhggDYQ#08l3-DoknmG_Np3A-sseu_GQjDPg77P4rNZzExBOwmg and replace the one in the Edit Scripts folder.

Start xLODGen, only select WastelandNV and select to generate terrain LOD meshes only. Do not check to generate any terrain LOD textures or any object or tree LOD.
Upload new FNVLODGen_log.txt.

Here are the logs: https://paste.ee/p/tb8qw

[sheson]

2 hours ago, kontotnok said:

Here are the logs: https://paste.ee/p/tb8qw

What proton version are you using?
Have you installed dotnet48 through winetricks or are you using mono (which version)?