Linux/Proton: A call to an OS function failed

[itsdnoftheworld]

Hello, I have been having an annoying issue running Dyndolod 3 on my Proton setup related to texconv. While running Dyndolod after successfully generating xlodgen and texgen, I am unable to proceed with generating Dyndolod, as I eventually get a message saying "Could not execute "Z:\home\dillon\Games\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -gpu 0 -y -m 1 -aw 256 -f R8G8B8A8_UNORM -o "C:\users\steamuser\Temp\DynDOLOD_SSE" "C:\users\steamuser\Temp\DynDOLOD_SSE\379794EC272A4EF98BAF93F0B0726085.dds". Then I get a pop-up about Call to an OS Function Failed. No .esp is mentioned.

Since I am on Linux, I don't have UAC or AV on my computer, and after changing permissions of some of the folders, specifically my game folder, mods folder, and the Dyndolod folder I have been able to run it once or twice. However, after changing a texture mod, I have been completely unable to get around this Texconv issue. I believe that Texconv should be running, since I can see it in the task manager and I can see some .dds files being converted. I also was able to run texgen and xlodgen with no issue. In fact I have run Dyndolod successfully earlier today! I have tried placing Dyndolod folder in both my Z: drive (the native Linux system) and my C: drive (the emulated Windows drive), and both run into the same error. I also reinstalled and repaired dotnet and C++, and I was able to run texconv (the official one from Github) with the below output. I also was able to convert images into .dds from the command line. However I could not replicate this with the one downloaded alongside Dyndolod. Is this expected behavior?

I am currently unable to test on Windows since my install is all messed up, but I am just looking to see if there is anything I have missed in troubleshooting. Any advice would be greatly appreciated.
 

Bug Report: https://pastebin.com/B3Te9XG4

SSE Log: https://paste.ee/p/Hhg78

Texconv Test: https://paste.ee/p/6vtVc

[itsdnoftheworld]

Sorry, here's a truncated version of log: https://paste.ee/p/d5j50.

[sheson]

55 minutes ago, itsdnoftheworld said:

Hello, I have been having an annoying issue running Dyndolod 3 on my Proton setup related to texconv. While running Dyndolod after successfully generating xlodgen and texgen, I am unable to proceed with generating Dyndolod, as I eventually get a message saying "Could not execute "Z:\home\dillon\Games\DynDOLOD\Edit Scripts\Texconvx64.exe" -nologo -gpu 0 -y -m 1 -aw 256 -f R8G8B8A8_UNORM -o "C:\users\steamuser\Temp\DynDOLOD_SSE" "C:\users\steamuser\Temp\DynDOLOD_SSE\379794EC272A4EF98BAF93F0B0726085.dds". Then I get a pop-up about Call to an OS Function Failed. No .esp is mentioned.

Since I am on Linux, I don't have UAC or AV on my computer, and after changing permissions of some of the folders, specifically my game folder, mods folder, and the Dyndolod folder I have been able to run it once or twice. However, after changing a texture mod, I have been completely unable to get around this Texconv issue. I believe that Texconv should be running, since I can see it in the task manager and I can see some .dds files being converted. I also was able to run texgen and xlodgen with no issue. In fact I have run Dyndolod successfully earlier today! I have tried placing Dyndolod folder in both my Z: drive (the native Linux system) and my C: drive (the emulated Windows drive), and both run into the same error. I also reinstalled and repaired dotnet and C++, and I was able to run texconv (the official one from Github) with the below output. I also was able to convert images into .dds from the command line. However I could not replicate this with the one downloaded alongside Dyndolod. Is this expected behavior?

I am currently unable to test on Windows since my install is all messed up, but I am just looking to see if there is anything I have missed in troubleshooting. Any advice would be greatly appreciated.
 

Bug Report: https://pastebin.com/B3Te9XG4

SSE Log: https://paste.ee/p/Hhg78

Texconv Test: https://paste.ee/p/6vtVc

Read https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs which DynDOLOD debug log to also upload when making posts.

>I also was able to run texgen and xlodgen with no issue.
>In fact I have run Dyndolod successfully earlier today!
>Is this expected behavior?
Yes.

DynDOLOD tried to execute the command line tool via a standard OS function. That OS function returned the error message.
This might mean Texconv might not have even executed at all. There might be more descriptive information in the winedebug log. You might also want to enable debug logging in MO2 and then check its log.

In any case, you might want to test with this version https://mega.nz/file/YNIlXQ7C#-NY9FcWU1B8N_QdSxtRyNrgIih7cSxSPNy0EJKBF834 to see if it makes a difference.

Test if adding LockTexconv=1 under [DynDOLOD] in Z:\home\dillon\Games\DynDOLOD\Edit Scripts\DynDOLOD\DynDOLOD_SSE.INI changes anything.

[Baguettedood]

I was experiencing this myself recently when trying to run DynDOLOD through Wine/Proton. After a few days of confusion, it appears to be this Wine bug that causes Texconv to fail if -nowic isn't supplied as an argument. This Wine bug has been fixed since Wine 9.0-rc2, and I was able to successfully generate DynDOLOD after upgrading to Wine 9.0 (in my case, I installed the full 9.0 release).

Hope this helps.

[sheson]

7 hours ago, Baguettedood said:

I was experiencing this myself recently when trying to run DynDOLOD through Wine/Proton. After a few days of confusion, it appears to be this Wine bug that causes Texconv to fail if -nowic isn't supplied as an argument. This Wine bug has been fixed since Wine 9.0-rc2, and I was able to successfully generate DynDOLOD after upgrading to Wine 9.0 (in my case, I installed the full 9.0 release).

Hope this helps.

See https://dyndolod.info/Changelog for Version 3.00 Alpha 36.
Use Texconv included in the DynDOLOD 3 Standalone archive.

"A call to an OS function failed" is a message from the OS and not the same as "FAILED [mipmaps] 80070057: Invalid parameter"

[itsdnoftheworld]

I actually got it to work a day after I made my original post, but again, I'm not sure how/forgot how. I was testing with the 32bit version of Texgen/Dyndolod and it worked fine, although of course, when running with my full mod list it runs out of memory. Later, I was switching back to 64bit and it worked that time. So I guess something on my end isn't playing well with the 64bit version of Texconv. 

[Delkry]

I've been having the exact same problem since yesterday when I try to run DynDOLOD 3 (latest version at the moment) on my Linux machine. Here are my logs:

https://file.io/2pbvPPebJGlO

You might notice that there are logs that aren't usually there (the lines with the writing and reading from files in the Temp directory mostly), because those are most likely there because I swapped the Texconvx64.exe file you provided with the one in the official repo (https://github.com/microsoft/DirectXTex/releases, the December 31st one). However, that didn't do anything and the error is still happening.

The steam log is there because the way my MO2 is set up, it opens when I open Skyrim SE on Steam instead of the launcher.

Also, I don't know how useful it is but I have a usvfs log in there. However I do know that every time the error happened

callContext.lastError()=1359

was in those logs.

Also, I tried what you said earlier in this thread (adding LockTexconv=1 to DynDOLOD_SSE.ini file under [DynDOLOD]), but that didn't do anything. I still haven't managed to run DynDOLOD successfully.

Let me know if that's useful.

[sheson]

8 hours ago, Delkry said:

I've been having the exact same problem since yesterday when I try to run DynDOLOD 3 (latest version at the moment) on my Linux machine. Here are my logs:

https://file.io/2pbvPPebJGlO

You might notice that there are logs that aren't usually there (the lines with the writing and reading from files in the Temp directory mostly), because those are most likely there because I swapped the Texconvx64.exe file you provided with the one in the official repo (https://github.com/microsoft/DirectXTex/releases, the December 31st one). However, that didn't do anything and the error is still happening.

The steam log is there because the way my MO2 is set up, it opens when I open Skyrim SE on Steam instead of the launcher.

Also, I don't know how useful it is but I have a usvfs log in there. However I do know that every time the error happened

callContext.lastError()=1359

was in those logs.

Also, I tried what you said earlier in this thread (adding LockTexconv=1 to DynDOLOD_SSE.ini file under [DynDOLOD]), but that didn't do anything. I still haven't managed to run DynDOLOD successfully.

Let me know if that's useful.

This is an error message returned from the OS. 1359 seems to be defined as an internal error occurred.

I can not really fix or are proficient in troubleshooting problems with the OS/wine/proton. Unfortunately the log is quite full of exceptions and what not, so it is hard to find which Texconv call relates to one with the actual error, because of the different timestamp formats.

You could try to minimize the need for Texconv by having less textures use BC7. For example set compression in TexGen to DXT1 and DXT5 for example.

You could test running DynDOLOD directly from a command prompt like
WINEESYNC=1 WINEFSYNC=1 protontricks-launch --appid 489830 "/linux/path/to/DynDOLOD/DynDOLODx64.exe"
to see if it runs through for the vanilla game without MO2 in the mix.

In that case install DynDOLOD Resources SE and TexGen output manually.

[Delkry]

I managed to run it successfully after setting all compression to DXT5 as you said above! I have one question though: will this result in noticeably lower quality in game compared to BC7?

I still can't say for sure that it wouldn't have worked eventually if I ran it enough time with BC7 compressed textures, but setting compression to DXT5 seems to increase the probability of DynDOLOD running successfully in this case.

Thanks for the help.

[sheson]

28 minutes ago, Delkry said:

I managed to run it successfully after setting all compression to DXT5 as you said above! I have one question though: will this result in noticeably lower quality in game compared to BC7?

I still can't say for sure that it wouldn't have worked eventually if I ran it enough time with BC7 compressed textures, but setting compression to DXT5 seems to increase the probability of DynDOLOD running successfully in this case.

Thanks for the help.

You typically not really going to see a difference. Maybe if you compare close up directly between screenshots, but overall it will pretty much look the same.

[dkfljaslkdfh]

Hi!

I'm trying to run xLODGen, but it fails after some time, showing different errors every time. The most common is "A call to an OS function failed", with TexConv64.exe.

I tried following the suggestions I found here:

But nothing worked.

I'm on Archlinux, tried proton GE, 8, 7 and 6 via Steam.

I compressed the logs all in this file: https://1drv.ms/u/s!AhMlyqThKuq1k4wj_uXXq-s2PgXOAg?e=X3SeWG

[sheson]

On 5/9/2024 at 2:06 AM, dkfljaslkdfh said:

Hi!

I'm trying to run xLODGen, but it fails after some time, showing different errors every time. The most common is "A call to an OS function failed", with TexConv64.exe.

I tried following the suggestions I found here:

But nothing worked.

I'm on Archlinux, tried proton GE, 8, 7 and 6 via Steam.

I compressed the logs all in this file: https://1drv.ms/u/s!AhMlyqThKuq1k4wj_uXXq-s2PgXOAg?e=X3SeWG

The SSELODGen_log.txt shows only one error message.
The bugreport.txt shows an out of memory error. Use the x64 version.

"Nothing worked" does not tell us anything. Instead list what was tried and how it failed.

There is not much a program can do if the OS reports an unspecified error.
It seems this error means Wine has some kind of problem to read or write to the disk.

Make sure the system is stable.
Make sure nothing interferes with file access.

If the used disk(s) are using NTFS, use different ones that use a native Linux file system.
At least the temp and output directories, for example -t:"z:\home\[USER]\xLODGen_Temp\" -o:"z:\home\[USER]\xLODGen_Output\"

If you use MO2, test without it.

A workaround might be to use 888 texture format and compress textures afterwards.

[dkfljaslkdfh]

Hey! Thank you for helping me!

I was able to generate the files just restarting xLODGen every time it failed.

Quote

The bugreport.txt shows an out of memory error. Use the x64 version.

I'm using a 1TB SSD with ext4 file system, 32GB of RAM, and I created a swap file of 6GB. The swap definitely helped with TexGen, but not with xLODGen.

I tried both versions of the exe.

Quote

If the used disk(s) are using NTFS, use different ones that use a native Linux file system.
At least the temp and output directories, for example -t:"z:\home\[USER]\xLODGen_Temp\" -o:"z:\home\[USER]\xLODGen_Output\"

The output directories were set according to the STEP instructions, in exclusive folders, separated from the game.

I didn't try the other options you mentioned because in the end, it worked. And now I have a bigger problem, Dyndolod is making fun of me!

But I'll try to solve the problem on my own before asking for help. Thanks again!

[dkfljaslkdfh]

Hi!

I'm having the same issue, now with Dyndolod. This is strange, because I was having the same issue with xLODGen, but now it's working fine.

It's exactly as described in the OP. The only thing that changes is the mod being processed at the time.

Can you take a look at the logs, please?

bugreport: https://paste.ee/p/z6QJ6
log: https://paste.ee/p/zaiPI

Thank you!

EDIT: I forgot to mention that the tools are on a drive outside of the MO2 and the system folders. Also, I'm using the x64 version. I tried with the x86 and it worked with a clean install of the game, but with the full mods it stopped without giving me an error, it just hanged.

 

Edited May 16 by dkfljaslkdfh

[sheson]

On 5/16/2024 at 3:52 AM, dkfljaslkdfh said:

Hi!

I'm having the same issue, now with Dyndolod. This is strange, because I was having the same issue with xLODGen, but now it's working fine.

It's exactly as described in the OP. The only thing that changes is the mod being processed at the time.

Can you take a look at the logs, please?

bugreport: https://paste.ee/p/z6QJ6
log: https://paste.ee/p/zaiPI

Thank you!

EDIT: I forgot to mention that the tools are on a drive outside of the MO2 and the system folders. Also, I'm using the x64 version. I tried with the x86 and it worked with a clean install of the game, but with the full mods it stopped without giving me an error, it just hanged.

Read https://dyndolod.info/Official-DynDOLOD-Support-Forum#Post-Logs which DynDOLOD debug log to also upload when making posts.

If you have the exact same issue, then the exact same explanations and tips apply.
Use backslashes only in paths.
Map drive letters to the mounted drives in the winetricks configuration to reduce path lengths.
Do not use -o command line with TexGen/DynDOLOD, let them use their default output folders or set them via the interface option.

The OS reports that it has a problem. This most likely has to do with file access problems, maybe because of file system drivers, their settings or MO2 virtual folders. Hence the repeated tip to test without MO2. Using MO2 works fine here, though.

[Maotok]

I believe I may have solved the "Call to an OS Function Failed" error. I am also on Linux using the latest version of PopOs.

What seems to be going on is some sort of permission error between MO2, the wine/proton PREFIX, and Linux's permissions. Either the permission is revoked after too long of a time or something in the coding between all the levels and programs doesn't have the correct permission. I can not change the code and I'm not digging through it all to figure that part out (I'll leave that to the code GODS).

HOWEVER, I was able to finally get it working after messing with a few other programs like synthesis. What you need to do is put everything within the wine/proton prefix that steam has created for skyrim. For me that path would look like this:

" /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/ "

I install everything skyrim related in that path including any windows programs needed to run whatever tool skyrim mods need. In this case that was MO2, vcredist, and dotnet. Make sure you install all the required windows software needed for Dyndolod (or any other tool) in that steam prefix. You will do this using this little segment of code ran in the terminal adjusted to your prefix location (my path looks like this):

" WINEPREFIX=/home/mrfuji/Games/steamapps/compatdata/489830/pfx winetricks --gui "

you will place your own install location after WINEPREFIX= *your prefix path to compatdata/489830/pfx* winetricks --gui

This will bring up the wintricks gui which you will use to install any windows drivers or software needed for dyndolod (or any other tool, this works for more than just Dyndolod) directly into the steam prefix folder for your skyrim installation. This is a critical step. We are skipping Linux drivers and treating our skyrim Prefix as its own little windows world. We are doing this to avoid any crosstalk of permissions between Linux and Wine/Proton.

You will also need to do this for MO2. Download the latest version and install it into the prefix.

Using my game paths this would look something like this:

1. *open terminal and input* WINEPREFIX=/home/mrfuji/Games/steamapps/compatdata/489830/pfx winetricks --gui   <--- change to your path

2. *Wine should run and open the gui*

3. Make sure "select the default wineprefix" is selected and hit OK

4. another page will open (if needed make sure windows 10 or 11 is selected in the winecfg, you'll know if a windows installer throws a version error at you).

5. on that same page there is a line called "Run an arbitrary executable (.exe/.msi/.msu)" Select that, hit OK

6. Browse to find the .exe for the MO2 installer and select it and run.

7. This will install MO2 into your steam prefix for skyrim.

You will use a very similar setup for every single dependent software installer needed to run some of the skyrim modding tools. Just repeat the steps above replacing "/home/mrfuji/Games" with your path to your steam apps folder. A tip: try to find the offline installers windows has to help for a smoother install of utilities, thats a howto for another day.

After you are done installing all the needed windows software you will install Dyndolod into its own folder inside wherever you installed your skyrims steamapps   

" /steamapps/compatdata/489830/pfx/drive_c/ " folder.

For me that looked like this:

" /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/DynDOLOD "

Now just launch the MO2 thats installed inside your prefix location and follow the DynDOLOD guide on how to use it with MO2.

IDK if this has anything to do with it however, I launch MO2 using a line of code in the terminal in order to skip using the steam gui launcher which keeps trying to update my Skyrim. (I know the workarounds I just don't care to change it and no I didn't pirate it. At least the pirates don't get updates....Thanks Todd.) However it may help anyone else experiencing the call to OS function error. Just modify this little line of code, turn it into a bash script or just copy paste it each time into the terminal doesn't matter to me, but modify the paths to point to your stuff. This points to my stuff following the paths listed above:

STEAM_COMPAT_CLIENT_INSTALL_PATH=/usr/games/steam STEAM_COMPAT_DATA_PATH=/home/mrfuji/Games/steamapps/compatdata/489830/ /home/mrfuji/.steam/root/compatibilitytools.d/GE-Proton9-7/proton  run /home/mrfuji/Games/steamapps/compatdata/489830/pfx/drive_c/Modding/MO2/ModOrganizer.exe

That points to my paths and if you copied all my paths above and mimiced them into your own drive this would run on your own side as is copy pasted int the terminal. Please dont do that,  learn your paths so you can better debug your stuff.

a quick breakdown:

STEAM_COMPAT_CLIENT_INSTALL_PATH=(this points to your steam install) STEAM_COMPAT_DATA_PATH=(This points to your compat data folder also know as your Prefix for Wine/Proton) (the next part points to where your proton is installed on your drive typically in your .steam/root/compatibilitytools.d/) (the last part always has "run (space)" and your path to your ModOrganizer.exe which should now be in your compatdata/489830/pfx/drive_c/ folder )

My compatibilitytools.d path points to my GE proton location and not directly to a steam released proton version, find your proton install path. If you put everything into your compat folder you should be able to now point MO2 to Dyndolod as per the DynDOLOD install guide. When selecting a file to run inside MO2 the file browser should now have a "C:" drive that is listed under MyComputer. You should be able to find your Dyndolod and any other .exe tool installed into your Prefix there. If you cant find the tool in the C: drive then you installed something outside the Prefix. You should not have to look outside the C: drive to find your .exe tools. Recheck your install paths and steps and try again. Do not point MO2 outside the Prefix for anything including your mod folder. (Downloads folder is probably fine.)

If all that checks out then DynDolod should run without throwing the Call to OS error. I wish you all well, if I need to clarify something I will try my best but I'm not a programmer and I have just kinda stumbled into half of it. So I will do what I can. I have run it successfully a couple times now for over an hour each time ( too many mods ) and it has completed its run each time. Before it would stop around the 7 to 10 minute mark and throw the call to OS error.