Jump to content

erasmux

Mod Author
  • Content Count

    11
  • Joined

  • Last visited

Community Reputation

0 Neutral

About erasmux

  • Rank
    Citizen

Profile Information

  • Gender
    Male
  • Location
    Israel
  1. Actually according to the only xLODGen source I could find (here), and according to the stack trace from the issue I had here, it seems that lodgen does load nifs in parallel. I will also add, that at least at one point I was quite familiar with the implementation of usvfs (the relevant part of MO2) and it only catches and redirects system calls such as file open requests. Meaning MO2 bugs may cause file open requests to fail, or in theory to open the wrong file, but it should be technically impossible for usvfs to effect the file contents (an up to date list of the functions usvfs hooks is here and you can see none of those hooks can effect the file contents). Additionally, as long as we are talking about files which existed and have not been renamed/moved/deleted from before usvfs was initialized (i.e. before the MO2 process spawned xLODGen and locked itself), there is no concurrency or synchronizations as nothing changes from the usvfs point of view (it is only interested in what virtual paths need to be mapped and to which actual paths). That said, if you believe you have identified an MO2 issue, it should be raised with current MO developers. BTW, any chance there are up to date sources for DynDOLOD and/or xLODGen and I missed them?
  2. At least the errors I have reported above, I have reproduced without MO running at all so they are not related to MO in any way. I am "99% sure" (not really but hey it's really easy to type 99% without any proof) this is a concurrency/synchronization issue(s) with DynDOLOD and/or LODGen. Such issues are very hard to track down and will usually work very well for a very long time and surface only in specific instances for unclear reasons.
  3. That is a shame (not your fault, of course). I tried to add bEnableStippleFade=0 or bEnableStippleFade=1 to my skyrim/skyrimcustom/skyrimprefs INIs and I don't see any difference. After some deliberation (with myself) and a few screenshots, I decided the 3D trees are too good to pass because of this issue. I have actually been using a much more broken "ultra trees" for some time now, so hopefully I don't even notice those harsh transitions unless I am specifically looking for them Here are some 2D vs 3D trees: Thanks again
  4. I have already updated my .NET and VC++ redistributable a week ago to resolve a different problem with 2.65 (which btw also looked like a concurrency/synchronization issue). Regarding attempting to reproduce this specific issue, I can only say that from my experience with multi-thread/multi-process programs such concurrency/synchronization issues are often quite hard to impossible to reproduce on a different system. I have seen synchronization issues which after they have been identified you scratch your head "how the hell did this ever work?" and yet it worked in production on many many machines without any problems until this one machine on which the issue "suddenly" surfaced. Hence my offer to help in any way I can to investigate it on my machine. In the meantime, I possibly found a workaround, by disabling all three of the "FasterXXX=0" in the DynDOLOD_SSE.ini (possibly disabling only "FasterBase=0" is enough). When disabling all three the runtime only slightly increases (from ~ 14 mins for all worlds to ~ 15.5 mins). Since I have done this I have not seen the "Error reading NIF" even once. Oddly when trying to test the stability of these options by selecting only the Sovngarde world (for quicker runs so I can test it multiple times quickly), it seems to deadlock on "Preparing texture data" about 50% of the times. At this point I have run quite a few runs on all worlds (including Sovngarde) with these options and none of them deadlocked. I am not sure how to explain this. I have also managed to reproduce the LODGen error with 2.5.6 (the version inside "Edit Scripts" of the xLODGen beta 42 archive): This still happens very rarely for me (a total of 3 times till now, once with 2.65, once with 2.66 and LODGen 2.6.2 and now once with LODGen 2.5.6). Btw using the new LODGen 2.6.2 (as opposed to version 2.5.6) DynDOLOD runs slightly faster and the result is smaller without any quality loss I can discern (do note I am not sensitive to delicate difference in quality). So at least from my point of view, very good job on the new LODGen. --- Meanwhile, I have actually managed to clean my tree mods and reach a DynDOLOD result I am pleased with . The only problem I still have, is that when generating "Ultra Trees" (TreeLOD=0), as I close in on far away trees they switch almost instantaneously from the LOD to the full model. Its definitively not the 2.7 seconds in the DynDOLOD MCM menu. I have tried to increase this setting to the maximum of 5 seconds with no noticeable difference. This bothers my eyes enough that I think I even prefer the much less refined TreeLOD=1 LODs, on which the transition is gradual (most likely the 2.7 seconds from the config, although I didn't bother to measure it). Is there something I can do to both have "Ultra Trees" and a gradual transition from the LODs to the full model? BTW many thanks for this great tool and your dedicated support.
  5. Obviously I have tested the computer general stability: I do not overclock, cpu temps are low, the computer is 100% stable (other than this DynDOLOD issue which I believe IS a DynDOLOD issue), prime95 tests run without error, memtest86 runs without errors, disk health is good and error free. Even if the computer was unstable, which it isn't, there is no chance such instability would reproduce these exact DynDOLOD errors consistently and only them (getting 0-3 such errors, usually 1 or 2, in each run and on the same files and no other errors). Also it's Windows 7 so it's not "MO2 is acting up with Windows 10". Furthermore, just for you, I have reproduced the error without MO, so it's definitively not an MO problem. I know it's not what a developer wants to hear but I am as sure as one can be in such matters that the issue is with DynDOLOD (or the tools it runs like LODGen or TexConv). Again, if there is anything I can do to further investigate the issue let me know (I do have the proper background, programming, etc.).
  6. Having some stability problems with DynDOLOD (running the latest 2.66 standalone with the 2.66 resources package), and getting errors such as the following intermittently: [00:01:02.806] <Warning: Error when loading "meshes\lod\mine\minescaffoldtop0sided01_lod_0.nif": Error reading NIF block 2 BSTriShape: Unknown NiObject: BSTriShape> [00:01:02.837] <Warning: Error when loading "meshes\lod\tundra\reach\tundrastreambend01reach_lod_0.nif": Error reading NIF block 2 BSTriShape: Unknown NiObject: BSTriShape> [00:01:02.853] <Warning: Error when loading "meshes\lod\solitude\sthalmorembassy_lod.nif": Error reading NIF block 2 BSTriShape: Unknown NiObject: BSTriShape> [00:01:02.931] <Warning: Error when loading "meshes\dyndolod\lod\trees\srg_treepineforest01_a84564afpassthru_lod.nif": Error reading NIF block 2 NiTriShape: Unknown NiObject: NiTriShape> [00:01:02.931] <Warning: Error when loading "meshes\lod\tundra\reach\tundrastreambend01reach_lod_0.nif": Error reading NIF block 2 BSTriShape: Unknown NiObject: BSTriShape> (Notice that in this example the same "meshes\lod\tundra\reach\tundrastreambend01reach_lod_0.nif" appears twice?!) Usually there will be only 1 or 2 such errors (the above 5 errors is the worst case I have seen). Running DynDOLOD again and again with the exact same options will eventually result in an error free run (always clearing the cache, export and output folders between runs and without any other changes). I seem to get these errors more frequently generating "regular" LODs (i.e. running DynDOLOD with the default TreeLOD=1 and simply selecting High from the "wizard"). Running with "TreeLOD=0", which is the only usage I have done with previous versions, also generates such errors but less frequently, so it's hard for me to say if this is new to the new version. I have also run into the following error in the LODGen_SSE_DLC2SolstheimWorld_log.txt (once): Can not get hash for shader in meshes\dyndolod\lod\trees\treepineforestdeadashl01passthru_lod.nif block TreePineForestDeadAsh01:1 - L2_:0 - L1_:0 Invalid handle. System.ArgumentNullException: Invalid handle. Rerunning DynDOLOD "resolved" the issue. This is similar to the error I reported here with 2.65, in that case rerunning DynDOLOD without any changes also "resolved" the issue (only clearing the cache, export and output folders). I have also verified all the above files are the same as their sources (which are the DynDOLOD Resource SE package, oldrim EVT or Enhanced Landscapes; The "sthalmorembassy_lod.nif" I could not verify as it is not a loose file in any of my mods). I can only guess this is some odd parallelism issue which causes the above instabilities. Please let me know if there is further information or testing I can provide to help investigate this issue.
  7. The above guide says: "Also, I highly recommend to make a modding folder for all tools, backups, custom mods and patches, etc. anywhere on a hard drive. You will be installing there a lot of important things (like xEdit) so it will be good to have it an easily accessible place." And he gives "D:\Skyrim Mods" as an example. I admit that does not specifically say that the "Modding Folder" must be outside special OS folder and outside the MO folder but common sense as well as sheson's post (i.e the one right above yours) will tell you not to do so. Did you try to use a "Modding Folder" outside the MO folder and outside special OS folders? Can you detail together the folder you are installing DynDOLOD to and the error message you are getting? As I side note, I would advise against installing MO under program files, obviously you can get (almost) any configuration to work with proper permissions on the OS level but why choose paths which may cause problems? The guide you linked specifically says "Mod Organizer 2 should not be installed inside a system protected folder such as Programs Files (x86). Also avoid installing MO2 inside the Game folder as this causes problems with the vfs library. Download the main file (Archive version), and extract the contents. Create a folder D: drive, name it Mod Organizer 2, and extract the contents to it. Run the executable, and select Portable, and then Skyrim Special Edition." I do not see any reason it has to be "D:\Mod Organizer 2" but that could work, as well as "C:\MO2" or "F:\Games\SkyrimMods\ModOrganizer2" and so on. Just make sure you don't install such tools inside windows folders (i.e. "program files"). Like already stated above, also don't install DynDOLOD under the MO folder.
  8. The issue seems to have been solved by updating the Visual C++ Redistributable (link on main post) and the .NET framework to the latest 4.8 version (from here). I can't believe how much time I wasted on this.... Even my fully modded DynDOLOD generation (through MO) finished without crashes like the above. I am getting a different error now: [spoiler=LODGen_SSE_Sovngarde_log.txt]============================================================ Skyrim Object LOD Generator 2.6.1.0 Created by Ehamloptiran and Zilav Updated by Sheson Log started at 14:52:13 Game Mode: SSE Fix Tangents: False, False, False, False Generate Tangents: True, True, True, True Generate Vertex Colors: True, True, True, True Merge Vertex Colors: True, True, True, True Merge Meshes: True Grouping: False Remove Faces under Terrain: True Remove Faces under Water: True Remove Faces Z-Shift: 10 Use HD Flag: True Ignore Materials: False Alpha DoubleSided: False Default Alpha Threshold: 128 Use Source Alpha Threshold: False Use Backlight Power: False Use Decal Flag: False Specific level: No Specific quad: No Max Level: 32 Output: C:\Games\Skyrim_SE_mods\DynDoLOD\DynDOLOD\DynDOLOD_Output\Meshes\Terrain\sovngarde\Objects\ Using UV Atlas: C:\Games\Skyrim_SE_mods\DynDoLOD\DynDOLOD\Edit Scripts\DynDOLOD\cache\DynDOLOD_SSE_atlas_map_sovngarde.txt Using Flat Textures: C:\Games\Skyrim_SE_mods\DynDoLOD\DynDOLOD\Edit Scripts\DynDOLOD\cache\DynDOLOD_SSE_flat_textures.txt Using Alt Textures: C:\Games\Skyrim_SE_mods\DynDoLOD\DynDOLOD\Edit Scripts\DynDOLOD\cache\DynDOLOD_SSE_alt_textures_sovngarde.txt Generating object LOD for worldspace Sovngarde Can not get hash for shader in meshes\dyndolod\lod\trees\srg_treepineforestsnow05_9cb969a5passthru_lod.nif block TreePineForestSnow05_LOD_FLAT:1 Invalid handle. System.ArgumentNullException: Invalid handle. at System.RuntimeTypeHandle.GetToken(RuntimeType type) at System.RuntimeType.RuntimeTypeCache.MemberInfoCache`1.PopulateLiteralFields(Filter filter, RuntimeType declaringType, ListBuilder`1& list) at System.RuntimeType.RuntimeTypeCache.MemberInfoCache`1.PopulateFields(Filter filter) at System.RuntimeType.RuntimeTypeCache.MemberInfoCache`1.GetListByName(Char* pName, Int32 cNameLen, Byte* pUtf8Name, Int32 cUtf8Name, MemberListType listType, CacheType cacheType) at System.RuntimeType.RuntimeTypeCache.MemberInfoCache`1.Populate(String name, MemberListType listType, CacheType cacheType) at System.RuntimeType.GetFieldCandidates(String name, BindingFlags bindingAttr, Boolean allowPrefixLookup) at System.RuntimeType.GetFields(BindingFlags bindingAttr) at System.Runtime.Serialization.FormatterServices.GetSerializableMembers(RuntimeType type) at System.Runtime.Serialization.FormatterServices.InternalGetSerializableMembers(RuntimeType type) at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory) at System.Runtime.Serialization.FormatterServices.GetSerializableMembers(Type type, StreamingContext context) at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitMemberInfo() at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter, SerializationBinder binder) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck) at LODGenerator.Common.Utils.ObjectToByteArray(Object obj) at LODGenerator.ShapeDesc..ctor(String gameDir, NiFile file, NiTriBasedGeom geom, StaticDesc stat, Int32 quadIndex, List`1 PassThruMeshList, List`1 notLODFlag, Boolean skyblivionTexPath, Boolean useOptimizer, Boolean mergeVertexColors, Boolean fixTangents, Boolean useDecalFlag, Boolean terrain, Boolean verbose, LogFile logFile) Log ended at 14:52:15 (0:1) Code: 3005 At least this one seems like a problem with my tree mods. Anyone have any idea?
  9. EDIT: The issue seems to have been solved by updating windows, see my post below. I am having trouble completing a DynDOLOD generation for 3D Trees: The generated crash dump (from WER) shows a similar stack trace: The issue occurs very frequently (for me), and with all worlds selected, it is unlikely that a build will complete without at least one LODGenx64.exe crash (even in the clean scenario I detail below). The issue is not consistent, with the LODGen crash occurring in different world(s) each run. Sometimes only one LODGen crash will occur and sometimes many (rarely it might even complete without an error, at least in the clean scenario below). Running such a world alone will complete without error (or possibly with a low probability of crashing, I did not test this thoroughly). This, together with the crash dumps, cause me to suspect this is a concurrency/race issue. After some troubleshooting steps I have reached the following clean scenario which still reproduces the issue frequently (at least for me): 1. Clean Skyrim SE installation (verified using "verify integrity" from steam). 2. Download DynDOLOD 2.65 from here or from nexus (I get the exact same files from both sources). 3. Installed DynDoLOD at various paths including on the root of a different physical drive. 4. Install DynDOLOD-Resources-SE.2.65.7z with default options (using MO2), then copy the entire output to the game Data folder. 5. Change TreeLOD=0 and run DynDOLODx64.exe -sse (without MO), select all worlds, "High" and start build. To be clear, the problem reproduces without MO and without any mods installed - except of course DynDOLOD Resources SE. An example of the log of such a run: DynDOLOD_SSE_log.txt LODGen_SSE_MarkarthWorld_log.txt If there is anything more I can do to help diagnose the issue please let me know. Alternatively, if someone sees something I am doing wrong I would appreciate pointing it out.
  10. To clarify the previous post: If you already have an umbrella "output" and it has build_32 and install_32 in it, you need a clean build into a clean output folder. If you only have build and install then you are already updated. Also, dependencies have been updated on the main post.
  11. I am pushing a major update to umbrella which will also update usvfs and modorganizer and many of our dependencies (such as boost, opensll, python, etc.). The main improvement here is that build_32 and install_32 are no more, duplicated usvfs code is no more. Everything is built under build and staged to install. Additionally usvfs now builds directly from git source controlled VS projects and does not use cmake any more. Staging has also been fixed and install\bin should now included everything needed for an MO installation. The current prerequisites for umbrella are: Windows Machine (64Bit) since MO2 is a 64bit binary Python 2.7.14 (64Bit) CMake (latest 64bit) 7zip (Latest 64Bit) Git (Latest) Qt 5.10 (Qt Packages: msvc2017 64-bit, qtwebengine,qtscript), you will usually get better results by selecting a good mirror for the offline installer but you can also use the online one.and if anyone missed it we are using VS2017 (not 2015). IF YOU ARE ALREADY USING UMBRELLA IT IS HIGHLY RECOMMENDED THAT YOU START FROM SCRATCH: Meaning you should build into a new clean output folder (you can and probably should keep your old output folder separately until you see everything works fine with the new build). The reason for this is that the most of the libraries have upgraded, the outputs have move to accommodate for the removal of build_32, usvfs structure has slightly changed and some of the progress files are a bit different now.
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.