Streamlining the SRLE - LOTD Installation with checkpoints

[epsilon_0]

My first time on this forum and I love the guides that have been put here. I think they are one of the most amazing pieces of work for Skyrim.

 

I am trying to install the SRLE LOTD guide and its not my first time installing mods (I have played with STEP-Extended before). But this guide is way to huge and takes a lot of time to install, no complaints on this front.

 

I wanted to know if it would be possible to make certain checkpoints in the guide which can tell you if you have messed up on the way. For example, the number of plugins you should have after each section, so you know that you haven't missed some parts of the special installation by mistake. I know that this might be too much to ask, but I don't know of any other way to understand how to debug this guide. It is similar to how you would print stuff out if you are writing a program, do sanity checks every step of the way (https://en.wikipedia.org/wiki/Sanity_check#Software_development).

 

One of the ways to do this would be to check for correct ESPs present after each section (just a list of current esp in the load order can be attached so that people can check their current installation).

 

Another check that needs to be done is that people need to have the correct versions of each esp present in their load orders (there are a lot of esp that need to be downloaded from dropbox or are updated using patches from the optional files). In some places you also need to manually clean the esp or remove some records. If someone misses this it is hard to understand without going through the whole guide and rechecking this. My proposal to do this is to have a SHA1 hashing for each esp present in the load order. This will tell you whether you have currently made a mistake somewhere.

 

 I am currently just before the "merging" stage of the. Meaning I have done the DDSOpt and installed all mods in their respective folders (I hope I have done it correctly). The thing is if you mess up the mod installation, you would have to redo the merging stage again (only the mods you messed up but still a pain). So I wanted to make sure that by current installation is correct.

 

I have attached my current load order with 384 esp/esm files. I have also attached the SHA1 hashes of all the current ESPs in the load order. I would appreciate if someone can tell me if I have gone wrong somewhere.

 

I have also written the program for calculating the SHA1 hash of each ESP currently in your Mod Organizers mod directory. Just paste the file there and run it, it will create a file called CHECKSUMS.tsv which has the information of each ESP. I think this will be helpful for a lot of debugging with the longer guides where you can mess up a whole lot. (I have currently made both files to be of type txt, you have to manually change the CHECKSUMS to .tsv and the hasher to .py)

 

If people think this is a good idea I might write a small script to check which ones are wrong on your end as well, currently you have to manually go through the files to check for discrepancies.

 

## To use the hash generator you need python

loadorder.txtFetching info...

CHECKSUMS.txtFetching info...

hasher.txtFetching info...

[DarkladyLexy]

yes the length and complexity of this guide of one of the major criticism and the fact we use some unorthodox modding techniques. I am all for some sort of sanity check list just never worked out the best way to implement one. I the guide is already massively wordy as it it. 

[theblackman]

Length and complexity aside (which I don't mind, btw), I find the layout to be more user friendly than STEP, although I understand why STEP is configured the way it is - easier to maintain.  Not having to Ctrl-click open a new browser tab to see what the additional instructions are, and then close the tab once I've finished installation, does save time, especially when the additional instructions are just one line.  With SRLE: LotD, it's all on the one page.

 

I also like the brief descriptions about the mods and notes which often help to answer: why am I installing this, what does this do, or, why am I configuring it like this?

 

But the OP has come up with a great idea - I'm looking forward to testing out the utility. :)

[DarkladyLexy]

Maybe we something as simple as a script to you run before the merge section so users know if they missed up and maybe be after the  merge section but that is way above my skills level to make i know next to nothing about programing.

 

I am always open to ideas an suggestions on how to make thing easier for the end user cos i have installed this guide and SRLE Base so many times i do certain things automatically and don't even think about why i am doing it. Then wonder why i get questions as to why i did something a certain way. Or is think things I do are basic common knowledge and there not like creating dummy plugins using Wyre bash so you can clean mod of there Masters as recent example.

[epsilon_0]

Thanks a lot for your feedback. I'd like to make this guide more user friendly as well.

 

Some other suggestions which people can comment on.

 

* So I wrote another script, checker.py, which takes the CHECKSUMS.tsv from the user and CHECKSUMS.guide.tsv (if that is what it is called for the guides mod list) and outputs all esps which are not the correct versions.

 

* Run the checker.py from inside Mod organizer. Not yet tested to see if it possible. But seeing as the skyproc patchers are java and can be run, this should be technically possible. This will need the a different script.

 

* The checker can also look at your whole installation and see what is missing, like if you forgot some texture file updates by finding a different hash. A downside is that this will go through all your files and take a bit of time. But it can be made such that it will run in the background and you can use your computer for other things (just don't install new files to skyrim mod organizer while this script is running). Or you can run it overnight and it will output the data to a file anyway.

 

A very small thing which bothered me: There are "special installation" and "post installation" things that are in the guide after some mods. I don't think they are very different from each other. Can such sections have a common name, just "special installation", so that you can go through the guide by using Ctrl+f, to see if you have missed some things? I don't remember if there are other sections with different names, one I remember is the LOOT section. It would be better if these sections can be mentioned at the top of the guide and for each mod they are bolded, so that they are easier to search through.

 

***** rename the checker.txt  to checker.py, make sure that you have run hasher.py in the Mod Organizer\mods folder to get CHECKSUMS.tsv, and the correct one for the guide should also be in the folder, called CHECKSUMS.guide.tsv. The checker will run and give you the names of all missing, incorrect and extra ESP files that you have which are different from the guides.

checker.txtFetching info...

[DarkladyLexy]

  On 9/28/2017 at 12:32 PM, epsilon_0 said:

Thanks a lot for your feedback. I'd like to make this guide more user friendly as well.

 

A very small thing which bothered me: There are "special installation" and "post installation" things that are in the guide after some mods. I don't think they are very different from each other. Can such sections have a common name, just "special installation", so that you can go through the guide by using Ctrl+f, to see if you have missed some things?

never thought about that yes i can sort that out and unify them all (it was mostly a lazy copy and paste thing anyway).

 

so you think there should a LOOT Rules Section with all the rules together? That might be best at the end.

[DarkladyLexy]

another thing i have been meaning to sort but not done due laziness on my part to change some of the the tags ie Patches from the nexus page should say patch central

[epsilon_0]

Two things which I can think of right over the top of my head

 

1) Make a LOOT Section at the end which tells what changes to make

 

2) Make a special installation section where there are links to all special installation mods (only the links). This makes it easy to keep track of these changes.

 

As an aside, can someone let me know the total number of ESP before starting the merge section? Is this easily available to someone? I am planning to start the merge section today.

[DarkladyLexy]

  On 9/28/2017 at 2:24 PM, epsilon_0 said:

Two things which I can think of right over the top of my head

 

1) Make a LOOT Section at the end which tells what changes to make

 

2) Make a special installation section where there are links to all special installation mods (only the links). This makes it easy to keep track of these changes.

 

As an aside, can someone let me know the total number of ESP before starting the merge section? Is this easily available to someone? I am planning to start the merge section today.

1). shouldn't be too difficult to implement and make sense really. I'll sort that out at my next update.

2). I'm not sure how to link to other sections of the guide so i will have to work that out.

 

i believe it around the 380ish mark not exactly sure and can't check at the moment.

 

I do know of the top of my head by the time you've finished everything as in your ready to start playing your active esp count should be around the 200 mark (if you haven't deviated from the

guide).

 

Edit: by the merge section your esp count should approx. 384

[epsilon_0]

  On 9/28/2017 at 3:26 PM, DarkladyLexy said:

 

2). I'm not sure how to link to other sections of the guide so i will have to work that out.

 

 

Edit: by the merge section your esp count should approx. 384

In html you can add a local link tag using href, https://www.echoecho.com/htmllinks08.htm. It won't be much harder than adding the LOOT links.

Thank you very very much for doing this.

 

Also I found I missed the desyncbirdofprey.esp, by making a mistake in the DynDOLOD Resources installation. These are the kind of mistakes that can be avoided if the hasher and checker that I gave are used and their output is given at the end of each section.

 

So the total number of mods at the end are at least 385. If I have not messed up somewhere.

 

Will continue doing merge and report on the working (or lack thereof) of the game.

[epsilon_0]

Good news: The whole system works and looks amazing. I played for around ~20 mins. Not enough to debug all quests and locations but it seems to working.

Just a trivial question, the load times are a bit on the longer size when switching cells. Is this normal? I haven't run DynDOLOD yet, does this impact the load times? Performance? VRAM consumption? I know it generates the LOD files but I don't know how the engine handles them... It's not the most versatile system I have seen but we gotta work with what we have.

Thanks a lot for the answers.

 

I have a correction to the previous post, but I am not sure how to edit posts. I can't seem to find that button. Can someone tell me how to edit a post?

You are correct in saying that there are 384 mods, just before the merge section. I had an extra one from FNIS Spells.

After the whole installation is complete you get 199 mods, after doing the SkyProc patchers and everything.

(------

Note for the Dual Sheath Redux Patcher:

It runs out of memory because there are a lot of mods installed in this guide. The Extra Memory patches don't work on a 64 bit system as it calls a .bat file which is run on a 64 bit command line. Mod Organizer cannot hook into that and gives an error.

Fix:

Add the Dual Sheath Redux Patcher.jar as an executable into Mod Organizer. MO adds the extra commands that are needed in the "Open In" and "Arguments" section of the executable. You need to add an extra argument at the start in the arguments box. Add "-Xmx1024m", without quotes before the "-jar" in the arguments. This increases the heap size allocated to the jar executable. As an extra fact, this guide makes the Dual Sheath Redux Patcher use ~580MB of RAM while it is running. The default heap allocation only extended to ~470MB for some reason.

Hope this part helps some poor soul who is banging his head to get this to work.

------)

In all the whole guide took around 5 days to install, with one mess up in DynDOLOD Resources, which was caught because it was in a merge.

My manual way of debugging the guide was to keep looking at the modwatch mod list that is present at the top of the guide. I also used that to cross check my final installation esp count and load order.

 

I want to make sure that there is a better way of keeping these checkpoints. I am still keen on the checker and hasher method. Though there are a lot of ways of extending the simple hasher that I have written to make it more customizable. Let me know if someone has any suggestions and I can put them in code.

 

Simple things which I can think of right now.

* Make it a .exe file which can be run by giving it the path for the mod installations so that people don't have to work with python and all.

* Make it so that you can choose which section you currently are on and it will automatically tell you what all should be present and if it is the correct version.

   This can be implemented so that it can check only for esp files, but it can also be made so that it can do it over ALL FILES that should be present in the installation. This will take time both for me to code and for people to run, so this is in the back seat for now.

 

Suggestions are welcome .

Edited September 29, 2017 by epsilon_0

[DarkladyLexy]

yes the DSR issue is a recent event I need to mention that the argument needs to be added. I guess one of my recent additions or changes has pushed us over the edge. As far as i know Skyproc patchers only works with 32 bit java so you need to use the 32 bit version even if you have a 64 bit OS.

 

as for editing posts you edit after 15 minutes unless you have a shiny badge which allow you too.

 

I like your idea of a executable you need to make it a simple as possible so a novice modder can use it. I know this guide is meant to for more advanced modders buy we getting new modders find the guide and jumping in at the deep end with little to no experience the moaning at me about it.

[hishutup]

Scripts are a possibility and packing a python script into an exe isn't hard.

One thing I would like to do is to convert skyproc patchers into zedit patchers. The scripting in zedit hasn't clicked with me yet and documentation isnt there yet.

[jdsmith2816]

  On 9/30/2017 at 12:31 AM, hishutup said:

Scripts are a possibility and packing a python script into an exe isn't hard.

One thing I would like to do is to convert skyproc patchers into zedit patchers. The scripting in zedit hasn't clicked with me yet and documentation isnt there yet.

Have you grabbed mator's documentation branch?   The docs for xelib are done I think and they're quite good.