SKYRIMLE Optimized Vanilla Textures (by tony971)

[alt3rn1ty]

I'm definitely interested in working with you to make further optimizations.

:) I think you misunderstood, I am thinking about maybe one further update to my VRT and I am done. Done as a done thing in a barrel full of done. I will still be interested of course but I have achieved the goals I wanted to for personal use in modding. These days I am mostly just trying to find holes in the game to append to the UPP bug tracker, and testing betas where I might be useful.

[tony971]

Gstaff replied in a good way on the forum thread you linked ... please share broadly for the record. Arthmoor may be interested. I think we are good to go with distributing a DDSopt-ed bundle of the vanilla textures, but I am awaiting an explicit 'yes' pertaining to STEP's use (he basically responded 'yes' by proxy of the thread post though).

This just made me think of Vano89's Bethesda Hi-Res DLC Optimized... He's got a standalone 1024 version that he says doesn't require the DLC...  :ermm:Edit: I've PM-ed The Vampire Dante about it. Let's see what he thinks. He cleared mine, after all.

Edited August 29, 2014 by tony971

[EssArrBee]

This just made me think of Vano89's Bethesda Hi-Res DLC Optimized... He's got a standalone 1024 version that he says doesn't require the DLC... 

I don't think that applies. They meant that you have to have the DLC that costs money. Vano's mod doesn't require the HRDLC, which is free and even if you don't own the DLC, the textures for it that are included will not get used for anything. 

[tony971]

Alright, I've finished making all of my installation tweaks.The Skyrim - Textures.bsa and all three HighResTexturePack BSAs are direct replacements. The other three DLC have their own new optimized texture BSAs.Non-MO users now have two loading options for the BSA DLCs:1. Use dummy ESMs.

2. Run an installer file that changes the required settings to hard-code them into the appropriate ini files.MO users' downloads have been separated so that they may be placed individually in the asset load order and managed with MO.Since these are all STEP optimizations, what would you think of me changing the author name to "The STEP Project" and then adding write permissions for the appropriate admins?

Edited August 29, 2014 by tony971

[z929669]

Your prerogative. If you think that STEP personnel contributions warrant it, then you could always say: By tony971 & the STEP Team.

[Kelmych]

Since it might be relevant to this mod, the batch files used in the DDSopt guide for vanilla sorting aren't frozen; they haven't been changed in a while because I hadn't seen comments on problems that indicated a need for changing. There hasn't been explicit testing of the approach I took for the roughly 10K small terrain textures, but I haven't heard any comments on problems so I suppose it's working alright.

 

Phazer brought up an issue recently about the exterior normal maps recently, and it might sense to make a small change to add the 1kx1k exterior normal maps in the set that DDSopt sees and thus can have their resolution changed. Currently only the 2Kx2K exterior normal maps are sent to DDSopt. There are only a few (~5 or so) of the 1kx1k ones, but Phazer pointed out that some users will want to reduce their size.

[tony971]

And there's the whole collapsing 1x1 textures. Those probably need un-collapsed.

[Kelmych]

No. According to Ethatron (DDSopt author) graphic cards are supposed to be able to directly handle collapsed textures, and loading larger files than needed just raises VRAM use. The issue is which collapsed textures cause problems, and that is what I tried to address in the optimization by preventing a portion of them from being collapsed.

[alt3rn1ty]

Well my two pence worth ..

 

I think Kelmych is right, and I am sure I have seen Ethatron mention the same too who is far more expert on such matters than any of us imho.

 

In the early days of putting Vanilla Reduced Textures together I experimented with DDSOpt ( in its earlier versions ) quite a lot before I decided I could get better detail out of reducing textures with a batch scripted GIMP without it doing mipmaps ( GIMP allows you to export textures in specific formats without mipmaps - Batch a few hundred textures in GIMP applying filters for a little more noise at lower resolution, and resizing, and then add to that mipmapping before it saves them .. It takes ages ! :) ), and just using DDSOpt for its excellent remipmapping of the lot after GIMP has finished them ( plus the odd bunch of normals were put through for increasing Z steepness too ), so these collapsed textures became a common sight.

 

I have tried them, and could not determine any problems in game.

 

 

However, I also think if the objective is to optimize textures, then leaving these collapsed textures the way they are is introducing unnecessary extra GPU processing on every one of them.

Each one now is adding an extra task of upscaling them each time they are all used.

 

To me logic says that = Less optimal than just including the original texture instead of the DDSOpt squished ones.

 

I dont think we will ever be able to detect the difference between the two cases. So is it worth caring over or not ..

Toss a coin :)

For me I ensured I never get the problem in my textures, it was just a pride thing really about making all the textures the best they could be for my objectives of major VRAM use reduction while preserving as much detail as could be, and optimizing as necessary ( though the latter was not a major part of my mods goals ). I did realise that preserving detail on these particular textures was not a consideration, because they are just one colour with no detail to preserve, but I consider the extra work the GPU would have to do as being less optimal

Maybe they would save a tiny bit more VRAM, but consider the types of machines my textures are likely to be used on, I saw one user attempting to get Skyrim running on a Pentium 4, with Geforce 3200 256mb VRAM. 

I talk to loads of people out there too who mistakenly think that Intel Integrated Graphics on a laptop are good enough for the game, so long as they have a higher than minimum spec CPU .. is Steam really crap at displaying Minimum Specs for a game ?, anyway I digress.

 

 

If you extract Tony's latest Skyrim - Textures.bsa from the silent updated main file yesterday, the no MO one, and have a look in the Terrain Tamriel folder, sort by size column, and highlight all the 1k files = 518 of them in that folder alone

Presumeably that is the STEP procedure results for everyone ?

 

 

 

In the sovngarde folder there are 377

In skuldafnworld folder there are 576

markarthworld has 33

deepwoodredoubtworld has 227

blackreach has 74

 

I have no doubt that the DDSOpt.ini and filtering when used correctly are stopping any others being collapsed, like the colour.dds files in the root of textures, and the odd file like the nail head, but the above is a fair chunk of textures to leave in that state if they are causing extra realtime game engine graphics processing to be necessary

 

Two pence worth.

 

Edit : This mod could be regarded as the STEP official endorsed version if you refine its details as necessary .. Why make another one ?, if you have corrections to any of these BSAs after further adjustments to the STEP procedures .. Just fire back a corrected BSA - Tony has already mentioned ( I think ? ) he would give any of the STEP team Admin rights to edit the mod pages files.

Edited August 30, 2014 by alt3rn1ty

[alt3rn1ty]

One more for Tony :

 

Reference Wrye Bash user on Nexus ( and NMM users ?, I havent used it much apart from making dual scripted installers for some of my mods .. but I think the same concern applies )

 

Those two modding tools will not ( by design ) extract and install your executeables

 

The authors years ago made a decision never to let the tools allow anyone to install malware ( which is why .dll and java executeables are not allowed too )

 

( Go to Wrye Bash Pictorial Guide, images tab, page two of the images, screen 13 of the Installers advanced section ( but note : These are the standards for the Oblivion game ) )

 

 

So how about an archive that includes your Skyrim - Textures.bsa renamed, with its own fake ESM, to have Skyrim and update as masters, and load before USKP ?

 

Archive root

  Standard

    OptimizedVanillaTextures.esm

    OptimizedVanillaTextures.bsa

  Dawnguard

    Dawnguard - Textures.esm

    Dawnguard - Textures.bsa

 

etc

 

This way you dont need the ini, you dont need the installer exe, and Wrye Bash and NMM users have an installer that works for both those tools, and you still get to place your fake esm before Unofficial Patch Project files so that their fixed textures which you dont include win aswell.

 

I know it means they still have the original Skyrim - Textures.bsa installed, which all of this will override, but I use a similar thing with my esps on an audience who typically do not have machines good enough for skyrim, and it works very well performance wise.

 

 

It would be better if we could include the original Skyrim - Textures.bsa so that you could provide a way to overwrite with the original upon un-installation .. But as we all know that option is definitely out of the picture legally.

 

So anyway, just another option for thought :)

Edited August 30, 2014 by alt3rn1ty

[alt3rn1ty]

If anyone wants to use it, I just made another batch file for the Terrain folder

 

 

This one is custom made just to separate the terrain files out of all the vanilla files, which DDSOpt would collapse ( Based on Tonys results having used DDSOpt and the files in his Skyrim - Textures.bsa. )

 

Extract this Working folder to somewhere with plenty of space on your HD

 

Get the original ( vanilla ) textures out of skyrim - textures.bsa

 

You will need to copy the terrain folder

 

And paste that terrain folder inside the working foldertextures

 

So you end up with 

 

Working folder

"Delete Non Collapsed Vanilla Files.BAT"

Textures

  Terrain

 

.. and all the vanilla files inside that Terrain folder

 

Now double click the "Delete Non Collapsed Vanilla Files.BAT" file

 

and it will delete all the files which DDSOpt will NOT COMPRESS to 1x1 resolution

 

 

Result = A bunch of vanilla files which have not been collapsed, which you can overwrite all of the collapsed ones with.

 

So just to be clear - This leaves you with all those files I highlighted in a previous post which had been collapsed, separated out of the vanilla files so you have the same files in their vanilla size to either process differently, or copy over the DDSOpt collapsed ones.

 

You could try and process them differently with DDSOpt ( if there is a way of making them not collapse but does optimise them compaired with the originals ? ) before overwriting the collapsed ones.

 

 

Batch File here https://dl.dropboxusercontent.com/u/86040572/working%20folder.7z

 

I have zipped it up in a working folder with a textures folder and empty terrain folder, so unzip this wherever you want to do the operation, and copy the terrain folder appropriately.

 

 

 

Edit : As per previous batch files, anyone is free to use as required.

Edited August 30, 2014 by alt3rn1ty

[tony971]

Performance difference before/after this change? It seems like you and Kelmych have disagreeing opinions on the effect.

[alt3rn1ty]

I think it will be barely detectable either way up. But its the STEP forum so nailing such details and getting it right might be important.

You now have the tools to do it whichever way a lot easier.

 

I did mine with GIMP, but I also need to bring them down in resolution from vanilla 256x256 LODs and Map textures, to 128x128 in my mods 512 options, and 64x64 in my mods 256 options ( Similar to the optional resolutions offered by Ethatron in his mods High Quality LODs + High Quality 3D Map ).

 

But if you havent used GIMP before it may be too much for just the one task to get to grips with.

So leaving them as per vanilla is an option IMHO is better than the collapsed ones as mentioned in the previous posts.

 

STEP forum is famous for testing one bunch of textures versus another, empirical evidence

 

Maybe running the game with this set of non collapsed textures, versus running the game with the collapsed textures needs to be tested for .. My batch file above would work on your textures too, leaving behind the 1x1 textures out of your processed one.

A. VRAM Reduction

B. Better / Worse performance

I can see VRAM reduction being very minor and not worth having collapsed textures for. And the trouble with the latter test is very much going to be dependant upon machine specs, and my feeling is ( being a regular user of a very low spec machine ) that non collapsed is better. But its just a feeling, could be rubbish / placebo / graphic card dependant ( I have a Geforce 8600M GS in my laptop, which now has 512mb VRAM, it used to have just 256mb ). It would need some very accurate measurement.

 

Consider your target audience and what machines are they likely to be using.

 

 

Right thats my last edit :), I do believe I have nothing further to divulge which may help this project and its goals.

Edited August 30, 2014 by alt3rn1ty

[alt3rn1ty]

Oh just realised something you may be concerned with, if your GIMP fu fails, you are still going to have to process them in some way to have them in the mod ( that is should you decide to go down the non collapsed route at some point ), and if DDSOpt will always squish them .. Sorry but I have no help there.

 

Maybe that other texture optimiser tool doesn't collapse them ?, but would force remipmap them ?. Dont know for sure I never used it.

 

https://www.nexusmods.com/skyrim/mods/12801/?

Edited August 30, 2014 by alt3rn1ty

[Kelmych]

For texturesterraintamriel files from Skyrim - textures, as an example, after optimization there are 5602 texture files that are 43 kb (the original size) and 518 texture files that are collapsed (1kb). Since DDSopt collapsed less than 10% of the files my expectation is that DDSopt analyzed the files and only collapsed those files that it felt should be collapsed.

 

The impression I get from the discussion is that you feel that textures should never be collapsed. If we are going to have that discussion we need to get some people involved who have a much better understanding than mine of exactly how the graphics cards process collapsed textures. Ethatron seemed fairly certain that collapsing some textures is appropriate and that it decreases (vs. increases) what the graphic card needs to do.