SKYRIMLE Optimized Vanilla Textures (by tony971)

[alt3rn1ty]

Ethatron seemed fairly certain that collapsing some textures is appropriate and that it decreases (vs. increases) what the graphic card needs to do.

I haven't seen that ( the way you put it ), but if true I would go with what Ethatron has said. Without a doubt.

[Aiyen]

Another thing to consider is we do not really know if some of the games shaders actually force only one level of mip to be used... it is not unheard of that certain shaders operate in this way because there is no point for them to load all levels. It is a fairly simple way to optimize GLSL and HLSL code after all. 

Hence even if for one texture one thing would make sense then it would not be equal to saying it would behave that way for all of them. We simply have no way of knowing. 

 

I guess one can say that this debate have reached the point where the returns are only academic.. and not practical at all. 

[alt3rn1ty]

Yes I understand.

 

I have found the bit that I remember Ethatron saying which made me cautious of what I am providing to people in a mod ..

 

https://forum.step-project.com/topic/190-wip-ddsopt-texture-overhauls/

 

Q2.

A2. "planar (1x1) textures": The number of textures which have only one distinct value. These textures may be converted to 1x1 size without loss. Though seems not all hardware likes that. Haven't yet understood what's the real cause.

 

 

This made perfect sense to me. Having seen the effect of a new modder providing non-power of two textures in a mod, and the varied effect it had on other peoples machines with different cards and capabilities, the results were between No problems to stuttering to CTDs.

 

And to me if Ethatron has doubts, or a lack of understanding about using 1x1 textures then I am certainly going to be cautious.

 

I recall Ethatron saying something similar too on BGS forum before he participated on the STEP forum on the same subject.

 

Anyway, since doing my own testing during development of VRT on my less than desireable but faithful laptop, and I realise I have nothing to prove anything but a feeling that one setup was better than the other, I decided not only did I think it was better with non collapsed textures, but also if I am providing files to a wide audience with whatever machines are in use out there...

 

... then best err on the side of caution if 1x1 res textures may give some users problems, and provide the textures with at least a bit of meat on them.

 

 

Edit : Are there any later quotes by Ethatron than the one I point out which put the information I found out of date with newer findings ?

Edited August 30, 2014 by alt3rn1ty

[Kelmych]

As mentioned in some of the previous posts, it would be very hard to test which approach is best since these terrain textures are not likely to have a significant effect on VRAM or GPU loading. I can't think of any way to isolate the portion of the VRAM or GPU load they cause. I expect that the collection of these smaller textures currently in the graphic card memory reduce VRAM usage a small amount, but of course this is just my supposition.

 

When I was expanding the procedures for optimizing vanilla textures in the DDSopt guide, I made changes because the procedures in the guide at that time would collapse all or nearly all the terrain textures. I wasn't sure this was necessarily bad, it just seemed unnecessary especially when I found that using different (and more general) Constraints menu parameters that it would only collapse a small percentage of these terrain textures.

 

When we asked Ethatron about this issue, his answers were quite short unlike those for other topics we brought up.

[tony971]

I've released one last silent update for the mod, adding support for Wrye Bash users. Unless further optimizations are found, the mod should be considered complete. I've also listed The STEP Team as co-authors. Would any admins like write privileges?

 

Edit: if this ends up getting added anywhere in STEP, I'd hold off until LOOT sorts the plugins in the correct order.

 

Also, LoJack has put in an issue for Wrye Bash to stop it from saying that the DLC have missing string localization files. See here.

Edited August 30, 2014 by tony971

[tony971]

Okay I lied. I updated again. 

 

Changes since 2.0:

 

1. Added Wrye Bash support.

2. Fixed Textures Uninstall.exe for Creation Kit users.

3. Added optional merged versions of my High Res DLC optimizations and Vano89's.

4. Incorporated the Unofficial High Resolution Patch into the Optimized High Res DLC (both versions).

5. Added a plugin-loaded DLC textures installation option for those who don't like the ini trick.

 

Changes since the last silent update:

 

1. Made Wrye Bash support better.

2. Fixed Textures Uninstall.exe for Creation Kit users (again).

3. Incorporated the Unofficial High Resolution Patch into the Optimized High Res DLC (both versions).

4. Fixed the file structure of Optimized High Res DLC Merged Part 3.

 

Known issues:

 

1. LOOT sorts the DLC plugins wrong. I've notified them and they should start sorting correctly soon.

2. TES5Edit and Wrye Bash throw missing string localization warnings if plugin-less install options are chosen. This is because those programs don't read inis like Skyrim does. They have been made aware and are developing updates to fix this. In the meantime, you can fix this by opening Skyrim.ini in Documents/My Games/Skyrim and Dawnguard.ini, HearthFires.ini, or Dragonborn.ini in your Data folder. Edit the sResourceArchiveList2 line in Skyrim.ini to match the line in the DLC inis.

3. Avira and a couple minor antiviruses think that Textures Install.exe and Textures Uninstall.exe are viruses. I've submitted false flag reports to them.

[Aiyen]

In line with what I mentioned earlier then diffuse textures should always be power of two since most engines expect this to be the case. The underlaying reason is that their shaders work in a way where the expect this to be the case... if it is not then the shader will fail.. which in turn will make the texture fail to load... which in turn can make the game CTD due to lacking an asset. 

 

One example of a single pixel texture is in ENB

The adaptation texture is 1x1... it is just the average value of the entire screen luminosity.. ie a greyscale value.  

 

Based on what I have dug up... heard, read etc. on the topic then this is more or less the same thing in most engines. For some effects etc. then if you can compress all the relevant information into a single pixel then you do that to conserve performance. However again this only works if the shaders are made to work with this. 

So in skyrims case then it would at the very least be safest to simply have the textures being a power of two. 

 

Also if anything but an EXTREMELY large number of  16x16 textures are what prevents you from running the game then you most likely REALLY need to upgrade. 

[alt3rn1ty]

Also if anything but an EXTREMELY large number of  16x16 textures are what prevents you from running the game then you most likely REALLY need to upgrade. 

Who do you mean by "you"

 

All the people out there, who may have a problem with this ( potentially ) STEP endorsed texture collection .. Because we couldn't be arsed to provide a few textures in the format the game expects them to be in for graphics cards that ordinarily can run the game ?

Now they have to upgrade because of a bunch of textures.

 

Bethesda do not provide the textures collapsed, so why should we ?

"Guys I have a great idea for optimized textures" - Superb

"Guys lets not care about some people who may need to upgrade to use this" - Bad juju

Most people are going to struggle on just understanding how to install this, let alone trouble shoot a little known graphics card technical problem.

 

Thats a bit of an elitist attitude to less fortunate people. This mod is likely to be used by many people from all walks of life. Especially when it is perceived as performance enhancing - This will attract people with less capable machines.

 

It will also attract Laptop gamers, who cannot so easily upgrade graphics cards. I think I am probably one of the lucky few out there who has a laptop machine which can upgrade its graphics card, but even when you can do that, you are restricted in how much you can upgrade by the efficiency of the laptop to shift heat out of its casing. And obviously by voltages required, tolerances.

 

I personally dont have a problem running them either way up apart from a feeling one is better than the other, but I know there are people out there who will, as does Ethatron obviously.

And I believe we are assuming it is only going to be older graphics cards which cannot process problematic textures. Is that the case ? ( I dont know to be honest ), could occasional graphics drivers updates also complicate matters ?

I recently 'upgraded' NVidia graphics drivers from 337.88 WHQL to 340.52 WHQL .. And pretty soon went back to 337.88 with a clean install routine. 340.52 for my machine were awful and introduced stutter I do not ordinarily experience. Whereas 337.88 are the best performing drivers I have ever used for NVidia cards.

 

( Edit : Incidentally the 340.52 drivers also introduced stutter on our far more capable family desktop machine, which also is far better with 337.88, so the problem with 340.52 was not just isolated to my laptop NVidia card. My eldest has another laptop which is awesome but unlike the other two machines in our house this one is an Optimus setup - So I have a range of machines at my disposal for testing things and my opinions are not just derived from one old laptop )

 

As I said previously, I think its better to err on the side of caution .. But thats up to you guys how you do this.

 

------------------

 

Okay I lied. I updated again.

 

 

 

:) Yeah, I have had quite a lot of "Final" versions. Never say never. Welcome to modding large projects.

 

Edit : Oh hell I nearly forgot to mention this - In addition to the Wrye Bash stuff we discussed in PM on Nexus forum : Something else you may find really useful in future as a good source for monkey see monkey do Wizardry = Metallicows Wrye Bash Wizard DEMO - Take it to pieces, it has examples of just about everything you can imagine doing with wizards in there. Wrye also did BAIN mod examples a few moons ago too. Compare these with the official Technical Readme command listings and you have everything you need to adapt to any mods installation requirements.

Just watch out for those common mistakes :), Wrye Bash goes belly up very easily with wizard syntax errors.

Edited September 1, 2014 by alt3rn1ty

[alt3rn1ty]

 

Ethatron seemed fairly certain that collapsing some textures is appropriate and that it decreases (vs. increases) what the graphic card needs to do.

 

I haven't seen that ( the way you put it ), but if true I would go with what Ethatron has said. Without a doubt.

 

I have read through the whole of the DDSOpt WIP topic, where Ethatron was answering Z's questions and your own questions, and cannot find anything ( hopefully I have not just missed something ) that is said by Ethatron which is worded similarly. Was it a PM ?

 

 

 

I have found the bit that I remember Ethatron saying which made me cautious of what I am providing to people in a mod ..

 

https://forum.step-project.com/topic/190-wip-ddsopt-texture-overhauls/

 

Q2.

A2. "planar (1x1) textures": The number of textures which have only one distinct value. These textures may be converted to 1x1 size without loss. Though seems not all hardware likes that. Haven't yet understood what's the real cause.

 

 

Edit : Are there any later quotes by Ethatron than the one I point out which put the information I found out of date with newer findings ?

 

My closing thought on this is :

 

There is sketchy evidence, and an expert opinion, that collapsed textures may be problematic for some peoples hardware

 

There is a solution to providing non collapsed textures in the terrain group ( see previous posts )

 

 

So what is the debate for using collapsed textures ?

( Ethatrons opinion - as Kelmych worded it - cannot be found )

 

So far its been, cant be bothered / not capable of addressing it ( tools provided to solve this ),

and / or they work on our machines so why should we care.

 

 

I know which path I took with this - But this is potentially a STEP endorsed project ( maybe even STEP will produce an Official STEP duplicate as has been mentioned ) - So its your decision collectively about the details of what goes into it ..

I have just pointed out what I think is a potential hiccup in the way its going to go.

Its just a game after all.

Edited September 1, 2014 by alt3rn1ty

[Aiyen]

alt3rn1ty:

 

I guess that single sentence made the entire point go past... so I will try again. 

 

If there are problems due to converting something to a single pixel texture... then do not do it! My point was that many shaders expect .dds textures to be power of two.. hence they should always be that to avoid any sort of conflict that we cannot identify/fix without shader source access. 

 

Also the point of the last sentence was the following. If instead of using a 1x1 you use for example a 16x16, and this cause some people to suddenly not being able to use it due to hardware... then yes they need to upgrade! We are talking about differences that are WAY less then 100kb.. and even if you have 1000 16x16 textures then the total difference is still negligible for just about ANY dedicated graphics hardware out there... and again I stress that IF it should somehow be that difference that cause a person not to be able to run the game (I highly doubt it would ever be anything but a thought up example).... then yes I think the proper response would be to go upgrade! Since at that point we are talking about hardware that is 10 or more years old. 

If you think that mentality is elitist then so be it.. but I really think one have to draw the line somewhere!  

 

Also about the struggling to install... like I said earlier. It is utterly AMAZING to me how installation of textures have suddenly become an even remotely complex issue. The fact that the unofficial patches and the whole bsa thing still haunts is just mind blowing. It is textures.... place them after and be done with it! If it is more complex then this then the installation process have in my opinion failed to deliver the plug and play experience textures should be. 

 

Finally for drivers... then yes every driver version have its own quirks and perks.. some work better some does not, and most of the time it also depends on the other hardware components in the machine. At least these days then driver installations perform a driver cleaning during their update so it is really just a reboot to try out a new driver. 

 

Hope that clarified things! :) 

[tony971]

Also about the struggling to install... like I said earlier. It is utterly AMAZING to me how installation of textures have suddenly become an even remotely complex issue. The fact that the unofficial patches and the whole bsa thing still haunts is just mind blowing. It is textures.... place them after and be done with it! If it is more complex then this then the installation process have in my opinion failed to deliver the plug and play experience textures should be. 

I've got the process as simple as it could possibly be to ensure the correct load order. MO users just have to place the BSAs wherever they want. MO handles BSA archives on its own. NMM/Wrye users are given the choice to hard-code the BSA files or load them with plugins. Hard-coding involves running a single script after installing the wizard. Plugins are to be sorted like every other plugin. Done.Also, I'm a subscriber to the "death by a million paper cuts" philosophy. If un-collapsing will help, let's do it. Are we sure it'll help?

Edited September 1, 2014 by tony971

[alt3rn1ty]

I think myself and Aiyen are slightly misunderstanding each other but basicly in agreement, myself in the first instance so I apologise if you believe I am being confrontational

 

Aiyen correct me if I am wrong :

 

You basicly agree that if 1x1 textures are going to be bad for some users because they are non-power of 2, then 16x16 would be better ?

 

In which case I agree

 

 

I only think that 1x1 textures would be bad if included at 1x1 size which is what DDSOpt is currently doing with them, I was thinking along the lines of leaving the collapsed textures at the vanilla size of 256x256, or going with similar choices to the resolutions Ethatron has chosen in his own mods mentioned earlier at either 128x128 or 64x64 - But 16x16 would work also if that in your opinion is a more optimal size for these textures ?

 

I just dont get where 16x16 came from during this conversation as a suggestion, they are certainly not going to be a problem for anyone at that size.

 

-------

 

@ Tony - "Are we sure it'll help" - Nope :), I would say its just the safest option for compatability with as many peoples setups as possible.

I think Aiyen is in agreement overall.

 

Unless Kelmych comes back with something from Ethatron which was alluded to earlier ( in which case collapsed 1x1 would be better ), then there is only evidence to suggest that as Ethatron mentions, and also Aiyen now mentions in regards to shaders and ENB, that there is potentially problems with 1x1 textures.

[Aiyen]

alt3ern1ty: Yes that is what I am trying to get at. There are no performance reasons to use 1x1.. only potential issues for not being a power of two texture. And in my experience in various game engines then most of the time then any .dds texture are always expected to be a power of two, and the shaders made accordingly. 

 

As for the most optimal size... the difference once you get below 128x128 is so silly small that it would not be noticeable. If it is just a pure color texture then going as low as possible is ofc more optimal.. but if it is a gradient or anything with detail in it then 256x256 is still preferred or you will loose too much of the gradient details. 

[tony971]

All in favor of changing the 1x1 textures to 128x128?

[Gekko64]

yay for no moar evil 1x1 textures