Potentially daft AA question

[Stazbumpa]

Hi all, I'm hoping someone can point me in the right direction.

 

Thanks to this place I have Skyrim running with STEP Extended very nicely, however I cannot get anti aliasing to work at all. So far I think I've been clever enough to work out it should be set via the ENB settings. I say clever, what I actually mean is that I followed what the guide said but now I'm stuck.

I installed ENboost as per instruction (Wrapper v302 iirc) and also edited the enblocal.ini to enable ENB stuff.  This seems to have given me access to the ENB in game overlay (Shift+Enter), and I've been trying to set the various AA options (TemporalAA et al) in the enblocal.ini but nothing is getting rid of the jagged edges. I've tried setting them and saving/applying changes with the in game overlay too, that's not working either.

I'm running the game at 3840x2160 using Nvidia down sampling, everything is set to Application Controlled in my gfx control panel and Nvidia inspector is similarly setup as per STEP instruction.  I don't have an ENB preset installed although I'm not sure I want to as all I'm after is smoothing the edges.

Incidentally, as a test I ran Skyrim from the regular Steam launcher (didn't load a save game, just the main menu) and the Skyrim dragon logo is lovely and smooth whereas from SKSE it's obvious that the logo has jagged edges.  From the Steam launcher, it seems to be listening to my gfx control panel settings for downsampling, whereas launching from SKSE via MO is not.  I have definitely got the resolution correct in SkyrimPrefs.ini.

I'm not sure if this is relevant, but my rig is i7 Ivybridge, 670GTX and 8Gb RAM. Skyrim currently doesn't budge from the 58fps limit I set.

 

I'm guessing I've missed a basic tweak or something, so any help anyone can give me would be much appreciated.

 

Thanks in advance, hope you all have a good weekend.

[TechAngel85]

Use SMAA. The ENB Guide on the wiki will guide you to installing it.

[Gunkin]

Have you tried doing it through your Graphics Control Panel? It would be Nvidia Control Panel for Nvidia, not sure what it would be called for ATI. There are several places for potential AA settings, but I would start there

[Stazbumpa]

@Gunkin:

I haven't but that's because I've been led to believe hardware AA isn't supported by ENB after v0.119 (I'm using v302). I could be wrong, but that's my understanding of it, hence everything being set to "Application Controlled" in my Nvidia control panel.  I'll have another look though.

 

@TechAngel:

I hadn't thought of that, will give it a whirl when I get in from work.  I found the instructions in the ENB guide, looks straight forward enough even for me. Should I keep my 4k downsampling resolution or set that back to 1080p?

 

Thank you both for your responses

[TechAngel85]

If you can afford the performance cost, you can keep the downsampling.

[Stazbumpa]

Well the interesting thing is that the downsampling didn't appear to make a jot of difference to the game, everything looked as it was before.  But, as I mentioned before, if I load Skyrim from the Steam launcher (Res set to 4k and MSAA + AF disabled in the Options in that launcher) the Skyrim menu screen looks like I've got AA and AF both set to max.

I've downloaded and run SMAA, I changed the d3d9.dll to d3d9_smaa.dll (I'm not using an ENB preset so is this right?) and I think it's made a difference.  While I was loading up a game it told me that Deffered Rendering was disabled in enblocal.ini and some effects wouldn't work, so being a bloke I set the parameter to true and the game CTD'd when trying to load a game. This quickly got changed back and the game loaded fine, however upon leaving the cavern I was currently in for the outside world, Skyrim told me in red letters at the top of my screen that, and I quote;

 

"bFloatPointRenderTarget=1 is not set in SkyrimPrefs.ini or you have multiple copies of this file".

 

bFloatPointRenderTarget=1 is most certainly set in SkyrimPrefs.ini and I can't find it anywhere else.  I'm now confused as hell, but (apart from the the red letters at the top of my screen) Skyrim does look a little less jagged and better for that (SMAA set to ULTRA btw).  I honestly don't think that Skyrim via SKSE is recognising the downsampling (DSR) despite the fact that in SkyrimPrefs the resolution is correct.

 

Addendum:  Am I being colossally stupid by leaving my monitor at 1080p but setting Skyrim to run at 4k DSR?

Edited February 27, 2016 by Stazbumpa

[TechAngel85]

No, that's how downsampling works but you can disable it since it's not doing anything for you. The red lettering is caused by having something set wrong in the enblocal file. Since you're not using a preset, all those settings should be turned off. Refer to the ENBoost wiki mod page for proper settings.

[Stazbumpa]

Ok, I'll revisit ENBoost and go through my settings again. Just to clarify, the bit in the SMAA install section that says;

 

"In the d3d9 folder (do not use the files from the d9d10 folder!!!) rename d3d9.dll to d3d9_smaa.dll. Non-ENB users do not need to rename this file."

 

Do I take "Non-ENB users" to mean people who are not using a preset but are using Enbseries + Enboost? If so I think I need to reinstall as I did change it from d3d9 to d3d9_smaa.  Sorry for the continued (stupid) questions, I am gradually getting my head round this though.

 

Thanks again.

[Audley]

Ok, I'll revisit ENBoost and go through my settings again. Just to clarify, the bit in the SMAA install section that says;

 

"In the d3d9 folder (do not use the files from the d9d10 folder!!!) rename d3d9.dll to d3d9_smaa.dll. Non-ENB users do not need to rename this file."

 

Do I take "Non-ENB users" to mean people who are not using a preset but are using Enbseries + Enboost? If so I think I need to reinstall as I did change it from d3d9 to d3d9_smaa.  Sorry for the continued (stupid) questions, I am gradually getting my head round this though.

 

Thanks again.

Anyone using ENB (whether it be a custom preset, the default preset, or just ENBoost by itself) needs to change the d3d9.dll to d3d9_smaa.dll in order for SMAA to be loaded correctly using the ENB proxy setting. The reason you do this is because the game cannot load more than one "d3d9.dll" file at a time because you cannot have two of the same name. By renaming the SMAA .dll you can set up a proxy in the ENBlocal.ini to load both .dll files so you can get both in-game at the same time. 

 

Also, there is no need to apologize for your questions. They are certainly not stupid. Everyone in modding starts at the beginning and only by asking questions and reading guides will you learn.

[Stazbumpa]

The reason you do this is because the game cannot load more than one "d3d9.dll" file at a time because you cannot have two of the same name. By renaming the SMAA .dll you can set up a proxy in the ENBlocal.ini to load both .dll files so you can get both in-game at the same time.

Aha, I get it now. I appreciate the reply, I've learned something today

[Stazbumpa]

I changed UsePatchSpeedhackWithoutGraphics from false to true and the red writing disappeared. I also seem to be having EdgeAA working as well as SMAA, in any case Skyrim now looks a hell of a lot nicer. Thankyou all for the replies

[Dark_wizzie]

I followed the Step guide, downloading the SMAA injector. Menus get turned white, fog in loading screen turned into milk, and graphics got changed. It's not clear enough to me... Speedhackwithoutgraphics=true still allowed for anisotrophic filtering via enb, but it doesn't look like it works for aa. Was the assumption all along that Speedhackwithoutgraphics was supposed to be off for any ENB AA to work at all, even through that one injector from mrhaandi?

[Neovalen]

SMAA is broken in the current Nvidia drivers I believe.