RESOLVED Wiki pages are very slow

[JUNKdeLUXE]

I've been doing the Clear and Present Danger install during the last 2 weeks, and they (and wiki pages it link to) have been progressively had more and more load time.

 

The forums on the other hand still seem fine. 

 

Not sure what the problem is.

[TechAngel85]

I just logged into the AdminCP and the server load is currently in the green for the forums. The wiki seems to be okay at the moment too. Possibly it was just a spike on the server for a while.

[Darth_mathias]

everything seems OK now

[Greg]

Yeah, the whole forum went down just as I was posting a reply. Seems to be working now, although I'm still getting a gateway timeout on the DynDOLOD topic. When I try to go to first unread post in this topic it seems as if the rest of the forum slows down as well.

[TechAngel85]

Yeah, the whole forum went down just as I was posting a reply. Seems to be working now, although I'm still getting a gateway timeout on the DynDOLOD topic. When I try to go to first unread post in this topic it seems as if the rest of the forum slows down as well.

I can confirm this behavior and it's only the last page which will not load. That is the largest topic we've every had. I've asked Sheson if he wants a support forum similar to MO, but he likes the single topic. I think a support forum will be better suited so users can better search for previous support quests. May have to make an executive decision there if the forum is being overwhelmed by that single topic. For now, I'll try splitting it out from the last update post to see if that fixes it.

[LordOfLA]

If the forums are running on mysql you probably don't want long threads unless you have a dedicated db cluster with lots of ram caching going on.

 

It's not the MySQL is bad, just that most web based forums aren't written with optimal DB querying.

[TechAngel85]

I've closed that topic and opened a new one by splitting off several pages of posts and moving the rest. It started loading properly on the first split so that was likely the issue. That topic was approaching 5000 posts. I've offered Sheson the support forum again.

[Darth_mathias]

i am still regularly getting 504 timeouts while trying to edit my SRLE extended LOTD guide.

[TechAngel85]

I got in touch with s4n. He was able to take a look. Seems like there has been a spike in requests the last couple days. He did some tuning to hopefully relieve some of the 504s. He will likely keep an eye on it for a few days. We have to be careful of remaining at high loads on the server for too long or we risk our host pulling the plug for a while; or worse.

[z929669]

Yes, we are still having issues related to limiting resource use on the server so that we don't risk our hosting relations. s4n has made some tweaks as a possible short-term fix. There must be an issue with thread size for forums, but the real issues are more likely the wiki, which is muuuuch larger than the forums in terms of storage by an order of magnitude at least.

 

However, LordOfLA makes a good point about DB query efficiency, and I am not sure if forums and wiki DBs differ in terms of optimization. Breaking large, popular threads into chunks might help, and we have some doozies like Skyrim Revisited, which is getting the largest number of topic views this year by far and second only to MO support in 2015. Mods, MO Support, General, and Sheson's DynDOLOD are other large and potentially high-trafic forums with hot topics that should be archived. Sheson's thread has not had nearly as much traffic since early 2015 though (probably due to DynDOLOD wiki supoort).

[TechAngel85]

Yes, we are still having issues related to limiting resource use on the server so that we don't risk our hosting relations. s4n has made some tweaks as a possible short-term fix. There must be an issue with thread size for forums, but the real issues are more likely the wiki, which is muuuuch larger than the forums in terms of storage by an order of magnitude at least.

 

However, LordOfLA makes a good point about DB query efficiency, and I am not sure if forums and wiki DBs differ in terms of optimization. Breaking large, popular threads into chunks might help, and we have some doozies like Skyrim Revisited, which is getting the largest number of topic views this year by far and second only to MO support in 2015. Mods, MO Support, General, and Sheson's DynDOLOD are other large and potentially high-trafic topics that should be archived. Sheson's thread has not had nearly as much traffic since early 2015 though (probably due to DynDOLOD wiki supoort).

You probably missed it, but I've created a support forum for Sheson at the top level and broken his topic up into multiple topics. He's now running that forum as a moderator as it'll be the place for support for his tool.

[z929669]

Yeah, I saw that and forgot about it when I wrote. Good move. We should start breaking up the others though and implement a post limit per topic ... maybe archiving the rest into subforums within  adjacent to their own Topic forums. The big topics with lots of page views, I mean that are under the forums I mentioned.

[Darth_mathias]

Yes, we are still having issues related to limiting resource use on the server so that we don't risk our hosting relations. s4n has made some tweaks as a possible short-term fix. There must be an issue with thread size for forums, but the real issues are more likely the wiki, which is muuuuch larger than the forums in terms of storage by an order of magnitude at least.

 

However, LordOfLA makes a good point about DB query efficiency, and I am not sure if forums and wiki DBs differ in terms of optimization. Breaking large, popular threads into chunks might help, and we have some doozies like Skyrim Revisited, which is getting the largest number of topic views this year by far and second only to MO support in 2015. Mods, MO Support, General, and Sheson's DynDOLOD are other large and potentially high-trafic forums with hot topics that should be archived. Sheson's thread has not had nearly as much traffic since early 2015 though (probably due to DynDOLOD wiki supoort).

I think we can attribute the higher number of topics for SRLE because of 3 new mod guide that hit there last year:

1st my SRLE Extended guide.

2nd my SRLE Extended Legacy of the dragonborn guide.

3rd paul666root brand new SRLE Legacy of the Dragonborn Requiem guide.

 

These are hugely popular (mine was a big surprise to me).

 

Now my 2 original SRLE Extended forums and the SRLE Extended support forum have both been closed at my request cos they weren't being used anymore and i was longer supporting that guide.

 

my SRLE Extended LOTD guide and forums are still going strong even thou I have now announced my semi-retirement from maintaining the guide.  this due the the birth of my twins back in August and lack of time to do so.

 

Paul Requiem guide is going strong and guide is still under construction.

 

So you might want to breakup both the SRLE Extended: Legacy of The Dragonborn and the SRLE:Legacy of The Drgaonborn Reqtified Threads into there own sub forums or something i don't know.

[stoppingby4now]

The recent issue with the forum revealing the database problems has nothing to do with thread size. It was a configuration on MySQL that was causing out of memory errors (I over-allocated). IPB is somewhat decent on its queries, using LIMIT's on a thread view to reduce the number of rows returned, and it caches frequented views. I currently don't see an issue with the forum, and breaking apart threads right now would only induce a placebo affect.

 

The Wiki is the real beast currently.

[Darth_mathias]

been getting alot of 503 and 504 errors from the wiki pages today forum are OK thou.

[paul666root]

it's been very bad lately. Around this time of day i can't even access the pages and it's happening for some time now not only today. Few months ago it was usually around 1-2 hours per day when the wiki stopped working but now i got around 5-6 hours when it throws connection error.