Thread split: Server Slowness

Elros · (This post was last modified: 10-28-2015, 01:37 PM by Elros.)

Mod edit: the following thread was split from its beginning with this post.

(10-19-2015, 09:17 PM)IndigoGeminiWolf Wrote: Nice work, but sometimes my posts take over a minute to post new ones.

Noticed that too.

The_Tired_Philosopher · (This post was last modified: 10-19-2015, 11:48 PM by The_Tired_Philosopher.)

Its a server response issue i think, I just click preview post again after its loaded for over 10-15 seconds. Or I open a new tab if I'm doing a thread to see if it already posted without saying so.

I noticed it too, thought it was just me in my work building until.I noticed others double posting a lot

Edit: i noticed one of the OP links to another OP with a grammar error in it, I'll point it out when I find it again.

*Bring4th_Jade* · 10-20-2015, 11:23 AM,

Steve is aware of the lag time, he's looking into it.

native · 10-23-2015, 10:32 AM,

(10-19-2015, 09:17 PM)IndigoGeminiWolf Wrote: Nice work, but sometimes my posts take over a minute to post new ones.

This was an issue before. If you get that lag, just stop the browser and click the link at the top of the thread you're in and it should load normally with your post already there.

*Bring4th_Jade* · 10-27-2015, 10:50 AM,

As an update on the lag...

Steve says it's likely that it's because we're over 100,000 posts that the slowdown is occurring (almost 200k!) so he's working on getting us some big-boy pants. You may be experiencing both times when it's working fine and times when it fake-lags. Apparently it is affecting the store, too. I think he's got some more work to do before he fixes what he believes is not working optimally, so thanks everyone for hanging in there. Smile

The_Tired_Philosopher · 10-28-2015, 07:39 AM,

This lag makes going through each page of threads 5 times slower Dodgy

Good catalyst for patience...

IndigoGeminiWolf · 10-28-2015, 03:48 PM,

I don't experience slowness when going through posts. Only when posting a new thread or replying to a thread. But editing my post is quick.

rva_jeremy · (This post was last modified: 10-28-2015, 05:58 PM by rva_jeremy.)

Heh, IGW, I was literally just going to post that same observation. As somebody who works with relational DBs like MySQL for a living, I can't think of any reason why an insert would be slow but an update quick, except that there's some sort of problem with indexing. For what it's worth, @bring4th_steve I found this article: https://www.percona.com/blog/2006/06/09/why-mysql-could-be-slow-with-large-tables/

Probably not simply no help but also annoying for me to post that; sorry. I'm at your disposal man (and I'm in Northern VA at this very moment!). Anything you can say about the characteristics of the server/VPS you're using? Is this possibly a memory issue (i.e. the working set not fitting in memory)?

IndigoGeminiWolf · 10-28-2015, 07:53 PM,

I've worked with MySQL a little bit (set up a dozen or so databases over the years). Also did 10 years in Oracle, and about 4 years in Sql Server. I just used clients, but I programmed Excel macros that could retrieve data from a database and fill in cells.

We worked a lot with Excel, and some ColdFusion.

I didn't really get into the internals though.

IndigoGeminiWolf · (This post was last modified: 10-28-2015, 08:56 PM by IndigoGeminiWolf.)

(10-28-2015, 05:56 PM)jeremy6d Wrote: I can't think of any reason why an insert would be slow but an update quick, except that there's some sort of problem with indexing.

The insert happens quicker than it shows. If you stop the browser after a few moments and then reload the page, the record will already be created.

I would recommend copying the text before doing so in case it doesn't post.

*Bring4th_Steve* · 10-29-2015, 12:18 AM,

(10-28-2015, 05:56 PM)jeremy6d Wrote: Heh, IGW, I was literally just going to post that same observation. As somebody who works with relational DBs like MySQL for a living, I can't think of any reason why an insert would be slow but an update quick, except that there's some sort of problem with indexing. For what it's worth, @bring4th_steve I found this article: https://www.percona.com/blog/2006/06/09/why-mysql-could-be-slow-with-large-tables/

Probably not simply no help but also annoying for me to post that; sorry. I'm at your disposal man (and I'm in Northern VA at this very moment!). Anything you can say about the characteristics of the server/VPS you're using? Is this possibly a memory issue (i.e. the working set not fitting in memory)?

Hey there, thanks to everyone for their input on the performance issues. Jeremy, I owe you an apology for not getting back to you in a timely manner... Regarding your JSON output email, I've already been working on a PHP export solution that supports unicode encoding, but I hadn't finished with it and was hoping to do so before I could graciously accept or decline further collaboration on that piece. So I promise to get to a more detailed response to you, but I'm still hacking on a couple of formatting tasks at the moment (trying to wrap up a json-to-MySQL export via PHP).

Ok, let me jump in here for a sec regarding databases. The problem actually has nothing to do with our MySQL resources/config, as far as I can tell. I've profiled and tuned the database a while back when we suffered other lag events, and thankfully it has been running fine ever since (with the occasional hiccup during garbage collection when MyBB tries to do something at the same time the server runs Cron jobs).

If you crack open firebug or chrome dev JS console while experiencing page submit lag, you can see that the issue is related to the page load that comes after the POST (page submit/action). For some reason, especially on the pm.php page, the reload times are taking up to 30s, even though the data from the form Inserts/Updates immediately. And, 30s happens to be the server timeout for attempting to execute a PHP script. So my bets are on that something weird is happening with the request, as the TTFB is precisely where the lag is. I'm looking to see how/if any of the pages are being cached, because that could potentially eliminate the problem right away. MyBB does not cache certain files, and pm.php is one of them. Just a matter of going through some tests and finding where the round trip bottleneck is.

One cool thing to mention is that Bring4th qualifies as a "large board" with posts over 100,000. We have almost double that, and so we're currently waiting to be accepted into a special dev group where big board operators can discuss some of the scaling issues that are the result of becoming too big for one's britches. Hopefully there will be some good insights once we are welcomed in (fingers crossed!).

The VPS is pretty beefy, and after having restarted services and monitoring through top while the hanging takes place, there's nothing evident showing up in load or memory usage. I don't recall the stats, but the drives are SSD, it's 4 proc with 8GB of RAM, and we're only about half way full on the memory during peak utilization.

So as I get the chance, I'm going to continue exploring these haunches, and should have something interesting to report back, soon.
Steve

*Bring4th_Jade* · 10-29-2015, 10:32 AM,

Thanks Steve, love and light to you in your ongoing battles with our forum "greeter". Tongue

rva_jeremy · 10-29-2015, 10:43 AM,

Awesome write-up, thanks Steve! That makes sense--it's too bad that a so-called "quick post" that creates the spinner upon submission even requires a full page request at all, since the spinner almost implies an AJAX update. The server definitely sounds beefy enough, that's for sure!

Regarding the Unicode issues, I hear you. If you've already got importing under control, then simply ignore my script and its output. Thanks for sharing some of the details there.