Important New ATS Performance Tweak

We were hammered by real traffic this day.

Here's the tale of the day (these are graphis of active concurrent connections):

Where you can see that we were hit with some massive traffic. And if you look at the weekly graph below:

You seel that this is something very new and massive.

After watching TCP packets come into the machine, and putting a "watch" on the website log file for ATS, it became clear that this is real traffic, and not some DOS attack. People are trying to read what we have to say... by far, the post popular page is "viewthread" which shows the content of posts.

I've been forced to reconfigure this page to limit the amount of information guests can see. What this means is that guests now only see your username, and post content. They no longer see your great profile colors, backgrounds, avatars, and moods. I did this to keep these perks intact for our members, so that none of the services you enjoy are lost.

As soon as I did this, server load went down by about 30%, while traffic continued to be high. This should mean that more people will be able to come to ATS, and performance should improve. However... in my experience, what will probably happen is that even more people will come, now that performance allows more users, and we'll be in this type of performance problem in another few weeks. The price of glory.


deny ignorance.


[Edited on 17-9-2003 by William]

[Edited on 18-9-2003 by William]

William

I support the denial of more and more privileges to lurker guests, and out of a sense of commerciality, the value of withholding those benefits for application only to members is obvious.

But more importantly - why this spike, where is the traffic coming from, and is it something that will be observed consistently from this point on?

Is it still permissible to label the useless, corrupt Bush administration as the worst government in US history, that needs to be brought up on criminal charges as soon as possible, based on all the evidence in here - with all these impressionable people reading?

[Edited on 17-9-2003 by MaskedAvatar]

Hey did you tell the guests that they are not seeing all the goodies?

I don't know where the spike came from... we'll know better by examining the compiled traffic reports in the morning. But none of the observed traffic carried the typical hallmarks of a denial-of-service attack, packet-flood, or other server exploit.

i wonder where all these hits came from, all of a sudden. maybe another really big website is linking to ATS.

Keep in mind that our average concurrent user connections at any given time of 50 is very high on its own!

Multiply your concurrent user connection of 50 by 50, allow nine months, calculate what computing power you need to manage that, then determine whether fight or flight is the appropriate sanity-preserving response.

Thats a lot of traffic...but it still seems to be a bit up and down,one minuite fast next slow...

Header data processed in 0.107 seconds
Page processed in 18.079 seconds

William, I suggest you make further contingency plans.
You might want to prepare a version that outputs posts in raw html tables (or maybe html para gaps) and with raw links at top: no graphics, no color, no time stamps, no database queries other than poster text. Then, when the day comes, you can use and still have the site up.

If anything this would serve as a control to comepare optimziations against.

In fact, it might be interesting to be able to "purchase" site layouts. I would opt for all text and no user names, points or anything else "extraneous". A poster and the World conversing on ATS, how beautiful. You could increase the point accumulating modifer if the user chose a less demanding layout (judged by kb).

Another suggestion, put a link at top called "logout" or "I'm leaving" that randomly ejects the user to one of the google links. Some may say its silly, they could just click on a google link, but if a 100 people use it, it'll pay off.

[Edited on 18-9-2003 by ktprktpr]

A wholesale redesign of XMB is a bit out of the question (for now).

You'd be surprised how little the "extras" cost in the way of server performance... every little bit helps... but easily 80% of all database load (maybe more) is occupied by the relatively simple tasks of querying the very large u2u, posts and threads tables. The rest of the tables used to store the extras are very efficient and query quickly.

I was in chat, and suddenly on a reload, I got a 404 error, and now that's all I get. could that be related to the server tweaks?

I just gave chat a bullet in the head.

Well, its obvious that the 'google' adventure is paying off in traffic. I'm just assuming here, though.

I'm still have some major slowage on getting in and accessing topics and when posting.
Other than that, I'm glad to see that ATS is growing in "leaps and bounds."

regards
seekerof

Originally posted by William
I just gave chat a bullet in the head.

is it just temporary?

Hmm... there's gotta be some functions we just don't really need, or that could be farmed out.

u2u - link to your email account.
search - google search on ATS cache. To be pretty have a input box the wraps up query and hands off to google. (might have to wait a day to be able to search recent posts)
Member list- don't need
Today's Post - Don't need (hehehe, yeah right)
Complain - don't need; part of u2u
Upload - defintely don't need; give link to free porn server
ATS store - perhaps. Question load of pictures.
Settings - No problem.
Buddies - don't need.
Chat - defintely don't need; give irc server and windows patches

well, all the above is imho.

Originally posted by ktprktpr
Chat - defintely don't need; give irc server and windows patches

well, all the above is imho.

I'd love an ATS IRC channel, but William already vetoed that idea quite a while back.

Dude... I appreciate the intent to help... but don't kill yourself pulling your hair out... I've already pulled out all of mine.

Google search: not an option for a dynamic site
Member list: 0.5% of all traffic, no big deal
Complain: hardly used, less intensive that the complaint posts that it replaced!
Upload: not a server load
Buddies/Settings: blah, nothing.

90% of all the traffic on this machine is in the forum directory of ATS... and 20% of all that traffic is the viewthread page (26,000 hits a day)... the second most loaded page is the forum home (less than half of the viewthread traffic).

It's the threads and posts databases... they're designed inefficiently, and now we're paying the price.

okay okay I was just throwing ideas that came to me. I still have a lot of hair left anyway

wow, that's some huge traffic.

At first sight, it really looks like an attack.
The peaks are freakin' high.

Hey i didnt know we had graphs.