Well, we don't really know why- but at least we've identified that you are all not crazy or lieing (my first assumption- naturally)
Here's a graph that Lucky put together. It represents the load on our two web servers. Something is making them spike almost hourly. We thought it might have been some scheduled processes- but that doesn't seem to be the case. We're still investigating and trying to find the problem- but the first step is admitting we have a problem. We have a problem.
Here's a graph that Lucky put together. It represents the load on our two web servers. Something is making them spike almost hourly. We thought it might have been some scheduled processes- but that doesn't seem to be the case. We're still investigating and trying to find the problem- but the first step is admitting we have a problem. We have a problem.
Stumble This
No genuine answers though. Do the spikes link to some freak net traffic spikes or is it just their own load that has this pattern?
A quick search for "server spike automated" over at google brought up this:
http://www.xav.com/scripts/search/help/1177.html
Could it be that?
Putting a 15 second cooldown between searches from the same IP is a good idea regardless. Someone could easily DDoS by spamming search queries. And make sure you disallow the search page in your robots.txt.
It could be that people go to videosift first thing after the end of whatever TV show they were watching that ended on the hour.
Like a server version of the superbowl water pressure myth?
I think you're using php, so you could look into a code accelerator like eAccelerator, though don't hold me accountable if your Xeon machines start spewing sparks.
I hope you find some of this helpful at least.