Server too busy

[Derlyn]

Been getting this quite often the last month or so.

Anything to worry about ?

[Dave A]

The server is getting hammered by bot crawlers from Amazon Technologies. The sheer volume of page calls is putting a strain on the server and occasionally causing the server to crash.

I have tried a few methods to slow them down and reduce server crashes, and one of those is to stop accepting new connections when the server is "overloaded" (which is the message you have shown there).

Unfortunately it isn't as simple as blocking IP addresses, which is the way I have managed abusive bot crawls in the past. Amazon Technologies have such a wide range of IP addresses available that when I block the IP addresses that are being used, they immediately cross over to new IP addresses.

[Blurock]

I've noticed that lately it is so difficult to post as the server keeps logging you out.
Typed some posts in Word and then cut and paste as it cancells the login before the post can be submitted.

[Dave A]

I've noticed that lately it is so difficult to post as the server keeps logging you out.
Typed some posts in Word and then cut and paste as it cancells the login before the post can be submitted.

May I suggest when you log in, tick the "Remember Me" box.

[AndyD]

I experienced server overload messages today and I stay logged in

[Dave A]

If it isn't Amazon, it's Alibaba Cloud Services
Looks like the storm started sometime yesterday evening.

Been wrestling with a solution from 10h00 today. Might have got on top of it now.
My apologies for the service disruptions, everyone.

[Blurock]

Time for an upgrade?

[Dave A]

I was forced to lock down the site due to what can only be described as abusive crawling practices by Amazon Web Services and Alibaba Web Services as the sheer volume of crawls was crashing the site virtually hourly.
Up until now I have managed to mitigate these attacks by blocking the worst offending IP addresses, but these two services in particular have changed tactics to crawl using tens of thousands of IP addresses.

I apologise for the inconvenience, but it hasn't been an easy problem to solve.

I am now hoping I have found the right tool to mitigate this problem and have re-opened the site. We shall see how it goes...
It may still need some tweaking.

[Derlyn]

Thank you for your dedication and hard work, Dave. It's good to have all the valuable information at our fingertips again. 👍👍👍

Much appreciated.