Auction Request Downtime: What We’ve Done to Make Sure it Doesn’t Happen Again

As a follow-up to our recent blog post regarding steps we’ve considered to avoid the downtimes & outages we’ve experienced, I’d like to update you on where we currently stand.

This past weekend, we switched the Auction Request service over to a new dedicated server. This server is faster than the old one, and has additional resources to manage increased load. While the load on the old server did not cause any of our outages, we felt it was a good investment to make now, with an eye towards future expansion.

This past month, several of our high-throughput clients were being mysteriously blocked from accessing our service. While this took us a number of weeks to diagnose, we did eventually discover that our old server’s hosting company was behind the blocks, as they saw the high-throughput API requests as a DoS (Denial of Service) attack. In order to mitigate this, we’re now incorporating Cloudflare’s proxy service. It’s important to note that Cloudflare does not enable us to survive a server outage at our host; since we provide an API service that interfaces with eBay, not a static website, viewing a cached copy of Auction Request doesn’t work. Cloudflare does enable us to engage their more sophisticated firewall to survive outside attacks, as well as a DNS failover service that can instantly redirect to another IP address should our current IP address not resolve.

What this means in layman’s terms: we’re going to utilize our old server as a backup server. Should Cloudflare detect that our current (new) server has gone down, they will automatically switch the service over to the backup. Theoretically, this will occur relatively quickly (within a few minutes or less), and again theoretically, should largely eliminate any further downtime. The odds of both servers being down at the same moment is mathematically pretty low.

The backup-redirection mechanics have not been enabled yet, but will be in the coming week, once we’re satisfied that the new server setup is performing optimally.

We thank you all for your continued patience as we overcome these technical challenges. Your business is very important to us, and we continue to make investments to provide the fast, reliable, professional product you deserve.

Leave a Reply

Your email address will not be published. Required fields are marked *