Betfair Service Apology
Over the last two weeks Betfair has been hit by a number of issues and outages. We would like to apologise again that we have not hit our normal levels of service. As part of that apology, we would like to explain why we have had so many outages/issues over such a small period of time. As we have grown, we have had to continuously change our infrastructure to deliver enough capacity to cope with the increasing bet volumes and site traffic. Our system has become hugely complex and now processes more transactions than all of the stock exchanges in Europe combined.
Last year we kicked off a huge investment into our core infrastructure to make sure it could cope with future demands. We have since rebuilt most of our systems from the ground up and have just a few more pieces of the jigsaw to put in place to make sure we have enough capacity for the foreseeable future.
In a business that operates all year round, 24 hours a day, there is no good time to have planned outages. However, it was essential for us to complete the work well before the Cheltenham Festival, when the best racing in the calendar causes a huge uplift in the volume of bets placed.
So what has hit us over the last few weeks? As you can imagine changes of this type take many months of planning and testing and it’s only since November we started swapping over the big things, leaving the riskiest items until last.
Our testing could not be more rigorous, using some of the most advanced technologies available from the best vendor/manufacturers around. However, no computer is without problems; no operating system is without bugs. We build our systems to be as resilient as possible but, as the world’s busiest betting exchange, we hit these vendors/manufacturers’ systems harder than other customers across the globe and we often uncover bugs in their hardware and software. These vendors and manufacturers work hand in hand with us to resolve bugs as quickly as possible.
We now know the root cause of each of the issues we’ve had recently and can stop them impacting on the service we deliver. Over the next week or so, we will put in permanent fixes and complete the final pieces of our complete system change.
So what are the benefits to you the customer?
- We will increase our capacity by a minimum of x3 and in some areas x100 – that will definitely get us through the great sporting events in store this year
- We have removed old specialist equipment and are now using universally available enterprise equipment that means speed and capacity now is much easier to scale – we should not have to do as big a project again in future
- Our API (that powers Betfair mobile and other betting applications) is now running faster than ever before
- Bet placement on the main exchange has approximately halved in time, meaning you will get better real time views of the markets
In the past, we have not communicated the exact details of what planned upgrades/outages are for. Going forward we want to improve that and where we are doing large planned pieces of work we will look to communicate them in a more open way so you can be more aware of the benefits and risks involved.
To that end we will be hosting a Q&A session via the Betfair forum on Tuesday 16th February between 18.00 and 19.30 GMT. We’d welcome questions on any topic of interest to Betfair customers.
To help us get through as many questions as possible please email them in advance to
livechat@betfair.com.This e-mail address is being protected from spam bots, you need JavaScript enabled to view it Unfortunately it is not possible for us to respond to each email individually but we will attempt to answer all questions raised via the live Q&A session.
Thank you for your patience and continued support.
LinkBack URL
About LinkBacks
Site Downtime (All sites)
Reply With Quote
Originally Posted by Andy
