Google has posted an official apology for the downtime that affected its services on Friday, as well as providing some details about the bug that caused havoc across Gmail, Google+, Calendar and Google Docs. The company says “most Google users” experienced issues, and for 1 in 10 the connectivity problems lasted longer than 30 minutes.
“Whether the effect was brief or lasted the better part of an hour, please accept our apologies,” wrote Vice President of Engineering Ben Treynor. “We strive to make all of Google’s services available and fast for you, all the time, and we missed the mark today. The issue has been resolved, and we’re now focused on correcting the bug that caused the outage, as well as putting more checks and monitors in place to ensure that this kind of problem doesn’t happen again.”
At 10.55am PST one of Google’s internal systems threw out an incorrect configuration, which then led to a variety of products and services getting in a muddle about how to behave. Incoming requests were ignored, error messages were thrown up, and Google users across the world began to panic. By 11.14am the source of the problem had righted itself, and by 11.30am the correct configuration file had spread across all of the affected systems.
Google is promising to upgrade its internal technology to improve error-checking audits and to spot problems more quickly when they crop up. If you want to read the post in full, you can find it on the Official Google Blog.
