At approximately 16:15 on Thursday the 3rd of May one of our core switches at our Auckland City DC failed resulting in packet forwarding being terminated abruptly. Due to the importance of these devices we deploy them in redundant pairs, and immediately the second core switch began taking over with services being reachable again at 16:18 approximately 3 minutes later.
We are currently working with our suppliers to identify and undertake any further actions to prevent this from occurring in the future. As of writing this, the incident has not reoccurred and the device has been put back into production.
While incidents like this are unfortunate they do happen from time to time, and in this case we're happy with how our automated systems recovered without any engineer action.
If you would like to discuss anything further around this incident please do not hesitate to get in touch.