Upcoming Critical Security Patching & Maintenance
Incident Report for SiteHost
Resolved
At this stage we have contacted all managed customers and completed scheduled maintenance across our fleet. As patches continue to be released – Intel even released new ones yesterday (https://newsroom.intel.com/news/latest-intel-security-news-updated-firmware-available/) – we will be regularly reviewing the patches for any that may need to be applied to our infrastructure and will be in touch if further maintenance is required. We'll also be reaching out directly to a handful of customers in the near future who will need to take additional steps to ensure they are safe (or have managed the risk) of these attack vectors.

Thank you for your understanding during the handling of these incidents, it's certainly been an interesting start to the year for cloud providers around the world, and we hope we don't have to deal with anything of this complexity and magnitude for a long time!
Posted Feb 22, 2018 - 17:20 NZDT
Update
We have now sent emails to our managed Windows Server customers with their scheduled maintenance details. We have also prepared blog posts outlining our recommended steps for Windows - https://sitehost.nz/blog/2018/meltdown-how-to-protect-your-windows-server - and Linux - https://sitehost.nz/blog/2018/meltdown-how-to-protect-your-linux-server - for those customers who manage their own servers. Preparing a Windows Server 2008 image without a sizeable performance impact is proving difficult and while we continue to work on this, we would recommend looking at upgrading to a newer Windows Server version where possible.
Posted Jan 24, 2018 - 18:05 NZDT
Update
We have now completed the rollout of updated Windows Server 2012 and Windows Server 2016 images, so any new provisions will be protected out of the box. Work continues on Windows Server 2008 with new images in final testing now, ahead of a targeted rollout tomorrow.

It is worth noting that the patches do have a performance impact for Windows servers. In our testing, performance remains mostly unchanged for Windows Server 2012 with the exception of storage, with a performance drop of 17% compared to the unpatched version. The performance impact on Windows Server 2016 is lower than that of Windows Server 2012, with again the only noticeable difference relating to storage - a 6% performance drop compared to before the patch.
Posted Jan 22, 2018 - 19:30 NZDT
Update
Another quick update on Meltdown and Spectre - we have rolled out images for Ubuntu, Debian and CentOS 7 which include updated kernels relevant to the vulnerabilites. We are still waiting on an updated CentOS 6 image but for other distributions, any new provisions will be updated out of the box. It is also worth noting that our testing has shown that within our infrastructure individual Linux virtual servers are not vulnerable to the Meltdown exploit. With our Windows images, Server 2012 and 2016 are in final internal testing with a view to being rolled out on Monday next week at which time we will also post more details about the impact of these patches.

For existing managed servers, we'll be in contact with customers directly on Tuesday next week with regard to our maintenance plans. We'll do our best to schedule these maintenances for off-peak times however the impact of these vulnerabilities means we may need to act quickly to ensure security for our customers.

Unmanaged customers: we recommend that you update as soon as you can, based on your OS vendor's recommendations. We've noted problems with the latest (at the time of writing) CentOS 6 kernels and don't recommend upgrading to it just yet, but outside of that we're not aware of any issues with the updates currently available. We will be preparing some knowledge base articles outlining the steps you need to take to be secure based on what we've learned over the last two weeks and hope to have those available for you early next week.

We will continue to update our status page with detail as more news becomes available.
Posted Jan 19, 2018 - 17:26 NZDT
Update
It’s time for another update about Meltdown and Spectre. Like almost every cloud service provider we have been learning more about these vulnerabilities every day as more information comes to light and more patches are made available.

One of the unique aspects of Meltdown and Spectre is that different hardware, virtualisation software, and guest operating systems are vulnerable in different ways. That means there’s no silver bullet across our entire fleet or product range.

That being said, we are currently testing updated images for Ubuntu, Debian, CentOS, CoreOS and Windows. Our goal is to release these early/mid next week once we’re confident in their stability and understand any potential performance impact they may have. From there we’ll be looking at if and when we need to apply these updates to managed servers.

If your server is unmanaged but a patch is available you can update to the latest kernel or OS version when ready by following the vendors documentation. Please do note that we cannot vouch for the stability or potential performance impact of any of these patches.

Patching customer servers is only part of the picture though. The underlying hardware nodes also need patching which is where our hardware and virtualisation vendors come in. In most cases we are still waiting on patches, but we are using this time to investigate our options to ensure our infrastructure is safe and secure in as timely fashion as possible while having minimal impact to reliability and performance.

We’ll post another update as soon as we have more news.
Posted Jan 12, 2018 - 16:18 NZDT
Update
Some key upstream vendors are still working on patches for these flaws and we're continuing to work with them. We expect to have more news later this week.
Posted Jan 09, 2018 - 10:43 NZDT
Investigating
Due to two severe vulnerabilities released today that impact almost all computers and cloud providers there will be some short notice critical maintenance that will require downtime in the coming days. We are currently investigating patches and as soon as we have them verified and tested we will confirm the exact timing of the outage windows on this site and via email. At this stage we believe this very likely affects all customers, but we believe our Dedicated and Private Cloud customers to be at a much lower risk.

– https://meltdownattack.com/
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2017-5753
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2017-5715
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=2017-5754

We will provide updates as we learn more.
Posted Jan 04, 2018 - 12:18 NZDT