Severity 2 (High)

This issue is now resolved and all affected virtual machines are now coming back online.

We are still investigating the cause of the issue but please accept our apologies for any inconvenience caused.

We are currently experiencing an unexpected issue with one of our XenServer hosts. This is affecting a number of client virtual machines. We are currently investigating the problem and, where possible, moving client machines to other hosts.

We will post updates here when we have more information. Please accept our apologies for any inconvenience.

We are currently experiencing an issue with one of our XenServer hosts that is affecting a number of customer virtual machines.

We are working to resolve the issue at the moment. Please accept our apologies for any inconvenience.

Update 07:13 All customer virtual machines are now back online. We are looking into the cause of the unexpected reboot of the host.

Unfortuantely we are experiencing an issue with the XenServer host that has been upgraded relating to the Flash storage used to boot the host, we are currently working to resolve this.

Most customer machines remain online but the small number of servers that were shut down may experience downtime beyond the two hour maintenance window whilst we resolve the issue.

Please accept our apologies for any inconvenience caused.

We have scheduled an upgrade of the Junos operating system on our main London firewall cluster. This upgrade is essential to ensure long-term availability and stability of the network. We expect minimal interruption to traffic not lasting more than a few seconds at a time, but the network is at risk for the entire window.

We apologise for any inconvenience caused.

We have received a custom build of the Junos software that our firewalls run from Juniper which should correct the recent firewall issues we have experienced at our primary site.

We are planning to install this next Wednesday evening, and will aim to keep network disruption to a minimum. The network is at risk for the entire two hour window, but we anticipate the actual disruption will be under 10 seconds.

We apologise for any inconvenience caused by this essential maintenance.

The upgrade has been completed and both nodes of our firewall cluster are now running the latest version of the operating system.

In addition the previously failed cluster member has been re-integrated.

In conjunction with our vendor, Juniper Networks, we have identified a requirement to upgrade the operating system on our firewall cluster as soon as possible to ensure the continued stability of services.

This evening we will be re-integrating a failed cluster member into our firewall cluster and performing the operating system upgrade. This work will be done over a period of several hours and the network will be at risk during this time, but we hope any downtime will be restricted to only a few minutes.

We apologise for any inconvenience caused by this work.

We are currently performing emergency maintenance on our XenServer pool as it has become unstable. We are upgrading each host to the latest version of the virtualisation software but unfortuantely this requires a reboot to complete and live migration of virtual machines is currently unavailable. Downtime for each virtual server will be kept to a minimum.

We apologise for any inconvenience this emergency maintenance may cause.

We will be installing new networking equipment (switches and firewalls) at our primary data centre in London on the morning of Sunday 10th January 2010. The new equipment will provide improved performance, redundancy and scalability. We will also be upgrading the software for our NetApp storage area network at the same time.

All hosting customers will be affected during this maintenance window and all virtual servers will be shut down and restarted after the work is complete.

We apologise for any inconvenience caused.