Reseller/shared hosting server migrations - 04-Jun-2016

Announcements concerning Networking & Related News, Planned Outages, Anything which may affect your services.

Moderator: Admins

Post Reply
porcupine
Site Admin
Posts: 703
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

Reseller/shared hosting server migrations - 04-Jun-2016

Post by porcupine »

Hi Folks,

Please be advised of the following maintenance window:

Maintenance Window: Saturday June 4th, 9:00pm - Sunday June 5th, 6:00am Eastern Time
Facility: Tor1 - 151 Front street west
Affected Device(s): www2.pcdc.net, www3.pcdc.net, www4.pcdc.net
Affected Customers: Shared/Reseller hosting customers
Service Impacting: Yes - Full customer migration to new hardware

Details:
We will be upgrading, and consolidating the www2.pcdc.net, www3.pcdc.net, and www4.pcdc.net servers into a single, considerably more powerful machine. The upgraded machine will include a base configuration of 40 cores (HT), 128GB of memory, and 2 RAID1 arrays (one pair of disks for general storage of the homedirs, one pair of SSD's for database/mail queues/logs/general OS/etc.). The new server will be running CentOS 7, Apache 2.4, PHP5.6, and the latest CPanel/WHM versions, and will have plenty of room for upgrades (both in drives, RAM, etc.) if that should prove necessary.

Since the vast majority of customers (all of the reseller customers, and a number of the shared hosting customers) have dedicated IP addresses (and/or dedicated shared IP addresses), we will be moving the accounts on a per customer basis. Once a customers accounts have been moved to the new server, their IP address(es) will be removed from the old server, and added to the new server, to ensure each customer experiences the least downtime possible, without any changes necessary on the customers part. In general, aside from updating your records (to reflect the new server hostname, for customers whose original/source servers were either www3.pcdc.met, or www4.pcdc.net), little to no customer intervention should be required throughout this process.

Possible complications customers should be aware of/make preparations regarding:
- Customers moving from the www3.pcdc.net, and www4.pcdc.net servers, should update their records to reflect their new server hostname as www2.pcdc.net (the hostname www3.pcdc.net will be immediately re-used for the consolidation of www5, www6, and www7 servers, so this is important to note!).
- There will be a short window where received emails may be lost (IE: received to the old server, as the customer in question is being moved to the new server/has just moved). We may elect to simply shut down SMTP on the source servers for the duration of the move to prevent this from occurring, though complications regarding this may still occur.
- Customers will want to ensure their sites/scripts/etc. are compatible with PHP 5.6, as the new servers will be running a newer PHP version (see PHP migration notes (v5.5.xx to 5.6.xx) - http://php.net/manual/en/migration56.php )
- Customers who have DNS zones for domains that are not hosted in the reseller servers whatsoever (IE: using the DNS servers to host records for entirely external domains) may need to re-create the zone files after the migration (as WHM has traditionally failed to allocate DNS zones to reseller users properly when there is no hosted account, meaning the additional DNS zones won't get copied during the migration).
- Customers existing bandwidth/stats may not migrate over cleanly, and may be lost during the upgrade (issues with some existing RRD files being corrupt/incomplete were detected with this during several of our dry-run pre-migration tests).
- Various system services may be disabled for the duration of this migration, to help protect against incidents/changes/etc. occurring during the migration itself.
- The new server will be utilizing a new CPanel configuration to send email out of each resellers IP address (instead of the main server IP address), in an effort to ensure that SPAM originating from one account does not impact other accounts. As such, SPF records will be added/updated on all accounts (to include the reseller netblock as a whole). Customers who require custom SPF records (IE: push email out through third party providers) should setup/modify the newly added SPF records to reflect this as they deem necessary (if they do not already have SPF records setup for their domain, as the entries we add will not account for third party mail providers).
- Customers with very old databases may find their passwords changed, in need of being reset -- Some very old databases had passwords stored in insecure formats which cannot be retrieved by MySQL v5.6, and are set to random strings (awaiting user reset) by the CPanel account transfer tool.
- Customers using outdated themes (IE: "Monsoon", and "Bluelagoon") will find their themes have been reset to the current ("Paper Lantern") theme. The older themes have been outdated for many years, and have not been available on new CPanel builds for several years.

All in all, we will endeavour to keep the outages on a per account basis, and as short as possible. We will be rebooting the new server several times throughout the process to effect changes, and have opened a considerable window to allow time for all individual transfers to complete, etc. -- This does not mean individual users will be offline for that amount of time, per customer outages for core services (Apache, MySQL, etc.) should be a few hits lasting several minutes each if everything goes according to plan.

Regards,
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
Post Reply