As emailed out, I thought I'd put a copy of this on the forum also for the masses, and reference.:
As noted in our original forum thread from Mid-February, we are finally getting the opportunity to relocate our equipment from the 704 suite, to the brand new 707 suite in 151 Front Street. The original thread can be found here:
viewtopic.php?t=171
Changes/Benefits:
- We will have nearly double the current space to house current equipment and future expansions, with additional adjacent space for further expansion
- Only power run under raised flooring, with all cat5 and fiber being run with appropriate ladder works, and fiber troughs, resulting in signifigantly improved airflow below the raised flooring
- All servers being moved from existing 2 post racks to rock solid 4 post racks
- Existing cable management being removed, cable trunks for patch panels run on overhead ladders, and D-rings down the back of the racks for cable management between the patch panels and the servers
- 3 new HVAC's producing signifigantly more air pressure under the raised flooring, with approximately 75 tons of HVAC capacity for a less then 5000 sqft suite (~200 BTU's of cooling per square foot)
- ** New Uplink ** - Adding Teleglobe to our BGP4 mix after the move has completed and everything has settled in
- ** New Router ** - We have acquired a new Cisco 7507 router with Dual Route Switch Process 4's (RSP4), and Dual AC Power Supplies, connectivity from NAC and Teleglobe will run through this new unit increasing capacity, and routing power to handle future expansions, and better handle Denial of Service attacks
- Custom made rack supports/wings/ears will support the end of (some) servers which do not have rail kits
- All colocation racks are 28" deep, following industry standards, Dell, Compaq, IBM, etc. rail kits should fit perfectly into the new racks, dedicated racks will remain 24" deep (the depth of the typical rackmount chassis which we build our dedicated servers into).
- KVM over IP setup is being implamented for all Dedicated Servers, and some Colocated servers, this will be available to staff only, but will result in signifigantly faster remote hands technical support.
- We will also be removing all stand alone server shelves from the facility, all colocation customers who are currently located on those shelves will be placed on rack-mountable shelves. Stand alone shelves will only be used for storage of spare parts, and as console carts.
- Colocation customers who have rackmount rail kits, or special 4 post mounting kits for their equipment (or wish to purchase them from their vendor), may supply this to us up to 1 week in advance, and we'll install it on their equipment in the new space free of charge. Rails must fit into the standard 28" deep and 19" wide 4 post racks.
Impact:
- All servers will be provided with new MRTG (switch) and APC information after the move is complete.
- APC Masterswitches may not be available during the move, same with MRTG graphs, reboots may be handled VIA the pager if you do not have functional APC information at any time during the migration.
- Customers in Private VLAN's may be temporarily moved to the default VLAN (VLAN 1) to assist in seamless migration, thus will temporarily receive more inbound "junk" broadcast traffic for the duration of the move.
- Customers will notice small cutovers as we migrate other network equipment, including our core 5513 Cisco switch, our routers, etc. Customers will not be notified for any move related downtimes that are expected to take less then 30 seconds (eg. cable swaps).
- To help ensure that the move goes smoothly, some customers may find themselves moving earlier then their expected time (in the listed overtime slots) or later then their expected move time, as needed by our staff to ensure that we do not run out of any type of resource (available switch ports, cables, apc ports, manpower, etc.) during the move.
- Customers (who host multiple servers) may be able to request to coordinate move times instead of being provided with general times, not all of these requests will be permitted for any of the various reasons outlined above.
- Since we will be utilizing 4 post racks, customers will no longer be permitted to use "mid-mount" rackmount ears, all colocation customer equipment which uses mid-mount rackmount ears has been verified that it can be mounted with the standard front/back mounts.
- Customer intervention will not be required during this move, our staff will be moving all equipment
- *IP addresses will not be changing, our network will remain the same, we are simply moving equipment down the hall to a new suite*
Having said that, we will update the afforementioned forum thread as necessary, and potentially post some pictures. Notably RACO has been bought out by Switch and Data previously this year, which is what caused the signifigant and lengthy delay in this relocation, as we can obviously only move our equipment once the new suite has been properly conditioned for us to do so.
Method:
Servers will be consoled into directly using the standard KVM console. For Linux, Windows, and FreeBSD servers, we will use the "ctrl + alt + del" keyboard combination to initiate a graceful shutdown. If the keyboard is unresponsive, or this shutdown sequence has been specifically disabled by the customer (enabled by default on all versions of FreeBSD and Linux to my knowledge), then we will attempt to SSH, or remote desktop to the server to initiate a gracefull shutdown. If this is unsucessfull, we will push the power button once. Many servers are configured to detect the power button and begin gracefull shutdown. If this is ineffective, we will hold the power button in until the server shuts down manually. Servers will then be removed from the current rack, and relocated to the new suite one at a time, with pre-run power and ethernet drops in the new suite for immediate hook up, and placement on the new racks.
Expected amounts of downtime:
Dedicated Servers - 4-10 minutes per server
Colocated Servers - 4-10 minutes per server
Webhosting/Reseller Accounts - 4-10 minutes per webhosting server. All Webhosting/Reseller servers should be relocated during the 15-Jul-2004 window.
Colocation packages with multiple servers, no switch - 4-10 minutes per server
Colocation Packages with multiple servers, and switch(es) - 2-6 minutes for each server, consequetive for all servers connected to the given switch
Move times (from starting day, to next day):
12-Jul-2004; 10:01am - 7:30am - Wiring clean-up, Customers will be removed from the core switch, and put on smaller 2900 and 3500 series switches. The 5513 switch will be removed from the current location, have both its Route Switch Module, and Redundant Supervisor engines upgraded (additional RAM, and newer releases of the IOS software), it will also have some cards replaced/upgraded. There will be several very short downtimes for each server (< 30 seconds) as wires are relocated, and customers are patched into new switches during this period. Downtime during this period should not exceed 5 minutes for any server.
13-Jul-2004; 8:00pm - 7:30am - Colocation Packages C1 - C40 will be relocated (standard window)
14-Jul-2004; 8:00pm - 7:30am - Colocation Packages C1-C40 will be relocated (late window), Colocation Packages C41 and above will be relocated (standard window), Dedicated Servers D1 - D40 will be relocated (early window)
15-Jul-2004; 8:00pm - 7:30am - Colocation Packages C41 and above will be relocated (late window), Dedicated Servers D1 - D40 will be relocated (standard window), Dedicated Servers D41 and above
will be relocated (early window)
16-Jul-2004; 8:00pm - 7:30am - Dedicated Servers D1 - D40 will be relocated (late window), Dedicated Servers D41 and above will be relocated (standard window)
17-Jul-2004; 6:00pm - 7:30am - Clean-up, anything that has not yet been moved will be relocated at this point, customers will be sent MRTG/APC information emails if possible.
18-Jul-2004; 6:00pm - 7:30am - Secondary Clean-up period, all equipment should be out of 704 at this time.
- Routers, monitoring servers, backup servers, and any miscellanious servers which are not directly providing a frontend service to any customers may be relocated at random during these periods to ensure the maximum amount of uptime, smoothness, and redundancy throughout the move.
Plans/Definitions:
Early Window - Servers will only be moved at this period if they are "stragglers" (eg. last unit on a given APC Masterswitch, thus the Masterswitch may be relocated to the new suite).
Standard Window - Time window in which you should expect your server to be moved.
Late Window - Servers will be relocated in this window if there was insufficient time in the standard window to move the server, or if there was any form of problem moving the server.
I believe this should sum everything up quite nicely for everyone, this indeed gives a detailed look at whats being done during the move (the process), and should give a default window to all customers thus they may announce maintenance as appropriate to their customers. Please remember that the scheduling window is simply a timeframe for when the given maintenance will be done, not an expected amount of downtime. If you have any questions regarding this, please do not hesitate to ask. And view the forum thread (top of email) for updates, and/or pictures as they become available.
. Lets hope for a smooth migration. If theres anything I've not addressed here, questions, comments, anything, just make sure to post/email us regarding it and i'll cover it in this thread.