Scheduled tor1 core network maintenance - 14-Jan-2022

Announcements concerning Networking & Related News, Planned Outages, Anything which may affect your services.

Moderator: Admins

Post Reply
Site Admin
Posts: 701
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario

Scheduled tor1 core network maintenance - 14-Jan-2022

Post by porcupine »

Hi Folks,

Please be advised of the following maintenance window:

Maintenance Window: Friday January 14th, 2022 - 11:00pm - 2:00am Eastern time
Facility: Tor1
Affected Device(s): core02.tor1
Affected Customers: Tor1 general network
Service Impacting: Yes - The work in question will have the potential to cause several small outages/connectivity drops.

Details: We will be replacing the current core02.tor1 router with a spare unit we have been maintaining on-site. Unfortunately the current core02.tor1 router has begun logging memory alignment errors between the TOE (Trinity Offload Engine), and the PFE (Packet Forwarding Engine). These errors indicate a major hardware fault, and while they're not currently impacting the flow of traffic through the device, they indicate a major fault is present.

The spare router will be pre-configured with core02.tor1's exact running configuration, and put into place immediately adjacent to the current router. Sessions to the current core02.tor1 router will be gracefully shut-down, and then cables/optics moved to the new device, which will then be brought back online.

BGP customers directly connected to the core02.tor1 router will see their sessions go down/get reset during this operation as a result.

The replaced core02.tor1 router will then be returned to the original vendor, and the spare router will then be replaced with a newly procured unit to be kept as a cold-spare.

As always, we will be carefully performing the maintenance work in question, and will use every reasonable measure to attempt to mitigate any unnecessary outages/incidents/etc.

While the maintenance window in question is 3 hours long to allow staff time to prep/test/etc., we expect actual interruptions in connectivity to be limited to several substantially shorter events, much like the previous maintenance operation.
Myles Loosley-Millman
Priority Colo Inc.
Post Reply