28-Oct-2003 12:15 - 12:50 Routing errors/problems

Announcements concerning Networking & Related News, Planned Outages, Anything which may affect your services.

Moderator: Admins

Post Reply
porcupine
Site Admin
Posts: 703
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

28-Oct-2003 12:15 - 12:50 Routing errors/problems

Post by porcupine »

Users may (or may not have) noticed some instability during this period, and network disconnects, etc.

Yipes had had a routing issue on their end, and started mis-routing packets destined to the Istop network starting around 12:15AM est, and continuing to about 1:30am est. The router in question continued to announce our routes to the rest of the world, but not properly route traffic (basically announcing to the world, "this is how you get to priority colo" and then routing the packets to nowhere once it received them). At 12:45 we were alerted to this problem (as we did not get paged as BGP did not go down and our network monitors still found ways out), and at 12:50, we turned off our BGP session with Yipes to allow the rest of the internet to find other paths (eg. Peer1).

Any customers connecting through TorIX would not generally have noticed much impact. As of 2:00am, we've turned back up the BGP session with Yipes as the route is no longer flapping and appears to be stable.

Sorry for any inconviniences caused, as one user asked, this is not normal BGP behavior, typically when a router dies, or has problems the BGP sessions drop, and thus they are no longer receiving a certain set of prefixes (ip's to advertise as being reachable through them), and thus traffic automatically reroutes through available channels).

Regards,
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
Post Reply