www3 Crash - 05-Jan-2005 - 2:00pm EST

Announcements concerning Networking & Related News, Planned Outages, Anything which may affect your services.

Moderator: Admins

Post Reply
porcupine
Site Admin
Posts: 703
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

www3 Crash - 05-Jan-2005 - 2:00pm EST

Post by porcupine »

Hi guys,

well unfortunatly the www3 server crashed this afternoon at 2:00pm (EST). The server was down for roughly 20 minutes before we could get someone to the console (server was up 7 minutes later after basic attempts to login timed out, and the server completed a reboot). We discovered that the regular login prompt was visible on the console (which typically rules out memory errors, kernel crashes, etc. as they all output errors to the console).

At the current time, our best estimate is that the server had crashed due to excessive load, though we have no logs to verify this. We currently run a load logger that watches specific system parameters and services (eg. mysql, apache, the "top" output, etc.) and runs every 3 minutes if the current load average is over 10.00 on this server. Unfortunatly the logger didn't get any matches during this timeframe.

We will most likely be adding an additional 512MB of RAM to this server as a "just in case" measure, and scheduling that as a night time/early morning maintenance period soon.

Sorry for any inconviniences this may have caused. Should it happen again, lets hope we get more to go on next time.

Regards,
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
Post Reply