www6 Upgrade - 29-Jul-2004 - 30-Jul-2004, 10pm - 7am window

Post by **porcupine** » Wed Jul 28, 2004 4:00 pm

Hey guys,

As people may have noticed, www6 has been "slow" for a fair bit of time. Mostly because of a few resellers who have had large spurts of growth. As such, the server finds itself with ever-fewer resellers, but an ever increasing load.

The compensate this, we will be pulling the drive from this server, and relocating it to a new system. The new specs are as follows:

AMD XP2600+
1GB DDR400 RAM
HD Will remain the same.

If more power is needed, we will probably add a SCSI drive for /var, /usr/local/apache/domlogs, etc. and simply map them onto the new drive. This is being avoided during this window simply because that would require rather extensive downtime, as the server would need to run in single-user mode during that time (several hours).

With this maintenance period, customers should expect 15-30 minutes of downtime for the www6 server, in 2-4 periods. We will be moving the drive to the new system, then recompiling the kernel, and upgrading apache to ensure maximum compatability. If serious problems are encountered, the drive will be returned to its original system.

Regards,

Post by **porcupine** » Fri Jul 30, 2004 6:33 am

One problem has been encountered.

New CPU was a AMD XP2600+ with 512K cache and a 333mhz FSB. The current motherboard only supports a 266MHZ FSB. As such, it only recognizes the CPU as a XP1800+.

We are most likely going to leave this server as-is, as the new RAM should proove a signifigant performance increase (from SDR to DDR). If further performance issues arise down the road we will be replacing the motherboard, as well as adding a SCSI drive as noted above.

This maintenance period is closed, and all affected customers should be fully online again after 2 short 10 minute downtimes.

Post by **porcupine** » Fri Jul 30, 2004 9:16 am

Well it would appear that www6 is rejecting it's new upgrades. The server has been down for some time now as it failed several hours after the maintenance period had passed. We are now disbatching someone to check on its status and try and figure out whats gone on.

Post by **porcupine** » Fri Jul 30, 2004 11:19 am

We've swapped the XP2600+/333MHZ FSB processor for a XP2200+/266MHZ FSB processor. Hopefully this corrects the crashing problem. Currently the server is up, and has one more reboot planned after the new kernel has been compiled. We will obviously be watching this server closely.

Anyone wishing to inquire about receiving compensation for this downtime should email billing@prioritycolo.com

Thanks for your patience everyone

.

Regards,

Post by **porcupine** » Sat Jul 31, 2004 6:03 am

Looks like the CPU swap has nailed the problem, ~20 hours without any incident thus far:

6:02am up 18:24, 1 user, load average: 0.28, 0.31, 0.28
124 processes: 122 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: 7.4% user, 6.4% system, 0.0% nice, 86.1% idle
Mem: 1025248K av, 991644K used, 33604K free, 0K shrd, 208268K buff
Swap: 1052248K av, 4784K used, 1047464K free 324396K cached

You may consider this maintenance period closed

.