www4 HDD Dying

Announcements concerning Networking & Related News, Planned Outages, Anything which may affect your services.

Moderator: Admins

Post Reply
porcupine
Site Admin
Posts: 704
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

www4 HDD Dying

Post by porcupine »

Hey guys,

Well it looks like the hard drive on the www4 server is on its last legs.

It was rebooted after producing an error, and does not appear to be coming back up. Thus www4 will have to be rebuilt. We have backups, and I believe that we will be able to extract almost all of the current data from the www4 server asis, it will simply involve downtime as a result.

Please expect the www4 server to be down for the remainder of the night (12-18-2003) , until early morning on Friday December 19th. I will be personally rebuilding this server (just heading out to start with a fresh Redhat 9 install now, and a fresh drive, and begin pulling the necessary data from the old drive).

I appologise for any inconviniences this may cause, and will ensure that everything within reason is done to remedy this situation.
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
porcupine
Site Admin
Posts: 704
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

Post by porcupine »

Ok guys,

Well I know many of you are itching for an update.

Data is being actively pulled from the old drive, and the server has a nice new 120 gig 7200/8mb cache drive in it, with Redhat 9 loaded on it. We're pulling the latest weekly backups, because the content is riddled with errors, as much of it was written during the issues.

We're also taking this time to move accounts to the new IP addresses, might as well get them all done while we have the chance (and what better chance then an extended disruption of service, regardless of how unfortunate, better to do them now with minimal additional impact then a few months down the road with more impact and disruption).

Anyhow, progress is slow, but steady, all sites should be returning to normal status by early tomorrow morning, or later this evening.
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
porcupine
Site Admin
Posts: 704
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

Post by porcupine »

The server restore for www4 is generally complete. all of the accoutns (with the exception of the following) have been restored, and services are being brought back online. If you had a SSL certificate on your account for HTTPS that will need to be reconfigured before it'll function correctly.

The following acconts errored out while attempting to restore:

a-blackj ... failed
aganan-a ... failed
arte-san ... failed
best-cc ... failed
best-onl ... failed
ha66uk ... failed
i-missed ... failed
it-help ... failed
k-sysco ... failed
murih ... failed
newfresh ... failed
pa-syco ... failed
queen-te ... failed
te150 ... failed
tribal-m ... failed

They will be retried, but WHM/CPanel did "not agree" with some of those and they simply failed. Some of those are probably recoverable errors, but definatly not all. We will be contacting the resellers in question, providing what content is available from said accounts (as sometimes its just an email address CPanel doesen't like, but all of the mysql, and home directory contents are intact and fine) to the resellers in question.
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
PeteA
newbie
Posts: 9
Joined: Tue Mar 04, 2003 12:20 pm

Post by PeteA »

Myles,

Have sent you an email from my work addy regarding pa-syco.

Looks like we've lost any data uploaded between now and the last backup :(
porcupine
Site Admin
Posts: 704
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

Post by porcupine »

Just an update on this topic:

- ha66uk didn't work because it appears to already have been setup in the httpd.conf, see below.
- newfresh - same problem as above
- te150 - same problem as above
- tribal-m - same problem as above

murih - Complete backup failure, invalid .tar.gz file for the weekly and the monthly, I have no clue how this happened, but i've gone in and tried to restore it by hand and all utilities indicate the files are corrupt.

All other previously listed accounts have restored, and reported as properly restored.

The first 4 listed accounts can be restored, but have duplicate entries, thus the system does not know what to do (they all appear to be subdomains, where a user [assumedly the reseller] has added them as an addon subdomain, then later made them into seperate accounts). To put it bluntly, the users outlined above have fallen victim to their own sloppy housekeeping, and the accounts can be restored (with the current subdomain records being deleted) upon request, but we will not do that unless requested to (as theres duplicate data, we dont want to be deleting the wrong set).
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
PeteA
newbie
Posts: 9
Joined: Tue Mar 04, 2003 12:20 pm

Post by PeteA »

re: ha66co

Sorry about the above, my customer cocked me about and there were many problems. I'm happy for all entries of this to be killed off, as I was never paid for it, etc...

thx
porcupine
Site Admin
Posts: 704
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

Post by porcupine »

The entries wont be killed off, we're just not overwriting the existing accounts (subdomains off a larger domain) with the individual usernames unless specifically requested to :).

Some people noticed their resellers were unable to login, this is because I forgot that CPanel does *not* backup, or store in any way shape or form (currently) the reseller privaledges or any of the information. This has all been re-input by hand, and the privaledges, etc. have all been reconfigured by hand.

This would cause people to be unable to login to WHM, or be able to login to WHM and not be able to see/do anything. This should now be corrected also.
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
ZeroFill
newbie
Posts: 32
Joined: Wed May 28, 2003 1:59 am

Post by ZeroFill »

has a newsletter been setup so we will be notified automatically via email? the newsletter containing the first update with a link to the forum would be great -- we could subscribe to the forum and get email updates whenever you add more content.

thanks
cheapsnet
newbie
Posts: 3
Joined: Tue Apr 15, 2003 5:58 am
Location: Prioritycolo
Contact:

Help to restore back my primary account

Post by cheapsnet »

Dear Sir/Madam,

Please restore back the "murih" account, because this account contains our mysql databases of all members from Indonesian country. This account is very important.

Let me know your response back.
porcupine wrote:The server restore for www4 is generally complete. all of the accoutns (with the exception of the following) have been restored, and services are being brought back online. If you had a SSL certificate on your account for HTTPS that will need to be reconfigured before it'll function correctly.

The following acconts errored out while attempting to restore:

a-blackj ... failed
aganan-a ... failed
arte-san ... failed
best-cc ... failed
best-onl ... failed
ha66uk ... failed
i-missed ... failed
it-help ... failed
k-sysco ... failed
murih ... failed
newfresh ... failed
pa-syco ... failed
queen-te ... failed
te150 ... failed
tribal-m ... failed

They will be retried, but WHM/CPanel did "not agree" with some of those and they simply failed. Some of those are probably recoverable errors, but definatly not all. We will be contacting the resellers in question, providing what content is available from said accounts (as sometimes its just an email address CPanel doesen't like, but all of the mysql, and home directory contents are intact and fine) to the resellers in question.
Cheapest and completed web domain over 158 ext. from over 80 country http://mydomain.cheaps.net.
porcupine
Site Admin
Posts: 704
Joined: Wed Jun 12, 2002 5:57 pm
Location: Toronto, Ontario
Contact:

Post by porcupine »

ZeroFill wrote:has a newsletter been setup so we will be notified automatically via email? the newsletter containing the first update with a link to the forum would be great -- we could subscribe to the forum and get email updates whenever you add more content.

thanks
Heya,

this has been asked before, and no, we do not have a general mailing list. That involves any staff posting to remember to basically double post, and is not as viable. We've had a setup in our database to contact customers in certain situations, but it's rarely used for several reasons, and hasn't been used recently whatsoever.
Myles Loosley-Millman
Priority Colo Inc.
myles@prioritycolo.com
http://www.prioritycolo.com
Post Reply