There was a catastrophic hard drive failure on the central database server very early this morning, Donncha was on the ball and switched all the traffic to a recent backup so most things would work while we investigated the hardware failure. This means that an old version of your site was shown for a few hours.
A few minutes ago we restored the up-to-date database and we’re currently syncing it to the backup to get back any posts you might have made during the semi-downtime. Even though we were able to recover everything, we’re looking at ways to make things even more redundant, so if this ever happens again the problems will be measure in seconds or minutes. Like I said in the last post, all of the new hardware hasn’t been brought fully online yet.
Technical details: The ext3 filesystem was badly corrupted but thanks to journalling there was no lost data.
Update: All posts have now been restored.
Matt on Not-WordPress, mostly photos 
Thanks for all of your hard work this morning Matt and Donncha. Several hours worth of work over the weekend lost induced a mild state of panic this a.m. but all appears to be back to normal. Thanks again.
LikeLike
Mr. Matt, will the referrer button be restored as well?
LikeLike
with the exception of the referer button…
LikeLike
Thanks for being quick and not losing data.
LikeLike
thank you. i got my lost posts back!!
LikeLike
Matt, a big thanks to you for restoring back the posts…..and a big hug from me…..one post were written with a lot of time(though may seem average to some)
LikeLike
really really great job. thanx
LikeLike
gr8 job
LikeLike
You guys did a great job, and even took the time out to answer a few of my emails with some details. I appreciate that level of customer service, it came as a complete surprise on a free service like this.
You have my loyalty.
LikeLike
Yeah, well looks like something is still wrong elsewhere. I had registered for the account ryanl earlier, written a bit and now it redirects to ben_b. After this happened I was able to register the same user account again but it still takes me to the alternate control panel and doesn’t allow me to have any options whatsoever other than the dashboard.
LikeLike
My latest post is still missing. Hope you can fix it. π¦
http://gaby.wordpress.com/2005/10/31/freelance-your-blog-skills/
It was a really good post and a lot of work. In stead of making a backup I’ve posted links to it. ahhhhhhh How unprofessional of me.
Please make it come back.
LikeLike
Wow! I am liking WordPress already. Great service. Thank you!
LikeLike
Thanks for restoring service. I really appreciate your service.
LikeLike
gotta love ext3 journaling. π
LikeLike
Well done over there; no lost data and only a few hours!! Good work…
LikeLike
Wonderful on everything being brought back up but I can’t get the “Referrer” link off the “Manage” section….
LikeLike
Thanks for sorting the problems out, good communication via the dashboard.
LikeLike
Your work is appreciated.
On the other hand, I was already thinking “major re-writes” and stuff… and then it all came back… arrrfff.
Your work, as I said, *is* appreciated. Thanks all.
LikeLike
Gaby, your post was made onto the backup hardware, and we just restored all of those. Your link works now and your post is back safe and sound. π
LikeLike
yay, my post is magically recovered. π
LikeLike
no, mine is combined with concious.wordpress.com
I’m forestglen.wordpress.com
my dashboard is frozen solid
I can see two of my posts.
my old password works
and computers are wonderful
and computers are not as nice as you guys.
no hurry, I’ll wait a while.
but since I had to recreate forestglen.wordpress.com
I want to keep it, thank you. I used 2 invites so far
to get this blog… worth it, too!
thank you.
LikeLike
Thanks for the speedy work to restore service!!
LikeLike
Define “normal”
LikeLike
You’re really grat. Thx π
LikeLike
Wow you guys really deserve a medal for this whole wordpress thing. If there’s anyway I can help out, just let me know.
LikeLike
I’m seeing a post which now has somebody else’s name as the author even though the post was made by me, I can’t seem to get to the Feedback button as it disappears (using Firefox on Mac OS X). Any ideas why this has happened?
LikeLike
Great job on fixing this issue as quickly as you guys did. And thank you for everything you do.
LikeLike
aah ya me habia asustad. Que el Amo del Merol los bendiga
LikeLike
great job.. last night, about 9 hours ago… some blogs at whatevertheuser.wordpress.com works but mine ain’t working? why is that so? is it because i used flock? are there any differences with blogs made from invites and the ones made with flock?
LikeLike
If anyone is having the problem where a post has the wrong user please use the contact form and we’ll fix that up.
LikeLike