Server Live and Building Info Feeds

We're pleased to announce our server is back online and building its information feeds as normal.

Still out, for a rebuild

We decided to upgrade the current installation of our server to full raid. After first copying the important recovered data from the working old mirror drive. We bought to new 1TB disks and setup, a Raid using ubuntu 9.10 install. This ment a fresh install of Linux. The server will be replaced in our Colo location, on Friday. However because are previous mirroring was performed by 'tar' and not something them preserves unix ownership and permission, many bugs may remain on the system.

Phew found saved data

Out of our three disks, one had a readable file-system, with a data back to the 20's. So we've lost two weeks of work. Our backup mirroring was just good enough, but we didn't have the redundancy we won't for real safety. Because we're now caching so many web pages, we fulled the primary hard drive so much, that backing up would fail, plus two weeks of the pressure of having a disk being pushed to full up all the time, finally made the primary drive fail. God bless rsync. What was really bad of us, is that we didn't have a copy of the database of the feeds anywhere else except the server. They should have been copies back to our development system made at least once a weak.

Going to improve our backup methodology next time. Onwards with the recovery.

Server outage

We had a hard drive failure over the easter holiday, and are currently offline. The server is current back in our labs, and we're recovering the data, and swapping over from the mirror drives.
We'll update you soon. Unfortunately we didn't budget for a spare live server, so our apologies.