Sunday, 11 April 2010
Server Live and Building Info Feeds
We're pleased to announce our server is back online and building its information feeds as normal.
Thursday, 8 April 2010
Still out, for a rebuild
We decided to upgrade the current installation of our server to full raid. After first copying the important recovered data from the working old mirror drive. We bought to new 1TB disks and setup, a Raid using ubuntu 9.10 install. This ment a fresh install of Linux. The server will be replaced in our Colo location, on Friday. However because are previous mirroring was performed by 'tar' and not something them preserves unix ownership and permission, many bugs may remain on the system.
Tuesday, 6 April 2010
Phew found saved data
Out of our three disks, one had a readable file-system, with a data back to the 20's. So we've lost two weeks of work. Our backup mirroring was just good enough, but we didn't have the redundancy we won't for real safety. Because we're now caching so many web pages, we fulled the primary hard drive so much, that backing up would fail, plus two weeks of the pressure of having a disk being pushed to full up all the time, finally made the primary drive fail. God bless rsync. What was really bad of us, is that we didn't have a copy of the database of the feeds anywhere else except the server. They should have been copies back to our development system made at least once a weak.
Going to improve our backup methodology next time. Onwards with the recovery.
Going to improve our backup methodology next time. Onwards with the recovery.
Server outage
We had a hard drive failure over the easter holiday, and are currently offline. The server is current back in our labs, and we're recovering the data, and swapping over from the mirror drives.
We'll update you soon. Unfortunately we didn't budget for a spare live server, so our apologies.
We'll update you soon. Unfortunately we didn't budget for a spare live server, so our apologies.
Friday, 12 March 2010
Tuesday, 9 March 2010
Server: Cached and Threaded
Just switched to our new version of our aggregation engine, which finally caches old pages to avoid reloading each web page, and possibly straining our target source websites. We've also multithreaded the engine, to use as many processor cores on the server as we want. Feed Distiller is now good for thousands of feeds, and hundred of thousands source pages. We'll be watching for bugs, over the next few days. But so far so good.
Friday, 26 February 2010
350 and Counting
Modest growth in Feed Distiller, where now up to 350 Feeds.
In the labs and coming soonish, a upgrade to the engine, which
should let it cope with a hundred times more blogs. Plus we're
in development of a chatroom. Which of course, has to be custom
built and ultra scalable. Incomes upping in triple figures. And
slightly ahead of january already.
In the labs and coming soonish, a upgrade to the engine, which
should let it cope with a hundred times more blogs. Plus we're
in development of a chatroom. Which of course, has to be custom
built and ultra scalable. Incomes upping in triple figures. And
slightly ahead of january already.
Subscribe to:
Posts (Atom)