ISC Server crashed and now recoveredDear all,
Most of you would have noticed ISC was down for about 3 days. I just wanted to let our members know what happened.
ISC and the sister sites are hosted on a server in the USA. We have a pretty old physical server sitting in a great data centre, while most other websites are now running on cloud-based systems. Since we are on a physical server, we have limitations on automated backups etc. We use a separate physical hard disk to take automated backups and periodically download such backups to other computers.
On 25 May 2021, our server went down. We made every efforts to restart the server remotely and with the help of the support team. I wasn't panic until 24 hours later the support team informed me that the server could not be restored after the Windows crashed and the backup drive it was was crashed which is the root cause of the problem. This left me with a big question - if the backup drive itself is crashed, then I have to rebuild a new server with the data I had downloaded many days ago. All valuable posts from our members for several days will be lost.
Since the server and support team was in the USA, I had to stay awake all night for 3 days motivating and begging them to make all possible efforts to recover the drives. Finally, they agreed to make a fresh installation of the Windows on the primary hard drive and make all efforts to keep the old data in the drive.
Finally, after 3 days, we could reinstall Windows on the primary drive and I was excited to see the old data and files remaining on the drive. So, even though we lost all backups, the original data files were still present and I could recover everything without any data loss.
We lost 3 valuable days but didn't lose any past data. Thanks a lot to the almighty God for this wonderful recovery after the serious situation we faced.
We have learnt a big lesson the hard way. Now the goal is to ensure this will not happen again.
In the next few days, we are moving to a more powerful server in India where we can get more support, a lot closer to us and in our time zone. Also, we will be going for a cloud-based server with automated backup systems in place.
Thanks to all members who patiently waited for us.