Hi,
As you may be aware, MadcowModels went down this evening (10/02/2014) at about 20:30, and came up at 23:45. The latest info from the hosting company is that there was an InnoDB corruption on the database server we use (essentially, the database corrupted).
This issue affected every customer they have on that database server/cluster (by my estimate, about 2000 (turns out its 6531)).
In a move to get everyone back up and running as soon as possible, they set up new a load of new hardware and loaded everyones databases from the backup.
Theres obviously no good time for a data corruption issue, however, this was probably the worst time as the data back up was about 17 hours old, as a back up is taken every single night at about 3am.
What this means is that you may notice a gap in the data, anything you sent, received, uploaded, deleted etc between 10/02/2014 03:00 and 10/02/2014 23:45 doesn't exist on the data file thats being run at the moment.
I've spoken to the hosting company and they're fighting the fight on 2 fronts at the moment. They're pushing the backups for everyone to get them back up, and attempting to recover the data on the corrupted database server.
If/when they manage to recover that data, I'll merge anything that was sent between the 2 above times (10/02/2014 03:00 and 10/02/2014 23:45) with the current data file so there is no data gap.
I sincerely apologies for the frustration, and inconvenience. I'll speak to the hosts again in the morning to find out what the situation is regarding the data gap.
Once again, I apologise for the events this evening,
James
MadcowModels Manager
As you may be aware, MadcowModels went down this evening (10/02/2014) at about 20:30, and came up at 23:45. The latest info from the hosting company is that there was an InnoDB corruption on the database server we use (essentially, the database corrupted).
This issue affected every customer they have on that database server/cluster (by my estimate, about 2000 (turns out its 6531)).
In a move to get everyone back up and running as soon as possible, they set up new a load of new hardware and loaded everyones databases from the backup.
Theres obviously no good time for a data corruption issue, however, this was probably the worst time as the data back up was about 17 hours old, as a back up is taken every single night at about 3am.
What this means is that you may notice a gap in the data, anything you sent, received, uploaded, deleted etc between 10/02/2014 03:00 and 10/02/2014 23:45 doesn't exist on the data file thats being run at the moment.
I've spoken to the hosting company and they're fighting the fight on 2 fronts at the moment. They're pushing the backups for everyone to get them back up, and attempting to recover the data on the corrupted database server.
If/when they manage to recover that data, I'll merge anything that was sent between the 2 above times (10/02/2014 03:00 and 10/02/2014 23:45) with the current data file so there is no data gap.
I sincerely apologies for the frustration, and inconvenience. I'll speak to the hosts again in the morning to find out what the situation is regarding the data gap.
Once again, I apologise for the events this evening,
James
MadcowModels Manager
Opinions expressed within this thread are those of the author and do not necessarily reflect those of ModelFolio, nor its associates.
ModelFolio accepts no responsibility legal or otherwise for the content or accuracy of authors comments.
ModelFolio accepts no responsibility legal or otherwise for the content or accuracy of authors comments.
