Our "clubsitenews" Feed
05 Sep 2018 12:58
Labor Day 2018 Server Outage
Between the evening of Labor Day 2018 and the following morning, the server on which the SBRTA site runs suffered an outage. During an attempt to upgrade the server to include so-called SSL security following procedures that are well documented by the hosting service (Linode), the server mysteriously went down denying all access to SBR websites (and others).
Recovery from such problems is normally trivial: backup to the most recent backup copy of the complete server. In this case, the last backup was only a few hours prior to the outage event. Backing up server software is a service provided by the hosting company, with backups being performed daily and weekly.
However, in the process of attempting that recovery, it was discovered that the most recent backup copy, and a few others preceding that one, were corrupted. The most recent backup turned out to be the last weekly update. Technical staff are so far unable to state what caused the corruption; they are analyzing the event. (There have been a number of times that backups were successfully and smoothly used to restore the state of the server.)
Use of that weekly backup proved successful such that early in the morning following Labor Day, the server and all websites were up and running. However, because of the time span between that weekly backup and the outage, recent updates by users of the site were unrecoverable. At that point, I went about manually adding those updates back into the sites based on email and other notes I had. With minor omissions, that was successful. It was also necessary to remove sign-ups for tennis sessions that pertained to days prior to the outage and therefore were no longer relevant; those were deleted.
Details about the SSL security upgrade
Regarding the security updates, there is a widespread movement to include SSL ("Secure Sockets Layer", a protocol security measure) on all servers. The effect of SSL is to encrypt all information transfers between servers and clients (e.g., your computer browser). Browsers are beginning to display a warning that the server is not secure. Approximately 35% of servers now include SSL.
In the case of all SBR websites, no commerce is performed that would involve the transfer of sensitive information such as credit card numbers. Also, by using the proper protocol for transfer of information to servers, there are two methods, one which displays the information in the URL, and one that does not. The latter is used for SBR sites (and all sites I develop). Therefore, the risk is minimized that sophisticated methods could be employed to capture a password. Should that password be captured by nefarious persons, they might be able to use it in combination with userIDs to break into other websites SBRTA members access. The common and well-known defense is to not re-use passwords for multiple sites.
Nevertheless, the goal remains to install SSL on our Linode server. I will not do that until senior Linode technical personal can provide information as to the cause of backup file corruption and provide assurance that SSL can be installed using their procedures. In the meantime, if you are concerned that someone might fake your identity and sign you up for a session or access your profile, you are advised to use a password that is unique to your SBRTA account and change it frequently (e.g., four times a year).
Backup measure employed
FYI, all SBRTA software (as opposed to server software) is continuously maintained on four independent servers/computers with one being protected against single failures (RAID storage).