Creating a 'Database is Down' Page

Earlier today the database for this site was unavailable for around 30 minutes, I imagine something was being rebooted somewhere. This doesn't really concern me too much, since this is just a personal site. So during this time anyone visiting the site was sent to a holding page explaining that the database was down, and providing some links of places for people to visit instead. I make the choice to hide the error details from the user, rather than displaying a page with cryptic error messages on it, or even worse an error message which prints out some critical information. After all, the user probably doesn't really care why my database is down, they just care that they can't get to the information they want. So instead, I log the error details internally and just give a nice page to the user.

I received an email from someone who obviously tried to visit my site during this time, asking how this was done. Since I haven't updated in a while, I thought it's be good to give a breif overview of how it's done.

It's not very difficult and just involves a simple catch statement. If there's an error when attempting to the connect to the database then I log it and email myself the error, and then redirect the user to the holding page.

In the DBAccess class for wblog (the name I gave to the backend code that runs this site), the connect() method looks like this,
function connect()
{
    global $DB_CONFIG;
    
    if ($this->getHandle() != null) return; // Already connected

    $pdo_string = 'mysql:dbname='.$DB_CONFIG['database'].';host='.$DB_CONFIG['hostname'];
    
    try
    {
        Log::create("DB","Connecting to ".$pdo_string." as ".$DB_CONFIG['username'], Log::INFO);
        
        $this->_handle = new PDO($pdo_string, $DB_CONFIG['username'], $DB_CONFIG['password']);
        $this->getHandle()->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    }
    catch (PDOException $e)
    {
        Log::create("DB","Exception caught while connecting. ".$e->getMessage(),
                    Log::ERROR,
                    Log::NOTIFY_ADMIN);
        header("Location: /dbdown/");
    }
    Log::create("DB","Connected to database", Log::INFO);
}


The relevant bit of the code is this,
    $this->getHandle()->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
catch (PDOException $e)
{
    Log::create("DB","Exception caught while connecting. ".$e->getMessage(),
                Log::ERROR,
                Log::NOTIFY_ADMIN);
    header("Location: /dbdown/");
}


I set the ATTR_ERRMODE attribute so that it will throw exceptions (See the PHP manual for details on setting attributes for PDO). I then catch any PDOException that's thrown. The "Log" class for wblog takes care of recording the error into a log file on the server, and the extra argument Log::NOTIFY_ADMIN tells it to drop me an email with a copy of the log (so that I get alerted to something going on). It's smart enough not to just email me the same thing everytime a user visits though, and has an increasing delay before it emails me the same thing again. (So first 5 minutes, then 10, 15, 20.. etc). You'll find people keep refreshing the site when they get an error page, so you'll just fill your inbox with emails if you're not careful.

Since you don't know what's in my Log class, you can just ignore those bits of the code. I'll probably make another post one day with the full code for this site anyway.

Once the error has been logged, I use the Location header to redirect the user to my holding page. Then I just make sure the holding page is static HTML file and doesn't touch the DB. It could also be PHP, but for my purposes a static HTML file is fine. As long as it doesn't touch the database, otherwise you're going to get into an infinite loop.

So there you have it, that's how I deal with issues when my database is down. If you have any better ways, or see something wrong with how I'm doing it, then let me know in the comments.
Picture of Rich Adams.

Hi! I'm Rich. By day I work on cloud security at Indeed. By night I wear a cape and fight crime1. I'm on Twitter as @r_adams, and also used to write things on the PagerDuty blog.

1 probably not true

References

A list of all the links that appear in this note,