Managing database backups for the NFT environment

Unlike the other environments which can use a Jenkins job to take backups of the database, for NFT we need to do the database backup manually. We can follow most of the steps already defined in ‘Restoring from a backup’ however, this guide was created as there are a few differences to be aware of.

Prerequisites

Before you run any commands, you need to be logged into cloud foundry and should be targeting the right environment:

cf target -o ccs-dmp-so-interim -s nft

Create backup of the database

The first step is to take a backup of the database from the right point in time. Fortunately, as we knew we would want to restore the NFT database often, a perminant backup was created called digitalmarketplace_api_db_backup. It is this database service we will need to take a copy of.

A full guide on how database backups can be created is found in the GPaaS documentation. However, we shall still lay out the specific steps needed for the NFT backup.

Get the ID of the service:

GUID=$(cf service digitalmarketplace_api_db_backup --guid)

Prevision the resource on GPaaS:

cf create-service postgres medium-ha-12 digitalmarketplace_api_db_backup-copy -c '{"restore_from_point_in_time_of": "'"$GUID"'"}'

At this point, the new database instance is being created which can take anywhere from 10-20 minutes. You can check the status of the service by running:

cf service digitalmarketplace_api_db_backup-copy

Once the state is ‘succeeded’ then the database copy has been created and you can move onto the next step of updating the NFT environment to use that database.

Telling the NFT environement to use the backup

You will need to update the vars/nft.yml by replacing the specified lines with:

maintenance_mode: maintenance

- digitalmarketplace_api_db_backup-copy

Once you have created a pull request for the changes and they have been merged, you need to release all the apps. This can be done by building the nft-deploy-all job in Jenkins.

Once this job has finished, the api should be using the database backup. You can check this by using the command cf service digitalmarketplace_api_db_backup-copy which should list api in the list of bound apps.

We now need to clean up these changes and move the application back into a live state.

Clean up

The first step is to rename the original database service. This can be done with the following command:

cf rename-service digitalmarketplace_api_db digitalmarketplace_api_db-old-obsolete

You can now delete the service but before you do, you will need to unbind any apps bound to it first. This can be done with:

cf unbind-service APP_NAME digitalmarketplace_api_db-old-obsolete

You should then be able to delete this service with:

cf delete-service digitalmarketplace_api_db-old-obsolete

We can now safely rename our new database to digitalmarketplace_api_db with:

cf rename-service digitalmarketplace_api_db_backup-copy digitalmarketplace_api_db

After doing this we need to update the vars/nft.yml by replacing the specified lines with:

maintenance_mode: live

- digitalmarketplace_api_db

Once you have created a pull request for the changes and they have been merged, you will again need to release all the apps using the nft-deploy-all Jenkins job.

Finally, once all the apps have been redeployed, you will have finished the database restoration and the NFT environment should be ready to run tests in again.