Code backups

Why code backups

Code backups are a way to mitigate attack vectors aimed at disrupting the development or running of the Digital Marketplace.

They protect the platform from:

• GitHub outages

• Accidental or malicious deletion of a GitHub repo by an internal or external actor

Note

GitHub would be a single point of failure in our development process without backups somewhere.

What the backup jobs do

For each repo we want to back up there is a backup job on Jenkins. These are created from the a Jenkins job template. The created job polls its given repo for changes to the master branch. On detection of a change to the master branch it will mirror clone the repository from GitHub and then mirror push to an AWS CodeCommit repo.

It relies, for permissions, on the default AWS profile. For our EC2 hosted Jenkins this is the Jenkins instance profile.

To add a repo to the backups

1. Create the CodeCommit repo you will be cloning the GitHub repo in to by:

• adding it to the list of repositories in our Terraform here:

  • digitalmarketplace-aws/terraform/accounts/main/codecommit.tf

• applying the changes to our AWS infrastructure by:

  • running make plan in digitalmarketplace-aws/terraform/accounts/main

  • running make apply in digitalmarketplace-aws/terraform/accounts/main

2. Create a job on Jenkins to automatically detect repo changes and back them up by:

• adding the repo name to the code_backups job definition here:

  • digitalmarketplace-jenkins/job_definitions/code_backups.yml

• creating the Jenkins job by running:

  • make jobs JOBS=code_backups in digitalmarketplace-jenkins/

3. Adding the new Jenkins job to the code_backups group of jobs by:

• Adding the job name to the list of code-backups jobs in:

  • digitalmarketplace-jenkins/playbooks/roles/jenkins/defaults/main.yml

• updating Jenkins by running:

  • make reconfigure in ``digitalmarketplace-jenkins/

Further reading

• Understanding git clone --mirror:

  • https://stackoverflow.com/a/3960063/3967444

• AWS’ guide to implementing this infrastructure:

  • https://aws.amazon.com/blogs/devops/replicating-and-automating-sync-ups-for-a-repository-with-aws-codecommit/

• GitHub’s guide on mirroring repos:

  • https://help.github.com/en/articles/duplicating-a-repository#mirroring-a-repository-in-another-location