Questions? Pop in our slack channel!
SHIELD provides an easy-to-use backup solution for BOSH deployed services, for operations and appliction delivery teams.
This repository packages SHIELD in a BOSH release for deploying backups to your BOSHified environment.
It provides both the core backup system (complete with a Web UI), as well as a lightweight agent for facilitating locally-initiated backup / restore operations.
To deploy SHIELD to any BOSH with a standard cloud-config:
git clone https://github.com/starkandwayne/shield-boshrelease
cd shield-boshrelease
export BOSH_ENVIRONMENT=<name>
export BOSH_DEPLOYMENT=shield
bosh deploy manifests/shield.yml
The IP of the shield instance can be found with bosh instances (10.244.0.7 in example below):
$ bosh instances
Instance Process State AZ IPs
shield/65424ae5-80b9-42b9-a223-2f732d6085c4 running z1 10.244.0.7
Alternately, to expose SHIELD via a public https endpoint, you can use https://ngrok.com
bosh deploy manifests/shield.yml \
-o manifests/operators/ngrok.yml \
-v ngrok-authtoken=${NGROK_TOKEN:?required} \
-v ngrok-subdomain=${BOSH_DEPLOYMENT}
open https://${BOSH_DEPLOYMENT}.ngrok.com
Open the https URL in your browsers - for example https://10.244.0.7 or https://${BOSH_DEPLOYMENT}.ngrok.com - and you'll need to login with basic auth credentials.
The username is admin, and the randomly generated password (stored in credhub) is can be found with:
./bin/shield-password
It should look something like this:
Or, if you prefer, you can install the SHIELD CLI and access your SHIELD core directly:
shield create backend my-shield https://10.244.0.7
shield backends
Most commands will require you to first login (as above, username admin and password comes from ./bin/shield-password):
shield -k jobs
Note, the -k flag is required for the automatically generated self-signed certificate.
If you want to deploy SHIELD and some agents on your vSphere, you can use Genesis and the shield-deployment template, which has site templates for vSphere:
cd ~/ops
genesis new deployment --template shield-deployment shield
genesis new site --template vsphere my-site
genesis new environment my-site my-env
cd my-site/my-env
From here, you'll want to make sure your name is correct in
name.yml:
---
name: my-shield
and that your BOSH director's UUID is set in director.yml:
---
director_uuid: YOUR-DIRECTOR-UUID
From there, with your BOSH director targeted, you can deploy it:
make deploy
Some data systems can only be backed up from a local process; Redis works this way, since it dumps the backup to local disk. For those systems, you must set up a SHIELD agent, and then configure SHIELD to initiate the backup via that agent.
It's easy.
Just add the release to the deployment manifest, add the
shield-agent template to the job(s) in question, and set up the
autoprovision property to the URL of the SHIELD
endpoint (so that it can pull down a host key for validating
backup/restore operation requests).
---
releases:
- name: shield
version: latest
jobs:
- name: first-job
templates:
- release: shield
name: shield-agent
properties:
shield:
agent:
autoprovision: https://my-shield-endpoint # <--- change this
SHIELD provides the following job templates:
This is the SHIELD core daemon. It provides the metadata services (defining targets, stores, schedules, retention policies and jobs), and runs jobs on schedule. It also provides the API for the command-line utility to use, as well as the Web UI for easier administration of a SHIELD installation.
Every SHIELD deployment requires an instance of this job.
SHIELD uses nginx to provide SSL/TLS termination. The nginx job
handles the certificates and keys, and proxies to the
shield-daemon port for application requests.
SHIELD stores its data in a backend PostgreSQL database. This job provides the machinery to run that database, including schema management.
Every SHIELD deployment requires an instance of this job.
The SHIELD agent is a small daemon that runs on target data system deployments (i.e. your Redis deployment, or your the database component of some other deployment), and fields requests for backup and restore operations.
Note: if your target data system can be easily and quickly backed up via network-accessible interfaces, you don't technically need this job.
Provides additional utilities for the postgres plugin to use, in
conjunction with shield-agent, for use when loading the agent on
PostgreSQL database VMs.
Provides additional utilities for the mysql plugin to use, in
conjunction with shield-agent, for use when loading the agent on
MySQL database VMs.
SHIELD ships with the following plugins baked right in:
A target plugin for backing up one or more (or all!) PostgreSQL databases on a cluster.
A target plugin for backing up one or more (or all!) MySQL databases on a node.
A target plugin for backing up a directory on-disk, as a tarball, complete with ownership and permissions metadata.
A store plugin for storing archive blobs to disk.
A store plugin for storing archive blobs in an S3 bucket.
A target plugin for backing up one or more (or all!) MySQL databases on a node.