BigID Documentation
Table of Contents
Installing BigID > Deploy BigID with Kubernetes > BigID Core > MongoDB
Backing up and restoring the MongoDB 3
Backing up and restoring the MongoDB
In Short
MongoDB is used by BigID for its internal storage. This document describes how to backup and restore
your BigID MongoDB to/from a Dump le and to/from a Docker container.
This document is useful for system administrators.
See more: MongoDB with SSL
Note
MongoS (shard router) is not supported.
Kubernetes: Backing up and restoring MongoDB
1. File: mongo-pvc.yaml # Builds the PVC
2. Set the storage size, for around what a compressed version of your database would be:
YAML Copy
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mongo‐backup‐pvc
spec:
accessModes:
‐ ReadWriteOnce
resources:
requests:
storage: 100Gi
Do not share. BigID confidential and privileged information.
storageClassName: default
2
Table of Contents
Installing BigID > Deploy BigID with Kubernetes > BigID Core > MongoDB
Backing up and restoring the MongoDB
Backing up and restoring the MongoDB
In Short
MongoDB is used by BigID for its internal storage. This document describes how to backup and restore
your BigID MongoDB to/from a Dump le and to/from a Docker container.
This document is useful for system administrators.
See more: MongoDB with SSL
Note
MongoS (shard router) is not supported.
Kubernetes: Backing up and restoring MongoDB
1. File: mongo-pvc.yaml # Builds the PVC
2. Set the storage size, for around what a compressed version of your database would be:
YAML Copy
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mongo‐backup‐pvc
spec:
accessModes:
‐ ReadWriteOnce
resources:
requests:
storage: 100Gi
storageClassName: default
3. File: mongo-dump.yaml # This dumps the MongoDB
YAML Copy
Do not share. BigID confidential and privileged information.
#mongo‐dump.yaml
apiVersion: batch/v1 3
kind: Job
storage: 100Gi
storageClassName: default
3. File: mongo-dump.yaml # This dumps the MongoDB
YAML Copy
#mongo‐dump.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: mongodump‐job
spec:
template:
spec:
containers:
‐ name: mongodump
image: mongo
command: ["/bin/bash", "‐c"]
args: ["mongodump ‐‐uri='mongodb://bigid:password@bigid‐mongodb‐
headless:27017/bigid‐server?authSource=admin&replicaSet=bigid‐replica‐
set&ssl=false' ‐‐archive=/backup/backup.gz ‐‐gzip"]
volumeMounts:
‐ name: mongo‐backup‐volume
mountPath: /backup
volumes:
‐ name: mongo‐backup‐volume
persistentVolumeClaim:
claimName: mongo‐backup‐pvc
restartPolicy: OnFailure
backoffLimit: 4
4. File: mongo-restore.yaml # This restores to the target database
YAML Copy
# mongo‐restore.yaml
## You will have to change the RESTORE URI TO MATCH THE TARGET RESTORATION
DATABASE THIS IS JUST AN EXAMPLE
apiVersion: batch/v1
kind: Job
metadata:
name: mongorestore‐job
spec:
template:
spec:
containers:
‐ name: mongorestore
image: mongo
command: ["/bin/bash", "‐c"]
args: ["mongorestore ‐‐uri='mongodb://bigid:password@bigid‐
mongodb‐headless:27017/bigid‐server?authSource=admin&replicaSet=bigid‐
Do not share. BigID confidential and privileged information.
replica‐set&ssl=false' ‐‐archive=/backup/backup.gz ‐‐gzip"]
volumeMounts: 4
‐ name: mongo‐backup‐volume
restartPolicy: OnFailure
backoffLimit: 4
4. File: mongo-restore.yaml # This restores to the target database
YAML Copy
# mongo‐restore.yaml
## You will have to change the RESTORE URI TO MATCH THE TARGET RESTORATION
DATABASE THIS IS JUST AN EXAMPLE
apiVersion: batch/v1
kind: Job
metadata:
name: mongorestore‐job
spec:
template:
spec:
containers:
‐ name: mongorestore
image: mongo
command: ["/bin/bash", "‐c"]
args: ["mongorestore ‐‐uri='mongodb://bigid:password@bigid‐
mongodb‐headless:27017/bigid‐server?authSource=admin&replicaSet=bigid‐
replica‐set&ssl=false' ‐‐archive=/backup/backup.gz ‐‐gzip"]
volumeMounts:
‐ name: mongo‐backup‐volume
mountPath: /backup
volumes:
‐ name: mongo‐backup‐volume
persistentVolumeClaim:
claimName: mongo‐backup‐pvc
restartPolicy: OnFailure
backoffLimit: 4
Docker: Backing up and restoring MongoDB
To backup MongoDB to disk:
1. Attach to the bigid-mongo docker container:
Bash Copy
docker exec ‐it bigid‐mongo bash
#run mongodump
mongodump ‐h localhost ‐u <username> ‐p <password> ‐‐
authenticationDatabase=admin ‐‐gzip
This dumps all MongoDB collections to a directory named dump .
2. Exit the bigid-mongo container: exit
Do not share.3. Copy
BigID theand
confidential dump folder
privileged to docker
information. host:
docker cp bigid‐mongo:/dump . 5
mongodump ‐h localhost ‐u <username> ‐p <password> ‐‐
authenticationDatabase=admin ‐‐gzip
This dumps all MongoDB collections to a directory named dump .
2. Exit the bigid-mongo container: exit
3. Copy the dump folder to docker host:
docker cp bigid‐mongo:/dump .
To restore the backup:
1. Copy the backup archive into the bigid-mongo container:
docker cp dump bigid‐mongo:/home/ubuntu
2. Run mongorestore and enter the username and password as required:
mongorestore ‐‐host MONGO_ADDRESS ‐‐username <username> ‐‐password <password> dump/ ‐‐
gzip
Backing up and restoring parts of the MongoDB to a
dump le
To backup only some of the collections to disk:
1. Attach to the bigid-mongo docker container:
Bash Copy
docker exec ‐it bigid‐mongo bash
2. Run mongodump:
For example:
Bash Copy
mongodump ‐h localhost ‐u <username> ‐p <password> ‐‐
authenticationDatabase=admin ‐‐collection ds_connections ‐‐gzip
mongodump ‐h localhost ‐u <username>‐p <password>‐‐
authenticationDatabase=admin ‐‐collection id_connections ‐‐gzip
mongodump ‐h localhost ‐u <username>‐p <password>‐‐
authenticationDatabase=admin ‐‐collection scan_profiles ‐‐gzip
3. Exit from the bigid-mongo container: exit
To exclude collections, use the ‐‐excludeCollection ag to specify all collections to be excluded.
This dumps all MongoDB collections to a directory named dump
4. Copy the dump folder to docker host: docker cp bigid‐mongo:/dump
To restore the backup:
1. Copy the backup archive into the bigid-mongo container:
Do not share. BigIDdocker
confidential cp dump bigid‐mongo:/home/ubuntu
and privileged information.
2. Run mongorestore and enter the username and
6 password, as required:
This dumps all MongoDB collections to a directory named dump
4. Copy the dump folder to docker host: docker cp bigid‐mongo:/dump
To restore the backup:
1. Copy the backup archive into the bigid-mongo container:
docker cp dump bigid‐mongo:/home/ubuntu
2. Run mongorestore and enter the username and password, as required:
Bash Copy
gzip ‐d ./dump/bigid‐server/*.gz
mongorestore ‐‐port 27017 ‐‐authenticationDatabase admin ‐‐db bigid‐
server ‐‐username <username> ‐‐password <password> dump/bigid‐server/ ‐‐
gzip
Backing up and restoring MongoDB inside Docker
Back up the data on MongoDB regularly. Backup and restore can be done using the standard Mongo utilities
mongodump and mongorestore .
To backup and restore MongoDB data inside Docker:
1. Ensure that suf cient space is available for the backup les.
2. Create a backup volume:
docker volume create ‐‐name bigid‐mongo‐backup
3. Make sure that the container to backup is up.
4. Run the following command on the host running mongo (enter the command without line breaks)
and enter the username and password, as required:
Bash Copy
docker run ‐‐rm ‐‐link <container name>:<alias> ‐‐net <network id> ‐v
bigid‐mongo‐backup:/backup mongo bash
‐c 'mongodump ‐‐out /backup ‐‐host <host ip>:<db port> ‐‐
username <username> ‐‐password <password> ‐‐authenticationDatabase admin
‐‐db bigid‐server
‐‐excludeCollection <collection_name_to_exclude>
‐‐gzip
Example (enter the command without line breaks) and enter the username and password, as
required:
Bash Copy
docker run ‐‐rm ‐‐link bigid‐mongo‐auth:mongo ‐‐net <network id>
‐v bigid‐mongo‐backup:/backup mongo bash
‐c 'mongodump ‐‐out /backup ‐‐host 192.168.99.100:27017 ‐‐
username
Do not share. BigID confidential <username
and privileged information.> ‐‐password < password>
‐‐authenticationDatabase admin
‐‐db bigid‐server
7
‐‐excludeCollection pii_findings ‐‐excludeCollection
‐‐gzip
Example (enter the command without line breaks) and enter the username and password, as
required:
Bash Copy
docker run ‐‐rm ‐‐link bigid‐mongo‐auth:mongo ‐‐net <network id>
‐v bigid‐mongo‐backup:/backup mongo bash
‐c 'mongodump ‐‐out /backup ‐‐host 192.168.99.100:27017 ‐‐
username <username> ‐‐password <password> ‐‐authenticationDatabase admin
‐‐db bigid‐server
‐‐excludeCollection pii_findings ‐‐excludeCollection
scanned_values
‐‐gzip
To nd the host_ip value and network id value, run docker inspect <container name/id> and
under NetworkSettings/Networks are two relevant values: Gateway is the host_ip and NetworkID.
Note
Use the excludeCollection param to exclude irrelevant collections. For example, to
exclude scan data (but to keep con guration and other data), exclude the following:
Bash Copy
'pii_records', 'pii_findings', 'enrichment_findings', 'lineage_findings',
'grouped_lineage_findings', 'lineage_pii_fields', 'lineage_nodes_1',
'lineage_nodes', 'collection_metadatas', 'piisummary_no_filters',
'piisummary_common_filters', 'pii_investigations', 'scanned_identities',
'scanned_attributes', 'scanned_values',
'discovered_systems', 'discovered_pii_locations', 'jit_records',
'scans'
5. Run the new container (if you’re using the same port as the container you backed up, rst stop the
backed-up container).
6. Inside the VM enter the command without line breaks:
Bash Copy
docker run ‐‐rm ‐‐link <container name>:<alias> ‐‐net <network id>
‐v bigid‐mongo‐backup:/backup mongo bash
‐c 'mongorestore ‐‐host <host ip>:<db port> ‐‐username
<username> ‐‐password <password> ‐‐authenticationDatabase admin
‐‐noIndexRestore
‐‐gzip backup/'
Example (enter the command without line breaks) and enter the username and password, as
required:
Do not share. BigID confidential and privileged information.
Bash Copy
8
docker run ‐‐rm ‐‐link bigid‐mongo:mongo ‐‐net <network id>
‐c 'mongorestore ‐‐host <host ip>:<db port> ‐‐username
<username> ‐‐password <password> ‐‐authenticationDatabase admin
‐‐noIndexRestore
‐‐gzip backup/'
Example (enter the command without line breaks) and enter the username and password, as
required:
Bash Copy
docker run ‐‐rm ‐‐link bigid‐mongo:mongo ‐‐net <network id>
‐v bigid‐mongo‐backup:/backup mongo bash
‐c 'mongorestore ‐‐host 192.168.99.100:27017 ‐‐username
<username> ‐‐password <password> ‐‐authenticationDatabase admin
‐‐noIndexRestore
‐‐gzip backup/'
To nd the host_ip value and network id value, run docker inspect <container name/id> and
under NetworkSettings/Networks are two relevant values: Gateway is the host_ip and NetworkID
7. Remove the backup volume:
docker volume rm bigid‐mongo‐backup
Powered by Document360
Do not share. BigID confidential and privileged information.
