Backup

Bacula Enterprise Only

This solution is only available for Bacula Enterprise. For subscription inquiries, please reach out to sales@baculasystems.com.

Backup Procedure

A MongoDB backup job is driven by the Bacula job level and the plugin parameters in the Fileset. Bacula scheduling, job level, Fileset scope, and plugin options define the backup plan and the operation to perform.

At the start of the job, the plugin performs these checks and preparation steps:

Read the Bacula job level, Fileset parameters, and optional config_file.
Build the MongoDB connection from uri or structured connection options.
Validate Java backend availability, MongoDB Java Driver access, and MongoDB Database Tools such as mongodump.
Inspect deployment topology: standalone, replica set, or sharded cluster through mongos.
Read MongoDB server version, feature compatibility version, source identity, and selected databases or collections.
Validate authentication, TLS, backup privileges, and optional security-capture privileges.
Prepare the local working path for small control files such as manifests and recovery-chain metadata.
For sharded clusters, validate the balancer guardrail before dumping data.

After preflight succeeds, the plugin produces logical MongoDB artifacts and sends them to Bacula under the /@mongodb virtual path. The Bacula Storage Daemon stores those artifacts according to the configured Pool, Volume, and Storage resources.

What Is Protected

The plugin protects MongoDB data as logical artifacts:

Selected databases and collections are dumped with mongodump archive output.
Collection metadata, indexes, and options are included in the dump archive generated by MongoDB Database Tools.
manifest.json records source topology, MongoDB version, selected scope, checksums, dump files, and plugin decisions.
Replica-set recovery-chain backups include chain-index.json and oplog BSON segments.
Sharded-cluster backups include cluster metadata used for restore planning and operator validation.
Users and custom roles are captured unless skip_security=yes is configured.

The plugin does not perform filesystem-level copies of MongoDB data files. The backup is a logical MongoDB backup designed for restore through MongoDB tools and driver operations.

If filesystem-level copies of MongoDB configuration files or data files are required, protect them with a separate standard File Daemon job. Keep MongoDB Plugin jobs focused on logical database data.

Backup File Structure

A typical Bacula backup contains files similar to:

MongoDB files protected by Bacula

/@mongodb/manifest.json
/@mongodb/control/stream-plan/appdb.json
/@mongodb/dump/appdb.archive
/@mongodb/control/stream-plan/analytics.json
/@mongodb/dump/analytics.archive
/@mongodb/chain-index.json
/@mongodb/oplog/local/oplog.rs.bson
/@mongodb/cluster-metadata.json

manifest.json is always expected. control/stream-plan/<database>.json is a small restore planning record written immediately before the corresponding dump/<database>.archive stream. Restore uses that FileIndex order to stream large dump archives directly into mongorestore instead of staging complete dump files in the File Daemon working path. chain-index.json and oplog/local/oplog.rs.bson are used by replica-set recovery chains. cluster-metadata.json is used by sharded-cluster restore planning.

Full Backups

A Full backup creates a complete logical artifact for the selected scope. For a standalone deployment, this is the standard consistency point available from mongodump for the selected databases and collections.

For a replica set, the Full backup also records the initial chain boundary. This boundary allows later Incremental and Differential jobs to capture and validate oplog ranges for point-in-time recovery inside the available oplog window.

For a sharded cluster, the plugin connects through mongos and captures the selected data plus routing metadata. The sharded backup is coordinated by the plugin guardrails, including the balancer check, but it is still a logical MongoDB backup rather than a config-server disaster-recovery image.

Incremental Backups

For replica sets, an Incremental backup captures oplog changes since the previous accepted chain boundary. The plugin resolves the previous Bacula artifact metadata, validates the source identity and chain digest, and confirms that the required starting timestamp is still present in the MongoDB oplog. If the oplog has rolled past the required boundary, the job must be treated as a new Full backup instead of producing an unsafe Incremental artifact.

For standalone deployments, Incremental jobs produce a full-style logical artifact because there is no replica-set oplog guarantee to use.

For sharded clusters, Incremental jobs also produce full-style logical artifacts because a single replica-set oplog chain does not represent a complete sharded cluster point-in-time state.

Incremental Artifact Lifecycle (What Is Stored, Where, and Why)

Replica-set Incremental chains are managed in two layers:

Bacula catalog/storage artifacts (authoritative recovery payload)
Local incremental-state snapshots (lightweight auto-resume cache)

The backup payload stored by Bacula remains under /@mongodb and includes the normal Full/Incremental chain files, such as:

manifest.json
chain-index.json (when chain metadata applies)
oplog/local/oplog.rs.bson (for replica-set delta segments)
small stream-plan records under control/stream-plan/
dump archive files under dump/

The dump archives are stored in Bacula catalog/storage as normal plugin files, but current Bacula restore streams them directly from Bacula into mongorestore --archive for normal live restore paths. They are not used to size the File Daemon working directory.

However, the plugin also keeps a local resume cache to resolve Incremental and Differential continuity without requiring operators to pass a manual resume_from path:

Base path: path parameter (or Bacula working path when path is not set)
MongoDB workspace root: <path>/mongodb/
Per-job family directory: <path>/mongodb/<JobName>/
Incremental cache root: <path>/mongodb/<JobName>/.bacula-incremental-state/
Snapshot directory format: <createdEpoch>-job-<JobId>/
Snapshot content: lightweight manifest.json only

The local cache exists to answer one question quickly and safely at job start: which previously accepted artifact boundary should this Bacula Incremental or Differential job resume from?

MongoDB incremental safety needs specific metadata from the previous accepted artifact, so this cache stores:

source fingerprint / replica set identity
exact segmentEndInclusive oplog position
optional boundary digest (segmentEndDigest) to detect rollbacks/divergence
scope compatibility (databases/selection)
security mode compatibility

What Happens If Local Incremental-State Is Missing

If no compatible local snapshot is found, the plugin cannot safely compute a resume boundary from Bacula job metadata alone. In that case:

The requested Incremental/Differential run is downgraded to a Full-style logical backup for that execution.
The job can still succeed, but the replica-set oplog chain is not extended by that run.
Bacula logs include an explicit warning that no compatible resume snapshot was found.

This is a safety behavior, not silent data loss. The backup still produces a usable Full-style artifact.

Retention and Pruning of Local Snapshot Cache

The plugin manages the local resume cache automatically. It keeps only the state that can still be useful for future Incremental or Differential jobs and prunes obsolete entries when they are no longer needed, typically after a new Full backup establishes a fresh recovery-chain boundary.

This automatic pruning controls local metadata growth while preserving enough chain history for normal scheduling patterns. It affects only the lightweight .bacula-incremental-state cache and does not remove Bacula catalog/storage artifacts.

Architectures That Support True Incremental Chains

True oplog-chain Incremental/Differential behavior is supported only when the source is a writable replica-set primary.

Replica set primary: true Incremental/Differential chain with oplog boundaries and digest continuity checks.
Standalone: Incremental/Differential requests run as Full-style logical backups.
Sharded through mongos: Incremental/Differential requests run as Full-style logical backups (single replica-set oplog is not a complete cluster-wide point-in-time source).

Differential Backups

For replica sets, a Differential backup captures oplog changes since the latest Full backup boundary. This gives Bacula administrators a shorter restore chain: the Full artifact plus the selected Differential artifact.

For standalone and sharded deployments, Differential jobs produce full-style logical artifacts for the same consistency reasons described for Incremental jobs.

Differential jobs in Bacula mode use the same local resume-cache model described in Incremental Artifact Lifecycle (What Is Stored, Where, and Why). When that cache is missing or incompatible, Differential also falls back to a Full-style logical artifact for that run.

Sharded Backup Procedure

A sharded backup must be performed through mongos. The plugin records routing information to support restore planning and validates that the balancer is not moving chunks while the backup is running.

The operational sequence is:

Configure the Fileset to connect to mongos.
Stop or disable the balancer according to the local MongoDB operating procedure. This can be a manual runbook step or a Bacula RunScript hook; see Sharded Balancer RunScript Hooks.
Start the Bacula backup job.
The plugin confirms the deployment is sharded and checks the balancer state.
mongodump reads the selected application databases through mongos.
The plugin records cluster metadata, manifest details, and checksums.
Re-enable the balancer after the Bacula job completes and local validation is finished.

The plugin does not claim to rebuild an entire sharded cluster config-server state automatically. The sharded artifact is intended for application-data restore through mongos with explicit metadata checks and operator control.

Security Capture

By default, the plugin captures MongoDB users and custom roles that belong to the selected restore scope. This is useful for isolated disaster recovery, test restores, and complete application recovery.

Use skip_security=yes when user and role management is handled by a separate identity or configuration process. Use include_admin=yes only when the Bacula backup must include administrative objects for a controlled isolated-DR workflow.

Failure Behavior

The plugin writes enough metadata for restore validation before accepting an artifact as usable. Checksums, manifest entries, source identity, topology, and recovery-chain boundaries are recorded so that restore can reject incomplete, corrupt, or inconsistent selections.

When a command fails, a required artifact is missing, the oplog boundary is no longer available, or a sharded guardrail is violated, the plugin reports the condition in the Bacula job log and fails the job according to abort_on_error. Production jobs should keep abort_on_error=yes.

Backup Example

Backup JobLog example

| 127.0.0.1-dir JobId 1: Start Backup JobId 1, Job=pluginTest.2026-05-12_13.23.57_03                   |
| 127.0.0.1-dir JobId 1: Connected to Storage "File" at 127.0.0.1:8103 with TLS                        |
| 127.0.0.1-dir JobId 1: Created new Volume="TEST-2026-05-12:0", Pool="Default", MediaType="File" in catalog. |
| 127.0.0.1-dir JobId 1: Using Device "FileStorage" to write.                                          |
| 127.0.0.1-dir JobId 1: Connected to Client "127.0.0.1-fd" at 127.0.0.1:8102 with TLS                 |
| 127.0.0.1-fd JobId 1: Connected to Storage at 127.0.0.1:8103 with TLS                                |
| 127.0.0.1-sd JobId 1: Labeled new Volume "TEST-2026-05-12:0" on File device "FileStorage" (/tmp/regress/tmp). |
| 127.0.0.1-sd JobId 1: Wrote label to prelabeled Volume "TEST-2026-05-12:0" on File device "FileStorage" (/tmp/regress/tmp) |
| 127.0.0.1-fd JobId 1: mongodb: Jar Version: 0.1.0 | Java version: 21.0.10 | Java Pid: 297566         |
| 127.0.0.1-fd JobId 1: mongodb: Preparing backup artifact workspace at /tmp/regress/working/mongodb/pluginTest.2026-05-12_13.23.57_03/mongodb-bacula-backup-1-17338533992950670230 using hosts=127.0.0.1:27043, replicaSet=rs0, directConnection=true, tls=false |
| 127.0.0.1-fd JobId 1: mongodb: Connected to MongoDB source with topology replica-set primary (connected member localhost:27017) |
| 127.0.0.1-fd JobId 1: mongodb: Doing backup of a selection of databases: appdb                       |
| 127.0.0.1-fd JobId 1: mongodb: Doing backup of all collections within each selected database.        |
| 127.0.0.1-fd JobId 1: mongodb: Creating dump for database appdb                                      |
| 127.0.0.1-fd JobId 1: mongodb: Emitted Bacula restore stream plan control/stream-plan/appdb.json before archive dump/appdb.archive (security: false, collection scoped: false, sharded: false). |
| 127.0.0.1-fd JobId 1: mongodb: Streamed database appdb directly to Bacula as appdb.archive           |
| 127.0.0.1-fd JobId 1: mongodb: Backup manifest written                                               |
| 127.0.0.1-fd JobId 1: mongodb: Backup artifact creation completed for 1 databases                    |
| 127.0.0.1-fd JobId 1: mongodb: Persisted Bacula incremental resume snapshot at /tmp/regress/working/mongodb/pluginTest.2026-05-12_13.23.57_03/.bacula-incremental-state/1778585042-job-1 |
| 127.0.0.1-sd JobId 1: Elapsed time=00:00:03, Transfer rate=2.846 K Bytes/second                      |
| 127.0.0.1-sd JobId 1: Sending spooled attrs to the Director. Despooling 1,964 bytes ...              |
| 127.0.0.1-dir JobId 1: Bacula 127.0.0.1-dir 18.2.4 (10Mar26):
  Build OS:               x86_64-pc-linux-gnu ubuntu 24.04
  JobId:                  1
  Job:                    pluginTest.2026-05-12_13.23.57_03
  Backup Level:           Full
  Client:                 "127.0.0.1-fd" 18.2.4 (10Mar26) x86_64-pc-linux-gnu,ubuntu,24.04
  FileSet:                "FS_MONGODB" 2026-05-12 13:23:56
  Pool:                   "Default" (From Job resource)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "File" (From Job resource)
  Scheduled time:         12-may-2026 13:23:57
  Start time:             12-may-2026 13:23:59
  End time:               12-may-2026 13:24:03
  Elapsed time:           4 secs
  Priority:               10
  FD Files Written:       5
  SD Files Written:       5
  FD Bytes Written:       6,380 (6.380 KB)
  SD Bytes Written:       8,538 (8.538 KB)
  Rate:                   1.6 KB/s
  Software Compression:   27.9% 1.4:1
  Comm Line Compression:  0.5% 1.0:1
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               yes
  Volume name(s):         TEST-2026-05-12:0
  Volume Session Id:      1
  Volume Session Time:    1778585036
  Last Volume Bytes:      9,487 (9.487 KB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK |
| 127.0.0.1-dir JobId 1: Begin pruning Jobs older than 6 months .                                      |
| 127.0.0.1-dir JobId 1: No Jobs found to prune.                                                       |
| 127.0.0.1-dir JobId 1: Begin pruning Files.                                                          |
| 127.0.0.1-dir JobId 1: No Files found to prune.                                                      |
| 127.0.0.1-dir JobId 1: End auto prune.                                                               |