When you set up and initiate a replica set in MongoDB, all databases and collections on the primary database are replicated to the secondary nodes. However, there is one crucial exception: the local database.

What is the local Database?

Each mongod instance maintains its own unique copy of the local database. It contains collections that store various information about that specific instance, and any replica set. Since it is not replicated, the local database serves as a centralized store for metadata and operational logs specific to that instance.

This design ensures that each instance can operate independently while still being part of the larger replica set.

Here are some key aspects of the local database:

Operational Independence

Each mongod instance relies on its local database for information that is essential to its operation but does not need to be shared with other instances. This includes details about the instance’s startup configuration, log files, and other server-specific data. By maintaining a unique local database, each instance can operate independently and manage its own settings and logs without interference from other nodes in the replica set.

Metadata Storage

The local database contains various collections that store metadata about the instance and the replica set. This metadata includes information about the instance’s configuration, such as network settings, storage paths, and process management options. It also includes logs and operational data that are crucial for troubleshooting and monitoring the instance’s performance.

Replication and Syncing

While the local database itself is not replicated, it plays a key role in the replication process. For example, the oplog.rs collection records all write operations performed on the primary instance. This collection is then used by secondary instances to replicate these operations and stay in sync with the primary. The local database, therefore, acts as a bridge between the individual instance’s operations and the larger replica set.

Server-Specific Data

Certain collections in the local database, such as startup_log, contain server-specific data that would not be relevant or useful to replicate to other instances. This data includes information about how the server was started, the command line options used, and the server’s build information. By keeping this data local, MongoDB ensures that each instance has access to its own operational history and configuration details.

Let’s dig into some of the import collections and operations of the local database:

The startup_log Collection

This collection stores detailed information about how the server was started. It includes data such as:

• startTime: The time when the server was started, which is useful for a number of administrative purposes.
• buildInfo: Information about the build of the MongoDB instance (this can differ in a replica set for example).
• cmdLine: The configuration for the server stored as a JSON object, detailing various parameters such as network settings, process management, replication, storage paths, and logging details.

Here’s an example of what the cmdLine object might look like:

"cmdLine": {
    "config": "/store/mongodb/node2.conf",
    "net": {
        "bindIp": "localhost",
        "port": 27022
    },
    "processManagement": {
        "fork": true
    },
    "replication": {
        "replSetName": "myReplSet"
    },
    "storage": {
        "dbPath": "/store/mongodb/node2"
    },
    "systemLog": {
        "destination": "file",
        "logAppend": true,
        "path": "/store/logs/node2/mongod.log"
    }
}

The oplog Collection

One of the most critical collections in the local database (especially for replica sets) is the oplog.rs collection, commonly known as the “oplog”. This is a capped collection that records every write operation to your MongoDB instance in a document.

By default, the oplog is capped to 5% of your disk space, but you can adjust this using the oplogSizeMB option in your configuration

MongoDB uses these oplog entries to replicate data. Since entries are stored in order, a secondary node can “replay” each entry to replicate the actions performed by the primary node. When the oplog reaches its maximum size, the oldest documents are deleted to make room for new entries.

Querying the oplog.rs Collection

You can query the oplog to see what actions have occurred, though you cannot manually modify the collection. This is a safety feature to ensure the smooth operation of your replica set. For example, you can run the following query to see inserts or updates:

> db.oplog.rs.find({ op: { $in: ["i", "u"] }, "ns": "testo.replo" })

This will return documents similar to the following:

{
    "lsid": {
        "id": new UUID("605abe18-4f00-4ec2-bfb0-6c13166fdf18"),
        "uid": ...
    },
    "txnNumber": Long("1"),
    "op": 'i',
    "ns": 'testo.replo',
    "ui": new UUID("d3184a0b-072b-4dbc-a95d-3de0961b57bb"),
    "o": { "_id": ObjectId("63c5f5e326fcbb455630de6a") },
    "o2": { "_id": ObjectId("63c5f5e326fcbb455630de6a") },
    "stmtId": 0,
    "ts": Timestamp({ t: 1673917925, i: 1 }),
    "t": Long("1"),
    "v": Long("2"),
    "wall": ISODate("2023-01-17T01:12:05.807Z"),
    "prevOpTime": { ts: Timestamp({ t: 0, i: 0 }), t: Long("-1") }
}

Each document represents a different kind of action performed by the server, with precise details about the time and changes made.

The op field identifies the type of action, which can be cross-referenced with MongoDB documentation for more details.

The Oplog Window and Replica Sets

The difference between the oldest and newest oplog entries is known as the “oplog window”. If a secondary node is disconnected, it uses this window to synchronize changes and catch up with the primary node. If the window is too old, the secondary node may need to restart from scratch to ensure no changes are missed.

Conclusion

Understanding the local database and its crucial collections like oplog.rs is fundamental for managing and troubleshooting MongoDB replica sets. By keeping a close eye on these components, you can ensure the integrity and consistency of your data across all nodes in the replica set.