Do you need to do a little New Years cleaning?

Well you are in luck … MongoDB’s Aggregation Framework makes it easy to archive your collection’s documents.

You can use the $out or $merge stages (along with just about any other stages) to construct a complex pipeline of “cleaning” or “transforming” for your documents. Then, these documents can be inserted into a new or existing target collection.

A Cleanup Pipeline

As a simple example, this pipeline would:

Stage One

$match all the documents older than January 1^st 2023

Stage Two

Modify them (using $addFields) … adding two new archive fields (archived and archivedDate)

Stage Three

Copy them to the archive database’s taco collection via $merge:

> db.myCollection.aggregate([
  {
    "$match": {
      "createDate": { "$lt": ISODate("2023-01-01") }
    }
  },
  {
    "$addFields": {
      "archived": true,
      "archivedDate": new Date()
    }
  },
  { 
    "$merge": {
      "into": { "db": "archive", "coll": "taco" }
    }
  }
])

Since this example uses a $merge stage it can import into an existing collection, if you choose to use $out it will replace the target collection.

Clean Everything Up

After the transfer completes, you can confirm everything looks fine by querying the archive database’s collection, and then delete those documents in the source collection.

Doesn’t everything feel so much more roomy now?