MongoDB in production

Practices for MongoDB behind a product you have to keep running, and for a studio shipping many apps that should not each rediscover the same mistakes.

MongoDB is forgiving early and unforgiving late. A schema that works fine at ten thousand documents can be unfixable at ten million, because by then the bad shape is in production data rather than in code. The expensive decisions are made in week one.

The one rule

Design for your queries, not for your data.

In a relational database you normalize first and optimize later. In MongoDB that order is inverted: the schema is a cache of your access patterns, so you cannot design it before you know them.

Before writing a single collection, write down the ten queries the app will actually run, with their frequency and their latency budget. That list is the design document. If you cannot produce it, you are not ready to model the data.

The four decisions that matter

Everything else is detail. These are the ones that are expensive to reverse.

1. Embed or reference

The default is embed, and the exceptions are precise.

Embed when all of these hold:

• The data is read together with its parent, most of the time
• It is bounded - there is a real ceiling, and you can state it
• It belongs exclusively to the parent
• It is updated in the same operation as the parent

Reference when any of these hold:

• The set grows without limit (orders, events, messages, audit entries)
• It is queried independently of the parent
• It is shared by many parents
• It changes on a different cadence, or by a different actor

The failure mode is the unbounded array. Comments inside a post, orders inside a customer, events inside a session. It works beautifully in development and then a document crosses 16MB and every write to it fails, permanently, in production. Long before that it is already slow, because the whole document moves on every update and the working set stops fitting in RAM.

If you cannot state a maximum, do not embed.

Full treatment, including the standard patterns (subset, computed, bucket, extended reference, schema versioning), in references/data-modeling.md.

2. Multi-tenancy model

Three options, and the choice is close to irreversible once real customer data exists:

Model	Use when	Main risk
Shared collection + tenantId	Default. Many tenants, similar shape, ordinary isolation needs.	One missing filter leaks across tenants.
Database per tenant	Few, high-value, compliance-driven tenants.	Index and connection count. Clusters degrade past a few thousand databases.
Collection per tenant	Rarely the right answer.	Combines the operational cost of both with the benefits of neither.

For a consultancy shipping many apps, shared collection with tenantId is the right default, with a hybrid escape hatch: route the one enterprise customer who demands isolation to their own database later.

The critical part is not the model, it is the enforcement. Tenant isolation must be structural, not a rule developers remember. One data-access layer that injects the tenant filter, and a query path that is incapable of running without one.

references/multi-tenancy.md has the enforcement pattern, index implications, and the test that proves isolation.

3. Indexes

The ordering rule for compound indexes is ESR: Equality first, then Sort, then Range.

// query
db.orders.find({ tenantId: t, status: "paid" }).sort({ createdAt: -1 })

// index
{ tenantId: 1, status: 1, createdAt: -1 }
//  E           E          S

Get the order wrong and MongoDB still uses the index, but adds an in-memory sort that fails outright past 32MB. One well-ordered compound index almost always beats several single-field ones.

Indexes are not free. Each one is written on every insert and update, and holds RAM. An unused index is a permanent tax.

references/indexing.md covers ESR in depth, explain() reading, index hygiene, and when and how to shard.

4. Security posture

The defaults are not safe for production, and the two that cause real breaches:

Exposed to the network. Every public MongoDB incident is the same story: a port open to the internet with no authentication. Bind to private addresses, put it in a VPC, allowlist application hosts only.

Query injection through operator objects. If a JSON request body reaches a query unvalidated, {"password": {"$gt": ""}} logs in as anybody. Validate into typed models before anything reaches the driver.

references/security.md covers RBAC, encryption, injection, auditing, and backup verification.

Starting a new app

A repeatable baseline, in order:

1. Write the query list. Ten queries, with frequency and latency budget.
2. Decide the tenancy model before the first collection exists.
3. Model from the queries. Embed by default, reference the unbounded.
4. Declare indexes in code, versioned with the app, not created by hand in a

shell. Every index needs a comment saying which query it serves.

1. Add $jsonSchema validation to every collection. It is the only thing

standing between you and a field that is a string in 3% of documents.

1. Add schemaVersion to every document from day one. Retrofitting it onto

live data is far harder than never needing it.

1. Create a least-privilege user per application. Never root.
2. Verify a restore, not just that backups run.

Steps 5 and 6 are the ones people skip and regret. Both cost minutes now and weeks later.

Anti-patterns that cost the most

In rough order of damage:

• Unbounded arrays. Hits the 16MB wall, unfixable in place.
• Missing tenant filter. A data breach, not a bug.
• Using MongoDB like SQL. Fully normalized collections joined with

$lookup on every read gives you the worst of both models.

• Sequential shard key. Monotonic keys (timestamps, ObjectIds) send every

write to the same shard, so you buy a cluster and get one node.

• No schema validation. Every consumer downstream grows defensive code for

shapes that should never have been written.

• Indexes created by hand in production. They exist on one cluster, are

absent in staging, and disappear on restore.

Maintainability

MongoDB is the sharpest case in this catalogue, because its expensive decisions are the ones you cannot revise by editing code. The shape is in production data.

Know which doors are one-way. Embed versus reference, and the tenancy model, are permanent the moment real customer data exists in that arrangement. Everything else here is a two-way door and should be decided quickly. Spend the deliberation where reversal actually costs something, and write down what you assumed, because the assumption is what will stop holding.

schemaVersion is the reversibility mechanism. It is what converts an irreversible shape into a migratable one: read code handles N and N-1, a background job moves documents, support for N-1 is dropped. Adding it on day one costs a field. Retrofitting it onto live data means you cannot tell what shape anything is without a full scan.

Every index needs an owner and an expiry. Declared in code with a comment naming the query it serves, so the answer to "can we drop this" is readable rather than archaeological. Then actually drop them: $indexStats shows which have not been used, and an unused index is a permanent tax paid on every write and in RAM. Indexes created by hand in a production shell are the version of this that cannot be reviewed, cannot be recreated, and vanish on restore.

Every denormalized field needs an owner and a reconciliation job. The extended reference and computed patterns trade write cost for read speed, and the part people skip is that a copy without a named update path drifts silently into being wrong. If you cannot say what recomputes it, do not store it.

Enforcement over discipline. The tenant filter is the example that matters: a rule developers remember is a data breach with a delay on it. One data-access layer whose query path cannot be constructed without a tenant makes the leak unwritable. $jsonSchema on every collection is the same move applied to shape. Both convert a review comment into a mechanism.

See the maintainability skill for reversible versus irreversible decisions and the enforcement ladder.

Copy: no em-dashes

Never use an em-dash (U+2014, the long dash) in user-facing strings or in documentation you generate. It is the loudest tell that text was generated. A full stop or a colon is almost always the better edit.

Sources

See references/sources.md for the reading list and for which claims here are opinion rather than documentation.