MongoDB Indexing Best Practices

Version 1.0 – September 2026

Introduction

In the world of data, speed is a survival skill—just as a honeybee must locate the richest bloom before the wind changes, a modern application must retrieve the right document before a user’s patience runs out. MongoDB’s flexible document model gives developers the freedom to store rich, nested data, but that freedom can become a double‑edged sword when queries start scanning millions of records. A well‑crafted index is the equivalent of a bee’s internal map of flower fields: it tells the engine exactly where to fly, cutting latency from seconds to milliseconds and conserving precious compute “nectar.”

For teams building real‑time dashboards, IoT sensor streams, or AI agents that learn from continuous data feeds, the cost of a missing or mis‑designed index is measurable in both dollars and ecological impact. A 2024 benchmark from MongoDB Inc. shows that a properly indexed read can be 30‑to‑100× faster than a collection scan, while the same query without an index can consume up to 85 % more CPU and four times the I/O bandwidth. Those numbers translate directly into server‑farm energy consumption—a factor that matters when you’re trying to keep the planet (and the bees) thriving.

This guide walks you through the most powerful index types—compound, multikey, text, and TTL—showing when to use each, how to tune them, and what pitfalls to avoid. We’ll sprinkle in real‑world examples, concrete performance metrics, and occasional analogies to pollinator behavior, so you can build MongoDB schemas that are both fast and sustainable.

1. Foundations: How MongoDB Indexes Work

Before diving into specialized indexes, let’s recap the mechanics that underlie every index in MongoDB.

1.1 B‑Tree Structure

MongoDB’s default indexes are B‑tree structures stored on disk in the WiredTiger storage engine. Each node holds a range of keys and pointers to child nodes, guaranteeing O(log n) lookup time. For a collection of 10 million documents, a single‑field index typically adds 1‑2 GB of storage (roughly 10‑20 % of the collection size) and consumes a similar amount of RAM when the hot portion of the index is cached.

1.2 Index Prefixes and Selectivity

Selectivity is the proportion of distinct values a field holds relative to the total number of documents. High selectivity (e.g., a UUID field) yields a small index prefix and excellent query discrimination. Low selectivity (e.g., a boolean flag) may not justify an index unless combined with other fields.

A quick rule of thumb: if the selectivity is < 5 %, consider a compound index that adds a more selective field, or use a partial index (see partial-indexes).

1.3 The Query Planner

When you run a query, MongoDB’s query planner evaluates every available index and picks the one with the lowest estimated cost. You can inspect its decision with db.collection.explain("executionStats"). The output includes:

• totalDocsExamined – how many documents were scanned.
• totalKeysExamined – how many index entries were inspected.
• executionTimeMillis – wall‑clock time.

A well‑indexed query typically shows totalDocsExamined ≈ totalKeysExamined ≪ collectionSize. If you see a large disparity, you likely need a new index.

2. Single‑Field Indexes: The Building Blocks

Even though this guide focuses on compound, multikey, text, and TTL indexes, a solid foundation of single‑field indexes is essential.

2.1 When to Create One

• Equality filters on high‑cardinality fields (_id, UUID, email).
• Range queries on timestamp or numeric fields (createdAt, price).
• Sorting on a field that is also filtered (status + sort: {updatedAt: -1}).

2.2 Example: Timestamp Index for IoT Sensors

db.readings.createIndex({ deviceId: 1, ts: -1 });

Why this works:

• deviceId provides high selectivity (many devices).
• ts is sorted descending, allowing efficient retrieval of the most recent readings without an extra in‑memory sort.

A benchmark on a 50 M‑document collection showed query time drop from 2 200 ms (collection scan) to 12 ms (index scan) and RAM usage fell by 73 %.

2.3 Index Size Considerations

A single‑field index on a 64‑bit integer consumes roughly 16 bytes per entry (key + pointer). For 100 M documents, that’s ≈ 1.6 GB. Keep an eye on the storageSize metric in db.collection.stats() and ensure your server has enough wiredTigerCacheSizeGB to hold the hot index portion.

3. Compound Indexes: Combining Fields for Precision

A compound index orders documents by multiple fields in a single B‑tree. It can satisfy both filter and sort requirements, dramatically reducing the need for in‑memory sorting or multiple index scans.

3.1 Index Prefix Rules

MongoDB can use the prefix of a compound index. For an index {a: 1, b: -1, c: 1}:

• Queries on {a: …} or {a: …, b: …} can use the index fully.
• Queries on {b: …} alone cannot use the index (unless a separate index exists).

Understanding this rule is crucial for query planning.

3.2 Designing the Right Order

1. Equality fields first – fields used with $eq or $in.
2. Range fields next – fields used with $gt, $lt, $gte, $lte.
3. Sort fields last – if the query also sorts, place the sort field after the equality/range fields, matching the sort direction.

Real‑World Example: Bee Observation API

Suppose you run an API that stores observations of bee species:

{
  _id: ObjectId,
  species: "Apis mellifera",
  location: { type: "Point", coordinates: [ -122.42, 37.77 ] },
  observedAt: ISODate("2026-09-20T14:23:00Z"),
  collectorId: ObjectId,
  notes: "Near lavender field"
}

A frequent query: “Give me all observations of Apis mellifera collected by user X, sorted by newest first.”

Optimal compound index:

db.observations.createIndex(
  { species: 1, collectorId: 1, observedAt: -1 },
  { name: "species_collector_observedAt" }
);

Why this order?

• species and collectorId are equality filters → placed first.
• observedAt is used for sorting → placed last with descending order.

A performance test on 5 M observation documents showed:

3.3 Covered Queries

If the index covers all fields referenced in the query (including the projection), MongoDB can return results directly from the index without fetching the full document. This is called a covered query and can shave off up to 60 % of I/O.

// Covered query: only need _id and observedAt
db.observations.find(
  { species: "Bombus impatiens", collectorId: ObjectId("...") },
  { _id: 1, observedAt: 1 }
).hint("species_collector_observedAt");

Because both fields exist in the index, the query never touches the collection.

3.4 Pitfalls

• Index bloat: Adding too many fields inflates index size. A 4‑field compound index can be 2‑3× larger than a single‑field index.
• Write penalty: Each additional indexed field adds to the write path. In a high‑throughput ingestion pipeline (e.g., sensor data arriving at 10 k writes/sec), each extra index can increase latency by 0.5‑1 ms per write. Use partial-indexes or hashed-indexes where appropriate.

4. Multikey Indexes: Indexing Inside Arrays

MongoDB automatically creates a multikey index when you index a field that holds an array. Each array element becomes a separate index entry, allowing queries that match any element.

4.1 How Multikey Works

Given a document:

{
  _id: 1,
  tags: ["pollination", "honey", "colony"]
}

A multikey index on {tags: 1} stores three entries: ("pollination", 1), ("honey", 1), ("colony", 1). Queries like {tags: "honey"} can now use the index.

4.2 Size Implications

Each array element creates an index entry. For a field that averages 10 elements per document, a 1 M‑document collection will generate ≈ 10 M index entries. This can increase index size by 5‑10 × compared to a single‑field index.

Rule of thumb: If an array is unbounded (e.g., a log of events), avoid indexing it directly. Instead, consider referencing a separate collection or using a bucketed approach.

4.3 Compound Multikey Indexes

MongoDB does not allow a compound index where more than one field is multikey. Attempting to create {tags: 1, categories: 1} where both fields are arrays results in an error:

Cannot create compound index with more than one multikey path

If you need to query on two array fields, you must create separate indexes and rely on the query planner’s index intersection (available since MongoDB 4.2). However, index intersection can be slower than a single well‑designed index, so test with explain.

4.4 Example: AI Agent Action Log

Suppose an autonomous AI agent logs its actions as an array of objects:

{
  _id: ObjectId,
  agentId: "alpha-1",
  actions: [
    { type: "move", target: "flower-23", ts: ISODate("2026-09-26T10:00:00Z") },
    { type: "collect", amount: 5, ts: ISODate("2026-09-26T10:00:02Z") },
    // ... potentially hundreds per session
  ]
}

We frequently query for all “collect” actions across agents. A multikey index on the nested field works:

db.agentLogs.createIndex({ "actions.type": 1 });

Performance tip: To keep the index size manageable, store only the most recent N actions per document (e.g., a sliding window of 100) or split each action into its own document in a separate agent_actions collection.

A benchmark with 2 M action logs (average 50 actions each) showed:

The separate collection approach reduces index bloat and simplifies TTL management (see Section 6).

4.5 Index Bounds and $elemMatch

When you need to match multiple conditions on the same array element, use $elemMatch. This enables the query planner to apply index bounds on the multikey index, avoiding a full scan of all array elements.

db.agentLogs.find({
  actions: {
    $elemMatch: { type: "collect", amount: { $gt: 4 } }
  }
});

With the multikey index on "actions.type" and "actions.amount", the planner can efficiently locate matching elements.

5. Text Indexes: Full‑Text Search Inside Documents

MongoDB’s text index enables language‑aware search over string content. It tokenizes text, removes stop words, and stores term frequencies, allowing queries with $text and relevance scores.

5.1 Creating a Text Index

db.articles.createIndex(
  { title: "text", body: "text", tags: "text" },
  {
    name: "articleFullText",
    default_language: "english",
    weights: { title: 10, tags: 5, body: 1 }
  }
);

Weights boost the importance of certain fields. In the example, a match in title contributes ten times more to the relevance score than a match in body.

5.2 Query Syntax

db.articles.find(
  { $text: { $search: "\"pollinator health\" -pesticide" } },
  { score: { $meta: "textScore" } }
).sort({ score: { $meta: "textScore" } });

• "pollinator health" – phrase search.
• -pesticide – exclude documents containing “pesticide”.

The $meta: "textScore" projection returns a relevance score that can be used for ranking.

5.3 Performance Characteristics

A text index is larger than a regular B‑tree because it stores term‑document mappings. For a 1 GB article collection with an average of 300 words per document, the text index may occupy 2‑3 GB. However, queries that would otherwise require a collection scan become O(log n) with a typical latency of 15‑30 ms.

Benchmark (1 M articles, 500 KB each):

5.4 Limitations & Best Practices

• One text index per collection – you cannot have separate text indexes on different field subsets.
• No partial text indexes – but you can combine a partial filter expression with a text index to limit the indexed documents (e.g., only status: "published").
• Stemming and stop words – the default language determines stemming rules. For scientific data about bees, you may want to disable stemming on Latin names (Apis mellifera) by using a custom analyzer or storing them in a separate field with a keyword index.

5.5 Bridging to Conservation

Researchers often need to search through thousands of field notes for specific phrases like “queen failure” or “pesticide exposure”. A well‑tuned text index can return relevant documents in under 50 ms, enabling real‑time dashboards that alert beekeepers and policy makers to emerging threats.

6. TTL Indexes: Automatic Data Expiration

TTL (Time‑To‑Live) indexes are a powerful tool for managing data that becomes irrelevant after a certain period—think sensor readings, session logs, or temporary AI inference results.

6.1 How TTL Works

Create an index on a date field with the expireAfterSeconds option. MongoDB’s background thread checks the index every 60 seconds (configurable via ttlMonitorSleepSecs) and removes documents whose date value is older than the threshold.

db.tempReadings.createIndex(
  { createdAt: 1 },
  { expireAfterSeconds: 86400 } // 24 hours
);

All documents where createdAt < now() - 24 h are automatically deleted.

6.2 Use Cases

6.3 Performance Impact

TTL deletions are non‑blocking and performed in batches (default batch size = 100). However, they generate write load because each deletion is a write operation. In high‑throughput scenarios (e.g., 500 k inserts per minute), the TTL monitor can consume ≈ 2‑5 % of CPU.

Best practice: If you expect a deletion rate > 10 k docs/sec, consider a sharded collection where each shard handles its own TTL deletions, or run a custom cleanup job using deleteMany with an indexed filter.

6.4 Monitoring TTL

Use the db.currentOp({ "command.ttlMonitor": { $exists: true } }) command to see the monitor’s activity. The ttlMonitor log entries look like:

[TTLMonitor] Deleted 12,345 documents from collection tempReadings

If you notice long gaps between deletions, verify that the expireAfterSeconds value is not set to a negative number and that the indexed field contains valid BSON dates.

6.5 Example: Bee Hive Temperature Logs

db.hiveTemps.createIndex(
  { recordedAt: 1 },
  { expireAfterSeconds: 259200 } // 3 days
);

A beekeeping app stores temperature readings every 5 minutes. After 3 days, the data is archived elsewhere. The TTL index keeps the working collection under 500 k documents, ensuring queries for “last 24 hours” remain fast (< 10 ms) while older data is safely pruned.

7. Index Maintenance: Monitoring, Rebuilding, and Size Management

Even the best index design can degrade over time due to fragmentation, document growth, or changing query patterns. Regular maintenance keeps performance predictable.

7.1 Detecting Fragmentation

Run db.collection.stats({ scale: 1 }) and compare storageSize vs size. A large gap (> 30 %) indicates fragmentation. For indexes, examine indexSizes:

db.observations.stats({ scale: 1 }).indexSizes

If an index’s size has grown disproportionately to its entry count, consider rebuilding it.

7.2 Rebuilding Indexes

MongoDB 5.0 introduced the reIndex command, but it locks the collection. For production, use online index builds (available since 4.2) with the background: true option:

db.observations.reIndex({ background: true });

Alternatively, create a new index with a different name, drop the old one, and rename the new index (via collMod). This avoids a full collection lock.

7.3 Index Size Reduction

If an index contains many null entries (e.g., optional fields), use a partial index:

db.events.createIndex(
  { eventType: 1, ts: -1 },
  { partialFilterExpression: { eventType: { $exists: true } } }
);

Partial indexes store only documents that match the filter, often cutting size by 40‑60 %.

7.4 Automated Monitoring

Set up a MongoDB Atlas alert or a self‑hosted script that runs:

db.adminCommand({ getParameter: 1, ttlMonitorSleepSecs: 1 });

and checks the metrics.indexes.total and metrics.indexes.missRatio in the MongoDB Monitoring Service (MMS). A miss ratio above 0.2 (20 % of queries not using an index) signals a need to revisit index design.

8. Bee Analogy: Efficient Foraging as a Metaphor for Index Design

Bees use a waggle dance to communicate the location of the most nectar‑rich flowers. The dance encodes distance, direction, and quality, allowing the colony to allocate foragers optimally. MongoDB indexes serve a similar purpose: