MongoDB is one of the most popular databases and is one of the easiest NoSQL databases to set up. Oftentimes, developers want a quick environment to just test out an idea they have for an application or to try and figure out a good data model for their data without waiting for their Operations team to spin up the infrastructure. What can sometimes happen is these quick, one-off instances grow, and before you know it that little test DB is your production DB supporting your new app. For anyone who finds themselves in this situation, I encourage you to check out our Percona blogs as we have lots of great information for those both new and experienced with MongoDB. Don’t let the ease of installing MongoDB fool you into a false sense of security, there are things you need to consider as a developer before deploying MongoDB. Here are five things developers should know before deploying MongoDB in production.
Interested in running MongoDB without getting locked in? Find out how in this comprehensive white paper.
1) Enable Authentication and Authorization
Security is of utmost importance to your database. While gone are the days when security was disabled by default for MongoDB, it’s still easy to start MongoDB without security. Without security and with your database bound to a public IP, anyone can connect to your database and steal your data. By simply adding some important security configuration options to your configuration file, you can ensure that your data is protected. You can also configure MongoDB to utilize native LDAP or Kerberos for authentication. Setting up authentication and authorization is one of the simplest ways to ensure that your MongoDB database is secure. The most important configuration option is turning on authorization which enables users and roles and requires you to authenticate and have the proper roles to access your data.
1 2 3 | security: authorization: enabled keyfile: /path/to/our.keyfile |
2) Connect to a Replica Set/Multiple Mongos, Not Individual Nodes
MongoDB’s drivers all support connecting directly to a standalone node, a replica set, or a mongos for sharded clusters. Sometimes your database starts off with one specific node that is always your primary. It’s easy to set your connection string to only connect to that one node. But what happens when that one node goes down? If you don’t have a highly available connection string in your application configuration, then you’re missing out on a key advantage of MongoDB replica sets. Connect to the primary no matter which node it is. All of MongoDB’s supported language drivers support the MongoDB URI connection string format and implement the failover logic. Here are some examples of connection strings for PyMongo, MongoDB’s Python Driver, of a standalone connection string, a replica set, and an SRV record connection string. If you have the privilege to set up SRV DNS records, it allows you to standardize your connection string to point to an address without needing to worry about the underlying infrastructure getting changed.
Standalone Connection String:
1 | client = MongoClient('mongodb://hostabc.example.com:27017/?authSource=admin') |
Replica Set Connection String:
1 | client = MongoClient('mongodb://hostabc.example.com:27017,hostdef:27017,hostxyz.example.com/?replicaSet=foo&authSource=admin') |
SRV Connection String:
1 | client = MongoClient('mongodb+srv://host.example.com/') |
Post-script for clusters: If you’re just starting you’re usually not setting up a sharded cluster. But if it is a cluster then instead of using a replicaset connection you will connect to a mongos node. To get automatic failover in the event of mongos node being restarted (or otherwise being down) start them on multiple hosts and put them, comma-concatenated, in your connection string’s host list. As with replicasets you can use SRV records for these too.
3) Sharding Can Help Performance But Isn’t Always a Silver Bullet
Sharding is how MongoDB handles the partitioning of data. This practice is used to distribute load across more replicasets for a variety of reasons such as write performance, low-latency geographical writes, and archiving data to shards utilizing slower and cheaper storage. These sharding approaches are helpful in keeping your working set in memory because it lowers the amount of data each shard has to deal with.
As previously mentioned, sharding can also be used to reduce latency by separating your shards by geographic region, a common example if having a US-based shard, an EU-based shard, and a shard in Asia where the data is kept local to its origin. Although it is not the only application for shard zoning “Geo-sharding” like this is a common one. This approach can also help applications comply with various data regulations that are becoming more important and more strict throughout the world.
While sharding can oftentimes help write performance, that sometimes comes at the detriment of read performance. An easy example of a poor read performance would be if we needed to run a query to find all of the orders regardless of their origin. This find query would need to be sent to the US shard, the EU shard, and the shard in Asia, with all the network latency that comes with reading from the non-local regions, and then it would need to sort all the returned records on the mongos query router before returning them to the client. This kind of give and take should help you determine what approach you take to choosing a shard key and weighing its impact on your typical query patterns.
4) Replication ≠ Backups
MongoDB Replication, while powerful and easy to set up, is not a substitution for a good backup strategy. Some might think that their replica set members in a DR data center will be sufficient to keep them up in a data loss scenario. While a replica set member in a DR center will surely help you in a DR situation, it will not help you if you accidentally drop a database or a collection in production as that delete will quickly be replicated to your secondary in your DR data center.
Other common misconceptions are that delayed replica set members keep you safe. Delayed members still rely on you finding the issue you want to restore from before it gets applied to your delayed member. Are your processes that rock-solid that you can guarantee that you’ll find the issue before it reaches your delayed member?
Backups are just as important with MongoDB as they were with any other database. There are tools like mongodump, mongoexport, Percona Backup for MongoDB, and Ops Manager (Enterprise Edition only) that support Point In Time Recovery, Oplog backups, Hot Backups, full and Incremental Backups. As mentioned, Backups can be run from any node in your replica set. The best practice is to run your backup from a secondary node so you don’t put unnecessary pressure on your primary node. In addition to the above methods, you can also take snapshots of your data, this is possible as long as you pause writes to the node that you’re snapshotting by freezing the file system to ensure a consistent snapshot of your MongoDB database.
5) Schemaless is a Myth, Schemas Still Matter
MongoDB was originally touted as a schemaless database, this was attractive to developers who had long struggled to update and maintain their schemas in relational databases. But these schemas succeeded for good reasons in the early days of databases and while MongoDB allowed you the flexibility to not set up your schema and create it on the fly, this often led to some poor-performing schema designs and anti-patterns. There are lots of stories out in the wild of users not enforcing any structured schema on their MongoDB data models and running into various performance problems as their schema began to become unwieldy. Today, MongoDB supports JSON schema and schema validation. These approaches allow you to apply as much or as little structure to your schemas as is needed, so you still have the flexibility of MongoDB’s looser schema structure while still enforcing schema rules that will keep your application performing well and your data model consistent.
Another aspect that is affected by poor schema design in MongoDB is its aggregation framework. The aggregation framework lets you do more analytical query patterns such as sorting, grouping, and some useful things such as unwinding of arrays and supporting joins and a whole lot more. Without a good schema, these sorts of queries can really suffer poor performance.
MongoDB was also popular due to its lack of support for joins. Joins can be expensive and avoiding them allowed MongoDB to run quite fast. Though MongoDB has since added $lookup to support left outer joins, embedded documents are a typical workaround to this approach. This approach comes with its pros and cons. As with relational databases, embedding documents is essentially creating a One-to-N relationship, this is covered in greater detail in this blog. In MongoDB, the value of N matters, if it’s One-to-few (2-10), one-to-many,(10-1000) this can still be a good schema design as long as your indexes support your queries. When you get to one-to-tons(10000+) this is where you need to consider things like MongoDB’s 16 MB limit per document or using references to the parent document.
Examples of each of these approaches:
One-to-Few, consider having multiple phone numbers for a user:
1 2 3 4 5 6 7 | { "_id" : ObjectId("1234567890"), "name" : "John Doe", "phone" : [ { "type" : "mobile", "number" : "+1-585-555-5555" }, { "type" : "work", "number" : "+1-585-555-1111"} ] } |
One-to-Many, consider a parts list for a product with multiple items:
1 2 3 4 | { "_id" : ObjectId("123"), “Item” : “Widget”, “Price” : 100 } |
1 2 3 4 5 6 7 8 9 10 | { "_id" : ObjectId("0123456789"), "manufacturer" : "Percona", "catalog_number" : 123456, "parts" : [ { “item”: ObjectID("123")}, { “item”: ObjectID("456")}, { “item”: ObjectID("789")}, ... ] } |
One-to-Tons, consider a social network type application:
1 2 3 | { "_id" : ObjectId("123"), "username" : "Jane Doe" } |
1 2 3 | { "_id" : ObjectId("456"), "username" : "Eve DBA" } |
1 2 3 4 5 6 7 8 9 | { "_id" : ObjectId("9876543210"), "username" : "Percona", "followers" : [ ObjectID("123"), ObjectID("456"), ObjectID("789"), ... ] } |
Bonus Topic: Transactions
MongoDB supports multi-document transactions since MongoDB 4.0 (replica sets) and MongoDB 4.2 (sharded clusters). Transactions in MongoDB work quite similarly to how they work in relational databases. That is to say that either all actions in the transaction succeed or they all fail. Here’s an example of a transaction in MongoDB:
1 2 3 4 5 6 | rs1:PRIMARY> session.startTransaction() rs1:PRIMARY> session.getDatabase("percona").test.insert({today : new Date()}) WriteResult({ "nInserted" : 1 }) rs1:PRIMARY> session.getDatabase("percona").test.insert({some_value : "abc"}) WriteResult({ "nInserted" : 1 }) rs1:PRIMARY> session.commitTransaction() |
Transactions can be quite powerful if they are truly needed for your application, but do realize the performance implications as all queries in a transaction will wait to finish until the whole transaction succeeds or fails.
Takeaways:
While MongoDB is easy to get started with and has a lower barrier to entry, just like any other database there are some key things that you, as a developer, should consider before deploying MongoDB. We’ve covered enabling authentication and authorization to ensure you have a secure application and don’t leak data. We’ve highlighted using Highly available connection strings, whether to your replica set, a mongos node list, or utilizing SRV, to ensure you’re always connecting to the appropriate nodes. The balancing act of ensuring that when you select your shard key you consider the impact to both reads and writes and understand the tradeoffs that you are making. The importance of backups and to not rely on replication as a backup method was also covered. Finally, we covered the fact that schemas still matter with MongoDB, but you still have flexibility in defining how rigid it is. We hope this helps you have a better idea about things to consider when deploying MongoDB for your next application and to be able to understand it better. Thanks for reading!
Learn more about the history of Oracle, the growth of MongoDB, and what really qualifies software as open source. If you are a DBA, or an executive looking to adopt or renew with MongoDB, this is a must-read!
