What is MongoDB Sharding?
Let us understand the need for and importance of database sharding with the help of an example.
Assume there is a database collection (with no sharding) of 50,000 employees working in a reputed Multi-National Company. The company maintains a database record of all the freshers and experienced employees in a single database.
Your task is to access a profile of an employee from the 50000-employee database. Without sharding the database, finding the result will take a lot of effort and time. The search pattern will follow a step-by-step approach and need 50,000 transactions to display the details of a single piece of data you are looking for in the database.
Going back to the same example, we now divide the total number of employees into sub-divisions such as freshers, experienced, job-profile, and other sections to make the search easier. For instance, if there are a total of 15000 Software-Developers working in the company, and you want the details of a particular developer, then the database will look only into the 15000 sub-divisions instead of looking through the entire database.
The database on the second scenario looks more organized, simplified, and clean after database sharding. Right? That’s where database sharding comes into the picture. The idea behind database sharding is to simplify the task into smaller divisions to reuse the data in a tech-savvy and efficient way. Technically, database sharding streamlines the searching process and attempts to find the search item from the list in the first go, thus saving time.
Database sharding is a data distribution process and stores a single data set into multiple databases. The purpose of database distribution is to enhance the scalability of applications. Sharding is an excellent way to keep the data safe across different resources. In MongoDB, database sharding is achievable by breaking down big data sets into sub-divided data sets across multiple MongoDB instances.
Attention: MongoDB uses database sharding for deployment support, especially when there are high-volume data sets that are relatively increased throughput operations. It is also important to note that each shard is an independent database, and all the shards consist of a single local database.
Sharded Cluster is a group of MongoDB instances. In simple words, these are a set of nodes that comprise MongoDB deployment. A sharded cluster has three main components:
.png)
On sharding a MongoDB dataset, a shard key is automatically created by default. The shard key can be in the form of an indexed field or indexed compound fields that will be used to distribute the data among the shards. Generally, the “shard key” is used to distribute the MongoDB collection’s documents across all the shards, where the key consists of a single field or multiple fields in every document.
MongoDB divides the range of shard key values into non-overlapping ranges of shard key values, where every range is linked with a chunk. Specifically, MongoDB tries to break down chunks in an even fashion among the different shards present in the cluster.
A shard key can be used to distribute data in the following
The balancer is a process that holds the responsibility of distributing the chunks evenly among the different shards. There is a balance specifier for each cluster that handles the chunk distribution. The balance specifier take care of running the primary job and even distributing chunks across all shards evenly. The process of this type of chunk distribution carried out evenly is popularly known as even chunk distribution.
The fundamental idea behind database sharding is to break complex data into subparts for easy accessibility anytime, anywhere. Check out the advantages of sharding a database:
In database sharding, when data gets distributed across the shards in the cluster, each shard contains a subset of the total data in the cluster. On increasing the data volume, the additional shards grow which leads to expanding the cluster storage capacity.
With an unsharded database, an outage in one database shard has the caliber to deteriorate the entire application and loosen its functionality or even stop. However, with a sharded database, if there is complete unavailability of one or more shard replicas, only a few parts of the application or website are unavailable to some users. However, the other shards continue their operation without any concern.
In MongoDB, the read and write workloads are easily distributed across the shards in the sharded cluster. It allows each shard to process a subset of the cluster operation. Both the read and write performance can be directly scaled horizontally across the cluster by increasing the shard count.
One more reason programmers love database sharding is that it facilitates horizontal scaling (also renowned as scaling out). That means it allows to have parallel backends and carry out tasks simultaneously with no hassle. Whether the focus is on writing or reading operations, scaling out can add a big advantage to enhance the performance and also eliminate complexities.
Whenever you submit a query on an unsharded database, it looks for the searched query in all the rows and columns of the table until it finds the searched query. For low-volume data, it may look insignificant, but it becomes problematic with a high-volume database. Unlike the unsharded database, the sharded database distributes the database into sub-sections where queries have to go to fewer rows, and the results are thus quick and efficient.
A database collection is not always uniform. That means the database will have a mixture of both sharded as well as unsharded collections of data.
Sharded Collection: A collection of data that are broken down in the cluster and are well partitioned is called a sharded collection.
Non-Sharded Collection: The database collection stored on a primary shard (the shard carrying all the un-sharded collection) is known as a non-sharded collection.
For connecting to a sharded cluster, you need to connect to the sharded router using the mongos process. That means you have to join the mongos router with collections (sharded and unsharded) in the sharded cluster. Never make the mistake of connecting to every individual shard for performing read and write operations.
For the distribution of data across the shared clusters, the MongoDB sharding follows the following strategies:
Hash-based database sharding is also known as key-based sharding. Here values are taken from newly registered data into the database and plugged into the hash function. Key-value, or we can call it the hash value, is the shard ID that determines the location of incoming or the registered data. Make sure to keep the values on the hash function in a sequential arrangement so that there is no mismatch of value and the shard.
Ranged sharding involves data distribution based on the ranges of the given shard values. For instance, there is a collection of data storing the inventory details the products will get placed based on the volume of data availability. The biggest drawback of range-based sharding is that it needs a lookup table for reading and write queries, so it may retard the application performance.
Directory-based sharding is a sharding strategy used to maintain a record of shard data. There is a lookup table (also called location service), where it stores the sharded key and tracks all the data entries. Using the shard and key pair, the client engine takes consultation from the location service and then switches to a specific shard to proceed for further work.
Geo-based sharding has similarities to that of range-based database sharding, with the only difference that queries here are geographically based. The data procession is down with a shard that corresponds to the user region under the range of 100 miles. The perfect example is Tinder, a dating app that uses Geo-Based sharding to keep balancing the production load of the geo-shards.
The perks of data sharding may impress you. However, there are many factors that need attention, else you may have to pay the price of data loss or damage. There are a few considerations you must focus on before proceeding with the database sharding:
Before we dive into the MongoDB zone, let us focus on understanding a zone.
A group of shards with a particular set of tags is commonly known as a zone. MongoDB zones available in shading allow distributing chunks based on chunks across shards. All the work, read and write documentation within a zone is done on shards matching the zone. When creating sharded data zones in the sharded clusters, you can link one or more shards in the cluster. Best of all, you can freely associate a shard with any number of zones. Just keep in mind that whenever there is a balanced cluster, migration of chunks in MongoDB takes place such that only those shards associated with the zone get migrated, covered by the zone.
Attention: MongoDB routes reads and writes falling into a zone range only to those shards inside the sharded cluster zone. Shard zones are easily manageable. All the basic operations like creating a zone layer, adding or eliminating shard from the zone, or overviewing existing zones are possible.
A group of transactions belonging to a single shard is known as collations. It consists of a transaction list and a collation header. The collation header comprises information submitted to the main chain, and the transaction list is the sequence of transactions.
Try using the shard Collection command along with the collation: { locale: “simple” } option to shard a collection with a default collation.
It becomes difficult for applications to respond to sudden changes.
From the upgraded MongoDB version 3.6, change streams enable applications to simplify the real-time data changes by leveraging MongoDB functionalities. That means applications can get data accessibility without the cost of tailing the operations log. Change streams come with robust and dynamic features like the total ordering that enables applications to receive changes sequentially as applied to the database.
The organized way of representing the change of state is known as transactions. Ideally, there are four properties called ACID:
MongoDB supports multi-document transactions. The MongoDB version supports 4.0, multi-document transactions on replica sets, whereas the upgraded Mongo version 4.2, supports multi-document transactions on replica sets and the sharded clusters.
Wrapping Up
Database sharding facilitates horizontal scaling and is a more effective way to speed up operational efficiency. Besides, sharding databases simplify the data-management and maintenance procedures. Perhaps, not all databases support database sharding. Worst of all, the sharded database cannot get unsharded. The biggest concern comes when dealing with complex data, especially when there is a data pull from multiple resources. Be careful and attentive, and remember the listed considerations mentioned above. As a gentle reminder, database sharding will only turn to your advantage if you know to use them effectively. Otherwise, if not done the right way, you might corrupt tables and even lead to data loss.
Research & References of What is MongoDB Sharding?|A&C Accounting And Tax Services
Source