{"id":8467,"date":"2024-08-13T07:00:36","date_gmt":"2024-08-13T14:00:36","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=8467"},"modified":"2024-08-13T08:37:42","modified_gmt":"2024-08-13T15:37:42","slug":"mongo-migrations-spark-databricks","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/mongo-migrations-spark-databricks\/","title":{"rendered":"How to migrate MongoDB to Azure Cosmos DB for MongoDB using Spark and Databricks"},"content":{"rendered":"

\"How<\/a><\/p>\n

MongoDB is a popular document database that offers high performance, scalability, and flexibility. Many organizations use MongoDB to store and process large volumes of data for various applications. However, managing and maintaining MongoDB clusters can be challenging and costly, especially as the data grows and the demand increases.<\/span>\u00a0<\/span><\/p>\n

Azure Cosmos DB for MongoDB, particularly the vCore-based model, offers several advantages over traditional MongoDB. It provides a fully managed, globally distributed database service with a 99.99% high availability SLA, ensuring robust performance and reliability. The vCore-based model supports high-capacity vertical and horizontal scaling, making it ideal for workloads with long-running queries, complex aggregation pipelines, and distributed transactions. Additionally, it integrates seamlessly with the Azure ecosystem, allowing developers to leverage Azure\u2019s security features and other services without needing to adapt to new tools. This makes vCore-based Azure Cosmos DB for MongoDB a compelling choice for scalable, secure, and efficient database solutions. By migrating MongoDB to vCore-based Azure Cosmos DB for MongoDB, you can reduce the operational overhead, improve the availability and reliability, and leverage the native Azure integration and features.<\/span>\u00a0<\/span><\/p>\n

There are several methods to migrate MongoDB to Azure Cosmos DB, tailored to your specific needs and preferences. For smaller datasets (less than 10GB), you can use MongoDB\u2019s native tools like\u00a0mongoimport\u00a0and\u00a0mongorestore. These commands are straightforward and effective for basic migrations.<\/span>\u00a0<\/span><\/p>\n

For larger datasets, the Azure Data Studio (ADS) extension<\/a> offers a user-friendly, self-service tool that supports both online and offline migrations. While the ADS extension is suitable for most scenarios, it has some limitations, such as handling complex migration requirements and private endpoint situations.<\/span>\u00a0<\/span><\/p>\n

If you need a flexible solution which also works in private endpoint situations, consider using the Spark-based Mongo Migration tool on Databricks. Additionally, it allows you to control the migration speed and parallelism and customize configuration settings to meet your specific needs.<\/span>\u00a0<\/span><\/p>\n

A new tool for complex and secure migrations<\/span><\/span>\u00a0<\/span><\/h3>\n

The Spark based MongoDB Migration tool is a JAR application that uses the Spark MongoDB Connector and the Azure Cosmos DB Spark Connector to read data from MongoDB and write data to VCore-based Azure Cosmos DB for MongoDB. It can be deployed in your Databricks cluster and virtual network, and run as a Databricks job. You can access the GitHub repository with the necessary binary files, sample configuration files, and detailed instructions on how to use the tool.<\/span>\u00a0<\/span><\/p>\n

\"Image<\/a><\/h6>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

The Spark based MongoDB Migration tool has the following features and benefits:<\/span>\u00a0<\/span><\/p>\n